This repository contains GGUF-v3 version (llama.cpp compatible) of
Chinese-Alpaca-2-1.3B-RLHF
, which is tuned on Chinese-Alpaca-2-1.3B with RLHF using DeepSpeed-Chat.
The optimal context length is 1K for this model. Specify
-c 1024
when using with llama.cpp
Performance
Metric: PPL, lower is better
Quant
original
imatrix (
-im
)
Q2_K
20.1066 +/- 0.29236
18.8209 +/- 0.27561
Q3_K
16.9214 +/- 0.26133
16.5729 +/- 0.25706
Q4_0
15.8056 +/- 0.23749
-
Q4_K
16.1579 +/- 0.25064
15.7746 +/- 0.24476
Q5_0
15.4528 +/- 0.23911
-
Q5_K
15.3198 +/- 0.23627
15.4791 +/- 0.23959
Q6_K
15.3718 +/- 0.23764
15.2572 +/- 0.23549
Q8_0
15.3302 +/- 0.23727
-
F16
15.3291 +/- 0.23728
-
The model with
-im
suffix is generated with important matrix, which has generally better performance (not always though).
chinese-alpaca-2-1.3b-rlhf-gguf huggingface.co is an AI model on huggingface.co that provides chinese-alpaca-2-1.3b-rlhf-gguf's model effect (), which can be used instantly with this hfl chinese-alpaca-2-1.3b-rlhf-gguf model. huggingface.co supports a free trial of the chinese-alpaca-2-1.3b-rlhf-gguf model, and also provides paid use of the chinese-alpaca-2-1.3b-rlhf-gguf. Support call chinese-alpaca-2-1.3b-rlhf-gguf model through api, including Node.js, Python, http.
chinese-alpaca-2-1.3b-rlhf-gguf huggingface.co is an online trial and call api platform, which integrates chinese-alpaca-2-1.3b-rlhf-gguf's modeling effects, including api services, and provides a free online trial of chinese-alpaca-2-1.3b-rlhf-gguf, you can try chinese-alpaca-2-1.3b-rlhf-gguf online for free by clicking the link below.
hfl chinese-alpaca-2-1.3b-rlhf-gguf online free url in huggingface.co:
chinese-alpaca-2-1.3b-rlhf-gguf is an open source model from GitHub that offers a free installation service, and any user can find chinese-alpaca-2-1.3b-rlhf-gguf on GitHub to install. At the same time, huggingface.co provides the effect of chinese-alpaca-2-1.3b-rlhf-gguf install, users can directly use chinese-alpaca-2-1.3b-rlhf-gguf installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
chinese-alpaca-2-1.3b-rlhf-gguf install url in huggingface.co: