This repository contains the GGUF-v3 models (llama.cpp compatible) for
Chinese-Mixtral-Instruct
(chat/instruction model).
Note: When using instruction/chat model, you MUST follow the official prompt template! Example:
chat.sh
Performance
Metric: PPL, lower is better
Quant
Size ↓
PPL
IQ1_S
9.8 GB
9.5782 +/- 0.08909
IQ1_M
10.8 GB
7.4666 +/- 0.06741
IQ2_XXS
12.3 GB
6.3923 +/- 0.05674
IQ2_XS
13.7 GB
6.0606 +/- 0.05834
IQ2_S
14.1 GB
4.7617 +/- 0.04177
IQ2_M
15.5 GB
4.5911 +/- 0.04054
Q2_K
17.3 GB
4.8592 +/- 0.04303
IQ3_XXS
18.3 GB
4.3557 +/- 0.03846
IQ3_XS
19.3 GB
4.3328 +/- 0.03779
IQ3_S
20.4 GB
4.3138 +/- 0.03785
IQ3_M
21.4 GB
4.3024 +/- 0.03775
Q3_K
22.5 GB
4.4334 +/- 0.03937
IQ4_XS
25.1 GB
4.2324 +/- 0.03757
Q4_0
26.4 GB
4.2688 +/- 0.03787
IQ4_NL
26.5 GB
4.2384 +/- 0.03763
Q4_K
28.4 GB
4.2433 +/- 0.03768
Q5_0
32.2 GB
4.2142 +/- 0.03733
Q5_K
33.2 GB
4.2177 +/- 0.03743
Q6_K
38.4 GB
4.2184 +/- 0.03754
Q8_0
49.6 GB
4.2053 +/- 0.03732
F16
93.5 GB
x
Due to the file size limitation, for F16 model, please use
cat
command to concatenate all parts into a single file.
You must concatenate these parts in order.
@article{chinese-mixtral,
title={Rethinking LLM Language Adaptation: A Case Study on Chinese Mixtral},
author={Cui, Yiming and Yao, Xin},
journal={arXiv preprint arXiv:2403.01851},
url={https://arxiv.org/abs/2403.01851},
year={2024}
}
Runs of hfl chinese-mixtral-instruct-gguf on huggingface.co
407
Total runs
-8
24-hour runs
-28
3-day runs
-7
7-day runs
-63
30-day runs
More Information About chinese-mixtral-instruct-gguf huggingface.co Model
More chinese-mixtral-instruct-gguf license Visit here:
chinese-mixtral-instruct-gguf huggingface.co is an AI model on huggingface.co that provides chinese-mixtral-instruct-gguf's model effect (), which can be used instantly with this hfl chinese-mixtral-instruct-gguf model. huggingface.co supports a free trial of the chinese-mixtral-instruct-gguf model, and also provides paid use of the chinese-mixtral-instruct-gguf. Support call chinese-mixtral-instruct-gguf model through api, including Node.js, Python, http.
chinese-mixtral-instruct-gguf huggingface.co is an online trial and call api platform, which integrates chinese-mixtral-instruct-gguf's modeling effects, including api services, and provides a free online trial of chinese-mixtral-instruct-gguf, you can try chinese-mixtral-instruct-gguf online for free by clicking the link below.
hfl chinese-mixtral-instruct-gguf online free url in huggingface.co:
chinese-mixtral-instruct-gguf is an open source model from GitHub that offers a free installation service, and any user can find chinese-mixtral-instruct-gguf on GitHub to install. At the same time, huggingface.co provides the effect of chinese-mixtral-instruct-gguf install, users can directly use chinese-mixtral-instruct-gguf installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
chinese-mixtral-instruct-gguf install url in huggingface.co: