ISTA-DASLab / Llama-2-7b-AQLM-2Bit-1x16-hf

huggingface.co
Total runs: 4.3K
24-hour runs: 492
7-day runs: 1.3K
30-day runs: -214
Model's Last Updated: Marzo 12 2024
text-generation

Introduction of Llama-2-7b-AQLM-2Bit-1x16-hf

Model Details of Llama-2-7b-AQLM-2Bit-1x16-hf

Official AQLM quantization of meta-llama/Llama-2-7b-hf .

For this quantization, we used 1 codebook of 16 bits.

Selected evaluation results for this and other models:

Model AQLM scheme WikiText 2 PPL Model size, Gb Hub link
Llama-2-7b (THIS) 1x16 5.92 2.4 Link
Llama-2-7b 2x8 6.69 2.2 Link
Llama-2-7b 8x8 6.61 2.2 Link
Llama-2-13b 1x16 5.22 4.1 Link
Llama-2-70b 1x16 3.83 18.8 Link
Llama-2-70b 2x8 4.21 18.2 Link
Mixtral-8x7b 1x16 3.35 12.6 Link
Mixtral-8x7b-Instruct 1x16 - 12.6 Link

UPD (20.02.2024). We applied global finetuning on top of quantized model and improved results compared to first revision.

To learn more about the inference, as well as the information on how to quantize models yourself, please refer to the official GitHub repo .

Runs of ISTA-DASLab Llama-2-7b-AQLM-2Bit-1x16-hf on huggingface.co

4.3K
Total runs
492
24-hour runs
764
3-day runs
1.3K
7-day runs
-214
30-day runs

More Information About Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co Model

Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co

Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co is an AI model on huggingface.co that provides Llama-2-7b-AQLM-2Bit-1x16-hf's model effect (), which can be used instantly with this ISTA-DASLab Llama-2-7b-AQLM-2Bit-1x16-hf model. huggingface.co supports a free trial of the Llama-2-7b-AQLM-2Bit-1x16-hf model, and also provides paid use of the Llama-2-7b-AQLM-2Bit-1x16-hf. Support call Llama-2-7b-AQLM-2Bit-1x16-hf model through api, including Node.js, Python, http.

Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co Url

https://huggingface.co/ISTA-DASLab/Llama-2-7b-AQLM-2Bit-1x16-hf

ISTA-DASLab Llama-2-7b-AQLM-2Bit-1x16-hf online free

Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co is an online trial and call api platform, which integrates Llama-2-7b-AQLM-2Bit-1x16-hf's modeling effects, including api services, and provides a free online trial of Llama-2-7b-AQLM-2Bit-1x16-hf, you can try Llama-2-7b-AQLM-2Bit-1x16-hf online for free by clicking the link below.

ISTA-DASLab Llama-2-7b-AQLM-2Bit-1x16-hf online free url in huggingface.co:

https://huggingface.co/ISTA-DASLab/Llama-2-7b-AQLM-2Bit-1x16-hf

Llama-2-7b-AQLM-2Bit-1x16-hf install

Llama-2-7b-AQLM-2Bit-1x16-hf is an open source model from GitHub that offers a free installation service, and any user can find Llama-2-7b-AQLM-2Bit-1x16-hf on GitHub to install. At the same time, huggingface.co provides the effect of Llama-2-7b-AQLM-2Bit-1x16-hf install, users can directly use Llama-2-7b-AQLM-2Bit-1x16-hf installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Llama-2-7b-AQLM-2Bit-1x16-hf install url in huggingface.co:

https://huggingface.co/ISTA-DASLab/Llama-2-7b-AQLM-2Bit-1x16-hf

Url of Llama-2-7b-AQLM-2Bit-1x16-hf

Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co Url

Provider of Llama-2-7b-AQLM-2Bit-1x16-hf huggingface.co

ISTA-DASLab
ORGANIZATIONS

Other API from ISTA-DASLab