ISTA-DASLab / Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

huggingface.co
Total runs: 27
24-hour runs: -70
7-day runs: -67
30-day runs: -63
Model's Last Updated: May 31 2024
text-generation

Introduction of Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

Model Details of Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

An official quantization of meta-llama/Meta-Llama-3-8B using PV-Tuning on top of AQLM . For this quantization, we used 1 codebook of 16 bits for groups of 16 weights.

The 1x16g16 models require aqlm inference library v1.1.6 or newer:

pip install aqlm[gpu,cpu]>=1.1.6

Note that a large portion of this model are the 16-bit embeddings/logits matrices. You can significantly reduce the model footprint by quantizing these matrices, e.g. using bitsandbytes LLM.int8 or NF4 formats. This does not require additional training.

Model AQLM scheme WikiText 2 PPL Model size, Gb Hub link
meta-llama/Meta-Llama-3-8B 1x16g8 6.99 4.1 Link
meta-llama/Meta-Llama-3-8B (this) 1x16g16 9.43 3.9 Link
meta-llama/Meta-Llama-3-70B 1x16g8 4.57 21.9 Link

To learn more about the inference, as well as the information on how to quantize models yourself, please refer to the official GitHub repo . The original code for PV-Tuning can be found in the AQLM@pv-tuning branch.

Runs of ISTA-DASLab Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 on huggingface.co

27
Total runs
-70
24-hour runs
-69
3-day runs
-67
7-day runs
-63
30-day runs

More Information About Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co Model

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co is an AI model on huggingface.co that provides Meta-Llama-3-8B-AQLM-PV-1Bit-1x16's model effect (), which can be used instantly with this ISTA-DASLab Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 model. huggingface.co supports a free trial of the Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 model, and also provides paid use of the Meta-Llama-3-8B-AQLM-PV-1Bit-1x16. Support call Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 model through api, including Node.js, Python, http.

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co Url

https://huggingface.co/ISTA-DASLab/Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

ISTA-DASLab Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 online free

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co is an online trial and call api platform, which integrates Meta-Llama-3-8B-AQLM-PV-1Bit-1x16's modeling effects, including api services, and provides a free online trial of Meta-Llama-3-8B-AQLM-PV-1Bit-1x16, you can try Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 online for free by clicking the link below.

ISTA-DASLab Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 online free url in huggingface.co:

https://huggingface.co/ISTA-DASLab/Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 install

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 is an open source model from GitHub that offers a free installation service, and any user can find Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 on GitHub to install. At the same time, huggingface.co provides the effect of Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 install, users can directly use Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 install url in huggingface.co:

https://huggingface.co/ISTA-DASLab/Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

Url of Meta-Llama-3-8B-AQLM-PV-1Bit-1x16

Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co Url

Provider of Meta-Llama-3-8B-AQLM-PV-1Bit-1x16 huggingface.co

ISTA-DASLab
ORGANIZATIONS

Other API from ISTA-DASLab