mobiuslabsgmbh
/

Mixtral-8x7B-v0.1-hf-attn-4bit-moe-2bit-HQQ

Text Generation

Mixture of Experts

Model card Files Files and versions Community

mobicham commited on Dec 18, 2023

Commit

88e72ff

·

1 Parent(s): c30159c

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -9,7 +9,7 @@ This is a version of the Mixtral-8x7B-v0.1 model (https://huggingface.co/mistral
 More specifically, the attention layers are quantized to 4-bit and the experts are quantized to 2-bit. This simple change yields a huge improvement in perplexity vs the all 2-bit model (4.69 vs. 5.90) for a slight increase in model size (18.2GB vs. 18GB).
-This idea was suggest by Artem Eliseev (@lavawolfiee) and Denis Mazur (@dvmazur) [in this Github discussion](hhttps://github.com/mobiusml/hqq/issues/2).
 ### Basic Usage
 To run the model, install the HQQ library from https://github.com/mobiusml/hqq and use it as follows:

 More specifically, the attention layers are quantized to 4-bit and the experts are quantized to 2-bit. This simple change yields a huge improvement in perplexity vs the all 2-bit model (4.69 vs. 5.90) for a slight increase in model size (18.2GB vs. 18GB).
+This idea was suggest by Artem Eliseev (@lavawolfiee) and Denis Mazur (@dvmazur) [in this Github discussion](https://github.com/mobiusml/hqq/issues/2).
 ### Basic Usage
 To run the model, install the HQQ library from https://github.com/mobiusml/hqq and use it as follows: