quantized model to 4 but or 8 bit
#1
by
thefcraft
- opened
Can you please quantized this model to 4 but or 8 bit i don't have access to high memory
I have Collab free version...
Other people have uploaded the quantized stuff already. There's a 4-bit quantization in the gptq format by anon82, and a 4-bit quant in the ggml format done on by me
Ya bro i already see it i forgot to close this but, thanks 👍
https://huggingface.co./eachadea/ggml-vicuna-13b-4bit/tree/main
https://huggingface.co./anon8231489123/vicuna-13b-GPTQ-4bit-128g/tree/main
thefcraft
changed discussion status to
closed