safetensors format?
#1
by
pdrolet
- opened
Hi! I want to experiment this model but would prefer to have safetensors files instead of gguf. Would it be possible for you to upload to hf this model (Q5KM or Q4KM).
You can't have safetensors format of GGUF, closest you can get is an AWQ quant or exl2, or loading it in 4 bits through transformers/bitandbytes