Q6 giving garbage
Tried Q6 in latest LM Studio with Llama-3 preset, tons of issues, can't answer anything without coming across as drunk, a lot of responses made no sense, often wouldn't stop either.
All other 10+ AI I have installed run still fine. Bartowski had similar issues with his GGUF, he wrote "This model may get strange behaviour due to several of its tokens being labelled as "special", and so most tools will not properly detect them. This is most noticeable with the stop token. ". It sounds a lot as what I read on Reddit too. It seems to me that you have to do some fixing too.
This model is a base model. It is not finetuned for instructions or chats. What you need (and for which the 'llama-3 preset' is correct) is this:
https://huggingface.co./QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
Thank you for your response, I will download yours too.