mistralai

/

Mixtral-8x22B-Instruct-v0.1

like 687

Follow

Mistral AI_ 2,573

Textgen webui supports it, I haven't tried the original safetensors but it works with a exl2 and gguf quant I made myself. LLama.cpp works too

how big is your gguf? With it being so huge, i didnt even bother trying.

You can use https://huggingface.co./spaces/NyxKrage/LLM-Model-VRAM-Calculator to check the different sizes for gguf quants

Textgen webui supports it, I haven't tried the original safetensors but it works with a exl2 and gguf quant I made myself. LLama.cpp works too

Textgen webui supports it, I haven't tried the original safetensors but it works with a exl2 and gguf quant I made myself. LLama.cpp works too

How does text generation web ui supports it? Which plugin uses mistral defined tool usage format?

I used 8x7 in oobas text gen 'as is' nothing special was needed.

I used 8x7 in oobas text gen 'as is' nothing special was needed.

How did it use tools?