Hugging Face's logo

DeusImperator
/
Mistral-Small-Instruct-2409_exl2_6.8bpw_rpcal_mk2

Text Generation
Transformers
Safetensors
mistral
conversational
text-generation-inference
Inference Endpoints
exl2
Model card Files Community
Configuration Parsing Warning: In config.json: "quantization_config.bits" must be an integer

Mistral-Small-Instruct-2409 - EXL2 6.8 rpcal_mk2

This is a 8bpw EXL2 quant of mistralai/Mistral-Small-Instruct-2409

This quant was made using exllamav2-0.2.2 with Fullmoon-light dataset for RP.

I tested this quant shortly in some random RPs (including ones over 8k and 16k context) and it seems to work fine.

Prompt Templates

Uses Mistral format.

For more details see original model.

Downloads last month
3
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Collection including DeusImperator/Mistral-Small-Instruct-2409_exl2_6.8bpw_rpcal_mk2