Edit Models filters

Inference status

Misc

AutoTrain Compatible

Inference Endpoints

text-generation-inference

8-bit precision

4-bit precision

Misc with no match

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

130

Full-text search

Active filters: vllm

mradermacher/nemotron-3-8b-chat-4k-sft-hf-GGUF

Updated Aug 16 • 3.87k • 3

neuralmagic/gemma-2-9b-it-quantized.w8a8

Text Generation • Updated Oct 9 • 201 • 2

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated Oct 10 • 907 • 2

mistral-community/pixtral-12b-240910

Image-Text-to-Text • Updated Oct 1 • 383

neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 61.2k • 11

neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.35k • 2

neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.88k • 2

neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 13.3k • 3

neuralmagic/Phi-3.5-mini-instruct-FP8-KV

Text Generation • Updated Oct 1 • 336 • 1

neuralmagic/pixtral-12b-FP8-dynamic

Text Generation • Updated 12 days ago • 3.26k • 1

mlx-community/Ministral-8B-Instruct-2410-bf16

Updated 27 days ago • 545 • 2

mlx-community/Ministral-8B-Instruct-2410-4bit

Updated 27 days ago • 248 • 4

mlx-community/Ministral-8B-Instruct-2410-8bit

Updated 27 days ago • 188 • 1

mistralai/Pixtral-12B-Base-2409

Updated 14 days ago • 65

aashish1904/Ministral-8B-Instruct-2410-HF-Q4_K_M-GGUF

Updated 24 days ago • 97 • 1

aashish1904/Ministral-8B-Instruct-2410-HF-Q2_K-GGUF

Updated 24 days ago • 31 • 2

mysticbeing/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-DYNAMIC

Text Generation • Updated 7 days ago • 243 • 4

Inferless/deciLM-7B-GPTQ

Text Generation • Updated Jan 4 • 8 • 1

Inferless/SOLAR-10.7B-Instruct-v1.0-GPTQ

Text Generation • Updated Jan 4 • 11 • 2

Inferless/Mixtral-8x7B-v0.1-int8-GPTQ

Text Generation • Updated Jan 25 • 47 • 2

neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV

Text Generation • Updated Jun 19 • 14.4k • 6

neuralmagic/Qwen2-0.5B-Instruct-FP8

Text Generation • Updated Jul 18 • 1.21k • 2

neuralmagic/Qwen2-1.5B-Instruct-FP8

Text Generation • Updated Jul 18 • 33

neuralmagic/Qwen2-7B-Instruct-FP8

Text Generation • Updated Jul 18 • 1.38k • 1

nm-testing/SparseLlama-3-8B-pruned_50.2of4-FP8

Text Generation • Updated Jun 25 • 16

FlorianJc/Hermes-2-Pro-Mistral-7B-vllm-fp8

Text Generation • Updated Jul 17 • 8

FlorianJc/openchat-3.6-8b-20240522-vllm-fp8

Text Generation • Updated Jul 17 • 20

FlorianJc/Llama3-ChatQA-1.5-8B-vllm-fp8

Text Generation • Updated Jul 17 • 16

Rallio67/magnum-72B-FP8

Text Generation • Updated Jun 26 • 12

neuralmagic/Meta-Llama-3-70B-Instruct-FP8-KV

Text Generation • Updated Jun 26 • 845 • 2