YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co./docs/hub/model-cards#model-card-metadata)

This is a high quality quantization of Yi-VL-6B and of the visual transformer. Q5_K is almost equal to fp16 in inference and Q6_K is basically the same (not as reliably tested for visual transformers but I assume it's equal to language models) You currently need to apply this PR to make it work: https://github.com/ggerganov/llama.cpp/pull/5093 - this adds the additional normalization steps into the projection

I do not like this model, it's hallucinating more than anything else based on llava.

Downloads last month: 134

GGUF

Model size

6.06B params

Architecture

llama

5-bit

6-bit

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.