leafspark

/

Llama-3.2-11B-Vision-Instruct-GGUF

like 30

@hoseaho

FROM ./Llama-3.2-11B-Vision-Instruct.Q4_K_M.gguf
FROM ./Llama-3.2-11B-Vision-Instruct-mmproj.f16.gguf

TEMPLATE """{{- range $i, $_ := .Messages }}<|start_header_id|>{{ .Role }}<|end_header_id|>

{{ .Content }}<|eot_id|>
{{- end }}<|start_header_id|>assistant<|end_header_id|>

"""
PARAMETER temperature 0
PARAMETER top_p 0.9

Hello, I'm interested in testing this model out. But I'm using LM Studio and it keeps through the following error.
llama.cpp error: 'error loading model architecture: unknown model architecture: 'mllama''

Is there something I'm missing or needing to edit to get lm studio to recognize this? Sorry for the N00b question.

does the GGUF version of this run on LM Studio? getting same error message

why the model doesn't work on LM Studio and give me this error ( ```
🥲 Failed to load the model

Failed to load model

llama.cpp error: 'error loading model architecture: unknown model architecture: 'mllama''

llama.cpp error: 'error loading model architecture: unknown model architecture: 'mllama''

lm studio does not support mllama runtime at present time, so it won't be able to run vision models...

For others following along. llama.cpp does not support mLlama architecture that this gguf is in. ollama apparently does, but that's a private fork of llama.cpp and there seems to be some debate on whether llama.cpp will ever be updated with support etc. (If you are an engineer, it's open source, so it's open to someone doing a pull request) That said, there is a repo of a gui called clean-ui that does apparently have a method of running this gguf/ llama 3.2 vision model. So, that is an option. I have yet to run clean-ui fully yet. but the internet seems to think this is the way to go. I like lmStudio as my model runner so I'll likely just have to wait it out until someone with deep C experience updates llama.cpp