Correct Template for Usage / Jinja Auto-Select Template issue - how to fix / run model.
#1
by
DavidAU
- opened
To run this model (as well as org version 32B QwQ):
You need to set Chatml template for usage.
There is an issue with Jinja Template ( embedded in the GGUFS) - (not certain if this is LMStudio or source files) ; so you need to manually select "Chatml" template.
Might be same issue in Ollama.
This model has been tested extensively in Lmstudio.
NOTE:
To set the template manually - set POWER user or Devel mode in Lmstudio -> Click the "beaker" -> then AFTER LOADING the model -> Template, lower right - > select: Chatml template.
For more details, examples, system prompts and such see org model card:
https://huggingface.co./DavidAU/QwQ-35B-Eureka-Cubed