Spaces:
Runtime error
Issues parsing newlines inside template
Hey there, as mentioned: https://github.com/vllm-project/vllm/pull/1756#issuecomment-1828072165
There seems to be issues with parsing newlines inside the template in my model: https://huggingface.co./Tostino/Inkbot-13B-8k-0.2/blob/main/tokenizer_config.json
You can see the newlines in the template, they do not appear in the image linked in the issue above though.
They are properly parsed by the HF tokenizer.
To properly load the single-line jinja in vLLM, I needed to use the following: https://github.com/vllm-project/vllm/pull/1756/files#diff-38318677b76349044192bf70161371c88fb2818b85279d8fc7f2c041d83a9544R58
Without the above, the newlines were messed up like I see in this application.
Thank you your suggestion.
It is a good feature to have. It will be a feature added soon.
Currently this UI is focusing on getting more users to use the chat_template attributes from huggingface transformers Tokenizer class.
It is recommended to be one-liner in the tokenizer_config.json. Current behaviour is that we will strip off implicit \n to convert it into a one-liner.
Thus, it would seem as if jinja2 template is not render correctly.
Yeah, I think this code is unnecessary: https://huggingface.co./spaces/EmbeddedLLM/chat-template-generation/blob/main/app.py#L12
The template is already a single-line jinja string if it's embedded in the tokenizer as a valid json value. It shouldn't have anything stripped from it after that.