Infinity conversation, sould eos be <|eot_id|> ?
#1
by
Komposter43
- opened
Model doesn't stop in vllm chat conversation, may be eos should be '<|eot_id|>' ?
put both eos tokens as stopping criteria i`d suggest.
See related merged PR on the meta-llama repo https://huggingface.co./meta-llama/Meta-Llama-3-8B-Instruct/discussions/4. That change to the tokenizer basically just needs to be done here as well
Fixed!
casperhansen
changed discussion status to
closed