model_type "llama"
#1
by
Phando
- opened
Thanks for sharing all the checkpoints.
I would like to discuss why the model_type is named "llama" in the model configuration. This seems a little bit tricky especially when people want to do some auto mapping in their codebase when using llama and openmoe at the same time.
Thank you for pointing out this!😃 The inclusion of “llama” in the model configuration was unintentional. During implementation, we adapted some code from LLama, such as LLamaConfig, which resulted in the presence of some config items that we didn’t have the chance to remove. Feel free to make the necessary adjustments.