facebook

/

MobileLLM-125M

like 90

Follow

AI at Meta 3.88k

Fixes https://github.com/huggingface/transformers/issues/34759

Proposed solution :
The safetensors file had the embedding layer missing.
I loaded the model from the existing weights file and saved it as safetensors

You can test the functionality of the updated safetensors with the following script

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("facebook/MobileLLM-125M", use_fast=False)

mobilellm_old = AutoModelForCausalLM.from_pretrained("facebook/MobileLLM-125M",trust_remote_code=True, use_safetensors=True)
mobilellm = AutoModelForCausalLM.from_pretrained("/Users/mayankagarwal/Documents/OSS/codebases/MobileLLM-125M",trust_remote_code=True, use_safetensors=True)

input = tokenizer("Hello word!", return_tensors="pt")


output_old = mobilellm_old.generate(**input)
decoded = tokenizer.decode(output_old[0], skip_special_tokens=True)
print("Old decoded output:", decoded)


output = mobilellm.generate(**input)
decoded = tokenizer.decode(output[0], skip_special_tokens=True)
print("Updated decoded output:", decoded)

Here's a screenshot of the output