Why does this model have no biases?
#48
by
Inoob
- opened
I used this code to extract the weight and biases of the model:
from transformers import AutoModelForCausalLM
model_name = "./Llama-3.2-3b-Instruct"
model = AutoModelForCausalLM.from_pretrained(model_name)
model_weights = model.state_dict()
weights = {}
biases = {}
for key, value in model_weights.items():
if 'weight' in key:
weights[key] = value
else:
biases[key] = value
print("Weights:", weights)
print("Biases:", biases)
However, the biases dict is empty.
Is this intentional to not have biases?