Wasn't this model supposed to take around 3-4 GB given it's only 1B parameters?
#21
by
ahsannawazch
- opened
I tried to load this model in colab (free tier) and it has started downloading this pytorch_model.bin file having sized 28.9GB?
Where Am I going wrong here?
I also wonder
I also wonder
I am now using qwen2 2B vl. working okay for now
This is because MolmoE is actually 7B model, but only activates 1B when inference.
Look for Mixture-of-Experts(MoE)