Computer RAM requirements

#12
by SVBilenko - opened

Can you tell me how much computer RAM is needed to deploy this model? My attempt to deploy the model on a computer with 64 gigabytes of RAM ended in failure due to lack of memory.

i added 40gb of swap and it took up 22gb of it, perhaps adding bf16?

BigCode org

yes, try using bf16 of fp16 by specifying torch_dtype when loading the model it will take ~30GB, load_in_8bit might save even more memory (as mentioned here)

would a ggml model run better on cpus?

@pseudon how to add the 40gb swap? am a bit of a noob sorry

how to add the 40gb swap? am a bit of a noob sorry

sudo dd if=/dev/zero of=/.swap bs=16777216 count=2560
sudo mkswap /.swap
sudo swapon -v /.swap
BigCode org

Closing this issue as we added a hardware requirements section here and we have a ggml implementation at starcoder.cpp

loubnabnl changed discussion status to closed

Sign up or log in to comment