Adding `safetensors` variant of this model
This is an automated PR created with https://huggingface.co./spaces/safetensors/convert
This new file is equivalent to pytorch_model.bin but safe in the sense that
no arbitrary code can be put into it.
These files also happen to load much faster than their pytorch counterpart:
https://colab.research.google.com/github/huggingface/notebooks/blob/main/safetensors_doc/en/speed.ipynb
The widgets on your model page will run using this model even if this is not merged
making sure the file actually works.
If you find any issues: please report here: https://huggingface.co./spaces/safetensors/convert/discussions
Feel free to ignore this PR.
I tested this with candle, when weights are correct. Example below.
$ cargo run --release --example t5 -- --model-id "google/flan-t5-small" --revision "refs/pr/14" --prompt "summarize: state authorities dispatched emergency crews tuesday to survey the damage after an onslaught of severe weather in mississipi." --temperature 0 --decode
Finished release [optimized] target(s) in 0.17s
Running `target/release/examples/t5 --model-id google/flan-t5-small --revision refs/pr/14 --prompt 'summarize: state authorities dispatched emergency crews tuesday to survey the damage after an onslaught of severe weather in mississipi.' --temperature 0 --decode`
Running on CPU, to run on GPU, build this example with `--features cuda`
state authorities dispatch emergency crews to survey damage
10 tokens generated (55.62 token/s)
Can it please be merged? The base model already has safetensors
Yes! 🤗 The endpoints also works so merging; Thanks for testing!