GPTQ-for-StarCoder
Visit GPTQ-for-SantaCoder for instructions on how to use the model weights here. If you want 4-bit weights, visit starcoderbase-GPTQ-4bit-128g.
Results
| StarCoderBase | Bits | group-size | memory(MiB) | wikitext2 | ptb | c4 | stack | checkpoint size(MB) |
|---|---|---|---|---|---|---|---|---|
| FP32 | 32 | - | 10.172 | 15.756 | 12.736 | 1.692 | 59195 | |
| BF16 | 16 | - | 10.173 | 15.765 | 12.745 | 1.692 | 29597 | |
| GPTQ | 8 | 128 | 10.174 | 15.767 | 12.739 | 1.692 | 16163 | |
| GPTQ | 4 | 128 | 10.387 | 16.056 | 13.005 | 1.708 | 8877 |
License
The model is licenses under the CodeML Open RAIL-M v0.1 license. You can find the full license here.
Acknowledgements
Thanks to everyone in BigCode who worked so hard to create these code models.
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.