File size: 867 Bytes
750eb41 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
---
inference: false
license: mit
tags:
- text-generation
- mamba
- long context
---
# DeciMamba Checkpoint
The official checkpoint of DeciMamba-130m, finetuned for Language Modeling over the PG-19 dataset as presented in [DeciMamba: Exploring the Length Extrapolation Potential of Mamba](https://arxiv.org/abs/2406.14528).
See our [Github Repo](https://github.com/assafbk/DeciMamba) for evalution and training scripts.
Bibtex:
```
@misc{benkish2024decimambaexploringlengthextrapolation,
title={DeciMamba: Exploring the Length Extrapolation Potential of Mamba},
author={Assaf Ben-Kish and Itamar Zimerman and Shady Abu-Hussein and Nadav Cohen and Amir Globerson and Lior Wolf and Raja Giryes},
year={2024},
eprint={2406.14528},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2406.14528},
}
``` |