---
license: bigscience-bloom-rail-1.0
datasets:
- swap-uniba/itwiki-march-2024
language:
- it
tags:
- bloom
- italian
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

The model is obtained by performing language adaptation on the original bloom-1b7 model.
In detail, we continued the pre-training on Italian-specific data without adaptation of the vocabulary.
We use about 2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024). The model is trained for one epoch using LoRA and SFT.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Developed by:** SWAP Research Group, Department of Computer Science, University of Bari Aldo Moro
- **Model type:** BLOOM
- **Language(s) (NLP):** Italian
- **License:** bigscience-bloom-rail-1.0
- **Finetuned from model [optional]:** bloom-1b7

## Training Details

### Training Data

2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024).

### Training Procedure

LoRA and SFT.

#### Training Hyperparameters

- **Training regime:** fp16

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

**APA:**

## Model Card Authors [optional]

Pierpaolo Basile, University of Bari Aldo Moro, Italy.

## Model Card Contact

Pierpaolo Basile, University of Bari Aldo Moro, Italy.