Flan

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.1764
• Validation Loss: 0.9237
• Epoch: 24

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 2e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
1.6824	1.3619	0
1.5733	1.3030	1
1.5199	1.2730	2
1.4833	1.2379	3
1.4591	1.2201	4
1.4278	1.1945	5
1.4101	1.1755	6
1.3870	1.1561	7
1.3740	1.1366	8
1.3559	1.1163	9
1.3420	1.1007	10
1.3265	1.0871	11
1.3082	1.0708	12
1.2984	1.0586	13
1.2832	1.0429	14
1.2724	1.0305	15
1.2634	1.0170	16
1.2486	1.0090	17
1.2375	0.9901	18
1.2259	0.9760	19
1.2128	0.9659	20
1.2038	0.9556	21
1.1965	0.9446	22
1.1833	0.9369	23
1.1764	0.9237	24

Framework versions

• Transformers 4.32.0
• TensorFlow 2.13.0
• Datasets 2.14.4
• Tokenizers 0.12.1