TenaliAI-FinTech-v1

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8315

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
2.3436	1.0	3809	1.9815
1.2453	2.0	7618	1.1621
0.9853	3.0	11427	0.9375
0.8483	4.0	15236	0.9018
0.8195	5.0	19045	0.8538
0.7579	6.0	22854	0.8540
0.7123	7.0	26663	0.8397
0.7064	8.0	30472	0.8405
0.6987	9.0	34281	0.8315
0.676	10.0	38090	0.8530
0.6566	11.0	41899	0.8504
0.6411	12.0	45708	0.8501
0.6432	13.0	49517	0.8545
0.6483	14.0	53326	0.8624
0.6447	15.0	57135	0.8635
0.6077	16.0	60944	0.8782
0.6208	17.0	64753	0.8925
0.624	18.0	68562	0.8834
0.6298	19.0	72371	0.9000
0.6488	20.0	76180	0.8922
0.6019	21.0	79989	0.9025
0.6412	22.0	83798	0.8963
0.6078	23.0	87607	0.9045
0.6163	24.0	91416	0.8898
0.6275	25.0	95225	0.9036

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1