--- library_name: transformers license: mit base_model: gpt2 tags: - generated_from_trainer model-index: - name: tfa_output_2025_m02_d02_t23h_29m_10s results: [] --- # tfa_output_2025_m02_d02_t23h_29m_10s This model is a fine-tuned version of [gpt2](https://huggingface.co./gpt2) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.5286 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-06 - train_batch_size: 2 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 8 - optimizer: Use OptimizerNames.PAGED_ADAMW with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: constant_with_warmup - lr_scheduler_warmup_ratio: 0.05 - num_epochs: 100 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-------:|:----:|:---------------:| | No log | 0 | 0 | 3.1242 | | 3.6266 | 0.6667 | 1 | 3.1242 | | 1.7421 | 1.0 | 2 | 3.1070 | | 3.5691 | 1.6667 | 3 | 3.0863 | | 1.8145 | 2.0 | 4 | 3.0757 | | 3.5803 | 2.6667 | 5 | 3.0580 | | 1.7267 | 3.0 | 6 | 3.0413 | | 3.5001 | 3.6667 | 7 | 3.0321 | | 1.756 | 4.0 | 8 | 3.0203 | | 3.5043 | 4.6667 | 9 | 3.0064 | | 1.6752 | 5.0 | 10 | 2.9966 | | 3.449 | 5.6667 | 11 | 2.9887 | | 1.7513 | 6.0 | 12 | 2.9786 | | 3.4134 | 6.6667 | 13 | 2.9696 | | 1.7109 | 7.0 | 14 | 2.9604 | | 3.4382 | 7.6667 | 15 | 2.9518 | | 1.6749 | 8.0 | 16 | 2.9431 | | 3.3344 | 8.6667 | 17 | 2.9329 | | 1.7577 | 9.0 | 18 | 2.9260 | | 3.3458 | 9.6667 | 19 | 2.9184 | | 1.6737 | 10.0 | 20 | 2.9092 | | 3.4422 | 10.6667 | 21 | 2.8993 | | 1.5953 | 11.0 | 22 | 2.8892 | | 3.3445 | 11.6667 | 23 | 2.8819 | | 1.631 | 12.0 | 24 | 2.8721 | | 3.3118 | 12.6667 | 25 | 2.8626 | | 1.6067 | 13.0 | 26 | 2.8547 | | 3.2708 | 13.6667 | 27 | 2.8468 | | 1.619 | 14.0 | 28 | 2.8393 | | 3.3062 | 14.6667 | 29 | 2.8312 | | 1.6059 | 15.0 | 30 | 2.8240 | | 3.2822 | 15.6667 | 31 | 2.8154 | | 1.607 | 16.0 | 32 | 2.8071 | | 3.2488 | 16.6667 | 33 | 2.7995 | | 1.6078 | 17.0 | 34 | 2.7929 | | 3.2022 | 17.6667 | 35 | 2.7875 | | 1.5858 | 18.0 | 36 | 2.7801 | | 3.1965 | 18.6667 | 37 | 2.7720 | | 1.5955 | 19.0 | 38 | 2.7656 | | 3.1891 | 19.6667 | 39 | 2.7606 | | 1.5755 | 20.0 | 40 | 2.7527 | | 3.0662 | 20.6667 | 41 | 2.7467 | | 1.6257 | 21.0 | 42 | 2.7411 | | 3.1364 | 21.6667 | 43 | 2.7350 | | 1.5211 | 22.0 | 44 | 2.7288 | | 3.157 | 22.6667 | 45 | 2.7235 | | 1.4631 | 23.0 | 46 | 2.7158 | | 3.1188 | 23.6667 | 47 | 2.7099 | | 1.4971 | 24.0 | 48 | 2.7063 | | 2.98 | 24.6667 | 49 | 2.7008 | | 1.634 | 25.0 | 50 | 2.6942 | | 3.016 | 25.6667 | 51 | 2.6879 | | 1.5771 | 26.0 | 52 | 2.6843 | | 3.0495 | 26.6667 | 53 | 2.6808 | | 1.4922 | 27.0 | 54 | 2.6750 | | 2.9655 | 27.6667 | 55 | 2.6711 | | 1.6188 | 28.0 | 56 | 2.6655 | | 3.0155 | 28.6667 | 57 | 2.6611 | | 1.4867 | 29.0 | 58 | 2.6567 | | 3.0117 | 29.6667 | 59 | 2.6515 | | 1.5069 | 30.0 | 60 | 2.6470 | | 3.0118 | 30.6667 | 61 | 2.6441 | | 1.4577 | 31.0 | 62 | 2.6375 | | 3.0372 | 31.6667 | 63 | 2.6350 | | 1.411 | 32.0 | 64 | 2.6295 | | 2.9611 | 32.6667 | 65 | 2.6267 | | 1.4289 | 33.0 | 66 | 2.6246 | | 2.9595 | 33.6667 | 67 | 2.6207 | | 1.437 | 34.0 | 68 | 2.6166 | | 2.9483 | 34.6667 | 69 | 2.6127 | | 1.4469 | 35.0 | 70 | 2.6114 | | 2.9291 | 35.6667 | 71 | 2.6067 | | 1.411 | 36.0 | 72 | 2.6021 | | 2.9534 | 36.6667 | 73 | 2.5988 | | 1.4295 | 37.0 | 74 | 2.5958 | | 2.9181 | 37.6667 | 75 | 2.5929 | | 1.4138 | 38.0 | 76 | 2.5891 | | 2.9133 | 38.6667 | 77 | 2.5855 | | 1.4172 | 39.0 | 78 | 2.5818 | | 2.8655 | 39.6667 | 79 | 2.5809 | | 1.3988 | 40.0 | 80 | 2.5780 | | 2.929 | 40.6667 | 81 | 2.5750 | | 1.3445 | 41.0 | 82 | 2.5712 | | 2.8141 | 41.6667 | 83 | 2.5696 | | 1.503 | 42.0 | 84 | 2.5668 | | 2.8483 | 42.6667 | 85 | 2.5636 | | 1.4017 | 43.0 | 86 | 2.5622 | | 2.8643 | 43.6667 | 87 | 2.5575 | | 1.3592 | 44.0 | 88 | 2.5553 | | 2.8332 | 44.6667 | 89 | 2.5537 | | 1.3675 | 45.0 | 90 | 2.5503 | | 2.742 | 45.6667 | 91 | 2.5478 | | 1.5006 | 46.0 | 92 | 2.5453 | | 2.7909 | 46.6667 | 93 | 2.5436 | | 1.4314 | 47.0 | 94 | 2.5406 | | 2.7937 | 47.6667 | 95 | 2.5382 | | 1.3617 | 48.0 | 96 | 2.5359 | | 2.8299 | 48.6667 | 97 | 2.5343 | | 1.3295 | 49.0 | 98 | 2.5306 | | 2.7586 | 49.6667 | 99 | 2.5297 | | 1.4496 | 50.0 | 100 | 2.5286 | ### Framework versions - Transformers 4.48.0 - Pytorch 2.5.1+cu124 - Datasets 3.2.0 - Tokenizers 0.21.0