metadata
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- ccdv/arxiv-summarization
metrics:
- rouge
model-index:
- name: arxiv-summarization-t5-base-2022-09-21
results:
- task:
name: Summarization
type: summarization
dataset:
name: ccdv/arxiv-summarization
type: ccdv/arxiv-summarization
config: section
split: train
args: section
metrics:
- name: Rouge1
type: rouge
value: 40.6781
arxiv-summarization-t5-base-2022-09-21
This model is a fine-tuned version of t5-base on the ccdv/arxiv-summarization dataset. It achieves the following results on the evaluation set:
- Loss: 1.8650
- Rouge1: 40.6781
- Rouge2: 14.7167
- Rougel: 26.6375
- Rougelsum: 35.5959
- Gen Len: 117.1969
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 2.3291 | 0.05 | 10000 | 2.1906 | 18.6571 | 7.1341 | 14.8347 | 16.9545 | 19.0 |
| 2.2454 | 0.1 | 20000 | 2.1549 | 18.5037 | 7.1908 | 14.7141 | 16.8233 | 18.9997 |
| 2.2107 | 0.15 | 30000 | 2.1013 | 18.7638 | 7.326 | 14.9437 | 17.072 | 19.0 |
| 2.1486 | 0.2 | 40000 | 2.0845 | 18.6879 | 7.2441 | 14.8835 | 16.983 | 19.0 |
| 2.158 | 0.25 | 50000 | 2.0699 | 18.8314 | 7.3712 | 15.0166 | 17.1215 | 19.0 |
| 2.1476 | 0.3 | 60000 | 2.0424 | 18.9783 | 7.4138 | 15.1121 | 17.2778 | 18.9981 |
| 2.1164 | 0.34 | 70000 | 2.0349 | 18.9257 | 7.4649 | 15.0335 | 17.1819 | 19.0 |
| 2.079 | 0.39 | 80000 | 2.0208 | 18.643 | 7.4096 | 14.8927 | 16.9786 | 18.9994 |
| 2.101 | 0.44 | 90000 | 2.0113 | 19.3881 | 7.7012 | 15.3981 | 17.6516 | 19.0 |
| 2.0576 | 0.49 | 100000 | 2.0022 | 18.9985 | 7.542 | 15.1157 | 17.2972 | 18.9992 |
| 2.0983 | 0.54 | 110000 | 1.9941 | 18.7691 | 7.4625 | 15.0256 | 17.1146 | 19.0 |
| 2.053 | 0.59 | 120000 | 1.9855 | 19.002 | 7.5602 | 15.1497 | 17.2963 | 19.0 |
| 2.0434 | 0.64 | 130000 | 1.9786 | 19.2385 | 7.6533 | 15.3094 | 17.5439 | 18.9994 |
| 2.0354 | 0.69 | 140000 | 1.9746 | 19.184 | 7.7307 | 15.2897 | 17.491 | 18.9992 |
| 2.0347 | 0.74 | 150000 | 1.9639 | 19.2408 | 7.693 | 15.3357 | 17.5297 | 19.0 |
| 2.0236 | 0.79 | 160000 | 1.9590 | 19.0781 | 7.6256 | 15.1932 | 17.3486 | 18.9998 |
| 2.0187 | 0.84 | 170000 | 1.9532 | 19.0343 | 7.6792 | 15.1884 | 17.3519 | 19.0 |
| 1.9939 | 0.89 | 180000 | 1.9485 | 18.8247 | 7.5005 | 15.0246 | 17.1485 | 18.9998 |
| 1.9961 | 0.94 | 190000 | 1.9504 | 19.0695 | 7.6559 | 15.2139 | 17.3814 | 19.0 |
| 2.0197 | 0.99 | 200000 | 1.9399 | 19.2821 | 7.6685 | 15.3029 | 17.5374 | 18.9988 |
| 1.9457 | 1.03 | 210000 | 1.9350 | 19.053 | 7.6502 | 15.2123 | 17.3793 | 19.0 |
| 1.9552 | 1.08 | 220000 | 1.9317 | 19.1878 | 7.7235 | 15.3272 | 17.5252 | 18.9998 |
| 1.9772 | 1.13 | 230000 | 1.9305 | 19.0855 | 7.6303 | 15.1943 | 17.3942 | 18.9997 |
| 1.9171 | 1.18 | 240000 | 1.9291 | 19.0711 | 7.6437 | 15.2175 | 17.3893 | 18.9995 |
| 1.9393 | 1.23 | 250000 | 1.9230 | 19.276 | 7.725 | 15.3826 | 17.586 | 18.9995 |
| 1.9295 | 1.28 | 260000 | 1.9197 | 19.2999 | 7.7958 | 15.3961 | 17.6056 | 18.9975 |
| 1.9725 | 1.33 | 270000 | 1.9173 | 19.2958 | 7.7121 | 15.3659 | 17.584 | 19.0 |
| 1.9668 | 1.38 | 280000 | 1.9129 | 19.089 | 7.6846 | 15.2395 | 17.3879 | 18.9998 |
| 1.941 | 1.43 | 290000 | 1.9132 | 19.2127 | 7.7336 | 15.311 | 17.4742 | 18.9995 |
| 1.9427 | 1.48 | 300000 | 1.9108 | 19.217 | 7.7591 | 15.334 | 17.53 | 18.9998 |
| 1.9521 | 1.53 | 310000 | 1.9041 | 19.1285 | 7.6736 | 15.2625 | 17.458 | 19.0 |
| 1.9352 | 1.58 | 320000 | 1.9041 | 19.1656 | 7.723 | 15.3035 | 17.4818 | 18.9991 |
| 1.9342 | 1.63 | 330000 | 1.9004 | 19.2573 | 7.7766 | 15.3558 | 17.5382 | 19.0 |
| 1.9631 | 1.68 | 340000 | 1.8978 | 19.236 | 7.7584 | 15.3408 | 17.4993 | 18.9998 |
| 1.8987 | 1.72 | 350000 | 1.8968 | 19.1716 | 7.7231 | 15.2836 | 17.4655 | 18.9997 |
| 1.9433 | 1.77 | 360000 | 1.8924 | 19.2644 | 7.8294 | 15.4018 | 17.5808 | 18.9998 |
| 1.9159 | 1.82 | 370000 | 1.8912 | 19.1833 | 7.8267 | 15.3175 | 17.4918 | 18.9995 |
| 1.9516 | 1.87 | 380000 | 1.8856 | 19.3077 | 7.7432 | 15.3723 | 17.6115 | 19.0 |
| 1.9218 | 1.92 | 390000 | 1.8880 | 19.2668 | 7.8231 | 15.3834 | 17.5701 | 18.9994 |
| 1.9159 | 1.97 | 400000 | 1.8860 | 19.2224 | 7.7903 | 15.3488 | 17.4992 | 18.9997 |
| 1.8741 | 2.02 | 410000 | 1.8854 | 19.2572 | 7.741 | 15.3405 | 17.5351 | 19.0 |
| 1.8668 | 2.07 | 420000 | 1.8854 | 19.3658 | 7.8593 | 15.4418 | 17.656 | 18.9995 |
| 1.8638 | 2.12 | 430000 | 1.8831 | 19.305 | 7.8218 | 15.3843 | 17.5861 | 18.9997 |
| 1.8334 | 2.17 | 440000 | 1.8817 | 19.3269 | 7.8249 | 15.4231 | 17.5958 | 18.9994 |
| 1.8893 | 2.22 | 450000 | 1.8803 | 19.2949 | 7.7885 | 15.3947 | 17.585 | 18.9997 |
| 1.8929 | 2.27 | 460000 | 1.8783 | 19.291 | 7.8346 | 15.428 | 17.5797 | 18.9997 |
| 1.861 | 2.32 | 470000 | 1.8766 | 19.4284 | 7.8832 | 15.4746 | 17.6946 | 18.9997 |
| 1.8719 | 2.37 | 480000 | 1.8751 | 19.1525 | 7.7641 | 15.3348 | 17.47 | 18.9998 |
| 1.8889 | 2.41 | 490000 | 1.8742 | 19.1743 | 7.768 | 15.3292 | 17.4665 | 18.9998 |
| 1.8834 | 2.46 | 500000 | 1.8723 | 19.3069 | 7.7935 | 15.3987 | 17.5913 | 18.9998 |
| 1.8564 | 2.51 | 510000 | 1.8695 | 19.3217 | 7.8292 | 15.4063 | 17.6081 | 19.0 |
| 1.8706 | 2.56 | 520000 | 1.8697 | 19.294 | 7.8217 | 15.3964 | 17.581 | 18.9998 |
| 1.883 | 2.61 | 530000 | 1.8703 | 19.2784 | 7.8634 | 15.404 | 17.5942 | 18.9995 |
| 1.8622 | 2.66 | 540000 | 1.8677 | 19.3165 | 7.8378 | 15.4259 | 17.6064 | 18.9988 |
| 1.8781 | 2.71 | 550000 | 1.8676 | 19.3237 | 7.7954 | 15.3995 | 17.6008 | 19.0 |
| 1.8793 | 2.76 | 560000 | 1.8685 | 19.2141 | 7.7605 | 15.3345 | 17.5268 | 18.9997 |
| 1.8795 | 2.81 | 570000 | 1.8675 | 19.2694 | 7.8082 | 15.3996 | 17.5831 | 19.0 |
| 1.8425 | 2.86 | 580000 | 1.8659 | 19.2886 | 7.7987 | 15.4005 | 17.5859 | 18.9997 |
| 1.8605 | 2.91 | 590000 | 1.8650 | 19.2778 | 7.7934 | 15.3931 | 17.5809 | 18.9997 |
| 1.8448 | 2.96 | 600000 | 1.8655 | 19.2884 | 7.8087 | 15.4025 | 17.5856 | 19.0 |
Framework versions
- Transformers 4.23.0.dev0
- Pytorch 1.12.0
- Datasets 2.5.1
- Tokenizers 0.13.0