---
license: mit
base_model: roberta-base
tags:
- generated_from_trainer
model-index:
- name: fine-tuned-roberta2-nosql-injection
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# fine-tuned-roberta2-nosql-injection

This model is a fine-tuned version of [roberta-base](https://huggingface.co./roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0134

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 500

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 1.2572        | 1.0   | 158   | 0.2235          |
| 0.1144        | 2.0   | 316   | 0.0546          |
| 0.0873        | 3.0   | 474   | 0.0943          |
| 0.0488        | 4.0   | 632   | 0.0324          |
| 0.0453        | 5.0   | 790   | 0.0970          |
| 0.1358        | 6.0   | 948   | 0.0003          |
| 0.0208        | 7.0   | 1106  | 0.0025          |
| 0.0409        | 8.0   | 1264  | 0.0000          |
| 0.0489        | 9.0   | 1422  | 0.0505          |
| 0.0808        | 10.0  | 1580  | 0.0000          |
| 0.0219        | 11.0  | 1738  | 0.0929          |
| 0.1204        | 12.0  | 1896  | 0.0000          |
| 0.0355        | 13.0  | 2054  | 0.0001          |
| 0.0093        | 14.0  | 2212  | 0.0671          |
| 0.0216        | 15.0  | 2370  | 0.0279          |
| 0.0388        | 16.0  | 2528  | 0.0173          |
| 0.0185        | 17.0  | 2686  | 0.0303          |
| 0.0069        | 18.0  | 2844  | 0.0160          |
| 0.0481        | 19.0  | 3002  | 0.0557          |
| 0.0459        | 20.0  | 3160  | 0.0022          |
| 0.0146        | 21.0  | 3318  | 0.0191          |
| 0.0313        | 22.0  | 3476  | 0.1119          |
| 0.025         | 23.0  | 3634  | 0.0000          |
| 0.0328        | 24.0  | 3792  | 0.0559          |
| 0.0251        | 25.0  | 3950  | 0.0343          |
| 0.0334        | 26.0  | 4108  | 0.0263          |
| 0.0048        | 27.0  | 4266  | 0.0424          |
| 0.0584        | 28.0  | 4424  | 0.0578          |
| 0.0861        | 29.0  | 4582  | 0.0495          |
| 0.0345        | 30.0  | 4740  | 0.0282          |
| 0.0193        | 31.0  | 4898  | 0.0412          |
| 0.0216        | 32.0  | 5056  | 0.0537          |
| 0.0241        | 33.0  | 5214  | 0.0124          |
| 0.009         | 34.0  | 5372  | 0.0272          |
| 0.0309        | 35.0  | 5530  | 0.0000          |
| 0.0069        | 36.0  | 5688  | 0.0000          |
| 0.0398        | 37.0  | 5846  | 0.0000          |
| 0.0474        | 38.0  | 6004  | 0.0017          |
| 0.0263        | 39.0  | 6162  | 0.0094          |
| 0.0128        | 40.0  | 6320  | 0.0000          |
| 0.0101        | 41.0  | 6478  | 0.0015          |
| 0.0415        | 42.0  | 6636  | 0.0004          |
| 0.0204        | 43.0  | 6794  | 0.0307          |
| 0.0342        | 44.0  | 6952  | 0.0348          |
| 0.0103        | 45.0  | 7110  | 0.0344          |
| 0.0389        | 46.0  | 7268  | 0.0238          |
| 0.0108        | 47.0  | 7426  | 0.0651          |
| 0.0486        | 48.0  | 7584  | 0.0290          |
| 0.0131        | 49.0  | 7742  | 0.0130          |
| 0.0027        | 50.0  | 7900  | 0.1448          |
| 0.0119        | 51.0  | 8058  | 0.0034          |
| 0.0142        | 52.0  | 8216  | 0.0014          |
| 0.0123        | 53.0  | 8374  | 0.0292          |
| 0.0364        | 54.0  | 8532  | 0.0000          |
| 0.0541        | 55.0  | 8690  | 0.0000          |
| 0.0309        | 56.0  | 8848  | 0.0032          |
| 0.0278        | 57.0  | 9006  | 0.0124          |
| 0.0247        | 58.0  | 9164  | 0.0000          |
| 0.0077        | 59.0  | 9322  | 0.0000          |
| 0.0104        | 60.0  | 9480  | 0.0150          |
| 0.0302        | 61.0  | 9638  | 0.0003          |
| 0.0225        | 62.0  | 9796  | 0.0358          |
| 0.0296        | 63.0  | 9954  | 0.0000          |
| 0.0533        | 64.0  | 10112 | 0.1224          |
| 0.0121        | 65.0  | 10270 | 0.0092          |
| 0.0249        | 66.0  | 10428 | 0.0087          |
| 0.0637        | 67.0  | 10586 | 0.0003          |
| 0.035         | 68.0  | 10744 | 0.0000          |
| 0.0117        | 69.0  | 10902 | 0.0393          |
| 0.0049        | 70.0  | 11060 | 0.0209          |
| 0.0225        | 71.0  | 11218 | 0.0003          |
| 0.0121        | 72.0  | 11376 | 0.0001          |
| 0.0079        | 73.0  | 11534 | 0.0006          |
| 0.0354        | 74.0  | 11692 | 0.0458          |
| 0.0071        | 75.0  | 11850 | 0.0009          |
| 0.0043        | 76.0  | 12008 | 0.0290          |
| 0.0239        | 77.0  | 12166 | 0.0001          |
| 0.0159        | 78.0  | 12324 | 0.0162          |
| 0.0051        | 79.0  | 12482 | 0.0673          |
| 0.0028        | 80.0  | 12640 | 0.0275          |
| 0.0155        | 81.0  | 12798 | 0.0101          |
| 0.0068        | 82.0  | 12956 | 0.0271          |
| 0.006         | 83.0  | 13114 | 0.0120          |
| 0.0107        | 84.0  | 13272 | 0.0008          |
| 0.0122        | 85.0  | 13430 | 0.0124          |
| 0.0028        | 86.0  | 13588 | 0.0000          |
| 0.0256        | 87.0  | 13746 | 0.0011          |
| 0.037         | 88.0  | 13904 | 0.0213          |
| 0.0044        | 89.0  | 14062 | 0.0000          |
| 0.003         | 90.0  | 14220 | 0.0000          |
| 0.0072        | 91.0  | 14378 | 0.0381          |
| 0.0309        | 92.0  | 14536 | 0.0000          |
| 0.0123        | 93.0  | 14694 | 0.0117          |
| 0.0002        | 94.0  | 14852 | 0.0024          |
| 0.0033        | 95.0  | 15010 | 0.0002          |
| 0.0181        | 96.0  | 15168 | 0.0071          |
| 0.0167        | 97.0  | 15326 | 0.0219          |
| 0.0136        | 98.0  | 15484 | 0.0001          |
| 0.0116        | 99.0  | 15642 | 0.0189          |
| 0.0118        | 100.0 | 15800 | 0.0000          |
| 0.0345        | 101.0 | 15958 | 0.0006          |
| 0.0032        | 102.0 | 16116 | 0.0355          |
| 0.0003        | 103.0 | 16274 | 0.0484          |
| 0.0281        | 104.0 | 16432 | 0.0000          |
| 0.029         | 105.0 | 16590 | 0.0319          |
| 0.006         | 106.0 | 16748 | 0.0016          |
| 0.0001        | 107.0 | 16906 | 0.0608          |
| 0.024         | 108.0 | 17064 | 0.0000          |
| 0.0187        | 109.0 | 17222 | 0.0000          |
| 0.0038        | 110.0 | 17380 | 0.0000          |
| 0.0046        | 111.0 | 17538 | 0.0045          |
| 0.0024        | 112.0 | 17696 | 0.0000          |
| 0.033         | 113.0 | 17854 | 0.0001          |
| 0.0048        | 114.0 | 18012 | 0.0511          |
| 0.0005        | 115.0 | 18170 | 0.0000          |
| 0.0166        | 116.0 | 18328 | 0.0000          |
| 0.0113        | 117.0 | 18486 | 0.0122          |
| 0.0043        | 118.0 | 18644 | 0.0122          |
| 0.0703        | 119.0 | 18802 | 0.0013          |
| 0.0051        | 120.0 | 18960 | 0.0009          |
| 0.048         | 121.0 | 19118 | 0.0082          |
| 0.0408        | 122.0 | 19276 | 0.0137          |
| 0.018         | 123.0 | 19434 | 0.0365          |
| 0.0125        | 124.0 | 19592 | 0.0111          |
| 0.0075        | 125.0 | 19750 | 0.0179          |
| 0.0379        | 126.0 | 19908 | 0.0000          |
| 0.0029        | 127.0 | 20066 | 0.0181          |
| 0.007         | 128.0 | 20224 | 0.0610          |
| 0.0069        | 129.0 | 20382 | 0.0203          |
| 0.0082        | 130.0 | 20540 | 0.0000          |
| 0.0136        | 131.0 | 20698 | 0.1019          |
| 0.0152        | 132.0 | 20856 | 0.0000          |
| 0.0078        | 133.0 | 21014 | 0.0089          |
| 0.0003        | 134.0 | 21172 | 0.0000          |
| 0.0069        | 135.0 | 21330 | 0.0278          |
| 0.0089        | 136.0 | 21488 | 0.0330          |
| 0.0642        | 137.0 | 21646 | 0.0251          |
| 0.0077        | 138.0 | 21804 | 0.0015          |
| 0.0062        | 139.0 | 21962 | 0.0218          |
| 0.0203        | 140.0 | 22120 | 0.0032          |
| 0.0173        | 141.0 | 22278 | 0.0000          |
| 0.0149        | 142.0 | 22436 | 0.0231          |
| 0.011         | 143.0 | 22594 | 0.0218          |
| 0.0142        | 144.0 | 22752 | 0.0068          |
| 0.0075        | 145.0 | 22910 | 0.0067          |
| 0.0021        | 146.0 | 23068 | 0.0172          |
| 0.009         | 147.0 | 23226 | 0.0134          |
| 0.0072        | 148.0 | 23384 | 0.0050          |
| 0.0297        | 149.0 | 23542 | 0.0162          |
| 0.0189        | 150.0 | 23700 | 0.0000          |
| 0.0065        | 151.0 | 23858 | 0.0006          |
| 0.0423        | 152.0 | 24016 | 0.0102          |
| 0.0001        | 153.0 | 24174 | 0.0147          |
| 0.0607        | 154.0 | 24332 | 0.0062          |
| 0.0247        | 155.0 | 24490 | 0.0000          |
| 0.0187        | 156.0 | 24648 | 0.0000          |
| 0.025         | 157.0 | 24806 | 0.0278          |
| 0.0069        | 158.0 | 24964 | 0.0002          |
| 0.0024        | 159.0 | 25122 | 0.0002          |
| 0.0194        | 160.0 | 25280 | 0.0000          |
| 0.0072        | 161.0 | 25438 | 0.0000          |
| 0.0552        | 162.0 | 25596 | 0.0304          |
| 0.0114        | 163.0 | 25754 | 0.0184          |
| 0.001         | 164.0 | 25912 | 0.0000          |
| 0.0076        | 165.0 | 26070 | 0.0004          |
| 0.0158        | 166.0 | 26228 | 0.0000          |
| 0.0017        | 167.0 | 26386 | 0.0145          |
| 0.0009        | 168.0 | 26544 | 0.0436          |
| 0.0199        | 169.0 | 26702 | 0.0147          |
| 0.0067        | 170.0 | 26860 | 0.0003          |
| 0.0082        | 171.0 | 27018 | 0.0151          |
| 0.0079        | 172.0 | 27176 | 0.0018          |
| 0.0016        | 173.0 | 27334 | 0.0002          |
| 0.0095        | 174.0 | 27492 | 0.0152          |
| 0.0166        | 175.0 | 27650 | 0.0000          |
| 0.0018        | 176.0 | 27808 | 0.0495          |
| 0.0025        | 177.0 | 27966 | 0.0824          |
| 0.01          | 178.0 | 28124 | 0.0000          |
| 0.0054        | 179.0 | 28282 | 0.0072          |
| 0.0082        | 180.0 | 28440 | 0.0019          |
| 0.0           | 181.0 | 28598 | 0.0131          |
| 0.0034        | 182.0 | 28756 | 0.0163          |
| 0.0083        | 183.0 | 28914 | 0.0175          |
| 0.0035        | 184.0 | 29072 | 0.0111          |
| 0.009         | 185.0 | 29230 | 0.0004          |
| 0.015         | 186.0 | 29388 | 0.0000          |
| 0.0011        | 187.0 | 29546 | 0.0182          |
| 0.0142        | 188.0 | 29704 | 0.0581          |
| 0.0048        | 189.0 | 29862 | 0.0002          |
| 0.0021        | 190.0 | 30020 | 0.0002          |
| 0.0           | 191.0 | 30178 | 0.0334          |
| 0.016         | 192.0 | 30336 | 0.0003          |
| 0.0109        | 193.0 | 30494 | 0.0000          |
| 0.0085        | 194.0 | 30652 | 0.0028          |
| 0.002         | 195.0 | 30810 | 0.0000          |
| 0.002         | 196.0 | 30968 | 0.0144          |
| 0.0142        | 197.0 | 31126 | 0.0000          |
| 0.0727        | 198.0 | 31284 | 0.0163          |
| 0.0238        | 199.0 | 31442 | 0.0000          |
| 0.0171        | 200.0 | 31600 | 0.0141          |
| 0.0158        | 201.0 | 31758 | 0.0000          |
| 0.0049        | 202.0 | 31916 | 0.0086          |
| 0.0016        | 203.0 | 32074 | 0.0000          |
| 0.0088        | 204.0 | 32232 | 0.0007          |
| 0.0048        | 205.0 | 32390 | 0.0164          |
| 0.0103        | 206.0 | 32548 | 0.0080          |
| 0.0039        | 207.0 | 32706 | 0.0441          |
| 0.0041        | 208.0 | 32864 | 0.0000          |
| 0.027         | 209.0 | 33022 | 0.0004          |
| 0.0229        | 210.0 | 33180 | 0.0149          |
| 0.01          | 211.0 | 33338 | 0.0154          |
| 0.0036        | 212.0 | 33496 | 0.0191          |
| 0.0036        | 213.0 | 33654 | 0.0491          |
| 0.0316        | 214.0 | 33812 | 0.0000          |
| 0.0014        | 215.0 | 33970 | 0.0198          |
| 0.0097        | 216.0 | 34128 | 0.0000          |
| 0.0011        | 217.0 | 34286 | 0.0001          |
| 0.0063        | 218.0 | 34444 | 0.0010          |
| 0.0143        | 219.0 | 34602 | 0.0243          |
| 0.0022        | 220.0 | 34760 | 0.0275          |
| 0.002         | 221.0 | 34918 | 0.0000          |
| 0.0095        | 222.0 | 35076 | 0.0119          |
| 0.0016        | 223.0 | 35234 | 0.0001          |
| 0.0165        | 224.0 | 35392 | 0.0000          |
| 0.0106        | 225.0 | 35550 | 0.0279          |
| 0.0316        | 226.0 | 35708 | 0.0006          |
| 0.0041        | 227.0 | 35866 | 0.0000          |
| 0.0492        | 228.0 | 36024 | 0.0123          |
| 0.0034        | 229.0 | 36182 | 0.0196          |
| 0.0035        | 230.0 | 36340 | 0.0113          |
| 0.0197        | 231.0 | 36498 | 0.0000          |
| 0.0234        | 232.0 | 36656 | 0.0026          |
| 0.0102        | 233.0 | 36814 | 0.0250          |
| 0.0113        | 234.0 | 36972 | 0.0001          |
| 0.0008        | 235.0 | 37130 | 0.0000          |
| 0.0032        | 236.0 | 37288 | 0.0001          |
| 0.1045        | 237.0 | 37446 | 0.0000          |
| 0.0028        | 238.0 | 37604 | 0.0143          |
| 0.0028        | 239.0 | 37762 | 0.0143          |
| 0.0002        | 240.0 | 37920 | 0.0171          |
| 0.0302        | 241.0 | 38078 | 0.0406          |
| 0.0046        | 242.0 | 38236 | 0.0273          |
| 0.0124        | 243.0 | 38394 | 0.0754          |
| 0.0055        | 244.0 | 38552 | 0.0396          |
| 0.0102        | 245.0 | 38710 | 0.0003          |
| 0.0139        | 246.0 | 38868 | 0.0201          |
| 0.0367        | 247.0 | 39026 | 0.0343          |
| 0.0216        | 248.0 | 39184 | 0.0015          |
| 0.0091        | 249.0 | 39342 | 0.0637          |
| 0.0115        | 250.0 | 39500 | 0.0053          |
| 0.0254        | 251.0 | 39658 | 0.0299          |
| 0.0202        | 252.0 | 39816 | 0.0011          |
| 0.0005        | 253.0 | 39974 | 0.0181          |
| 0.0024        | 254.0 | 40132 | 0.0223          |
| 0.0002        | 255.0 | 40290 | 0.0001          |
| 0.0102        | 256.0 | 40448 | 0.0593          |
| 0.0133        | 257.0 | 40606 | 0.0000          |
| 0.0051        | 258.0 | 40764 | 0.0001          |
| 0.0027        | 259.0 | 40922 | 0.0225          |
| 0.0053        | 260.0 | 41080 | 0.0000          |
| 0.0073        | 261.0 | 41238 | 0.0108          |
| 0.0033        | 262.0 | 41396 | 0.0118          |
| 0.0024        | 263.0 | 41554 | 0.0233          |
| 0.02          | 264.0 | 41712 | 0.0236          |
| 0.0078        | 265.0 | 41870 | 0.0180          |
| 0.0117        | 266.0 | 42028 | 0.0000          |
| 0.0156        | 267.0 | 42186 | 0.0000          |
| 0.0041        | 268.0 | 42344 | 0.0009          |
| 0.0003        | 269.0 | 42502 | 0.0108          |
| 0.0049        | 270.0 | 42660 | 0.0000          |
| 0.0021        | 271.0 | 42818 | 0.0174          |
| 0.0083        | 272.0 | 42976 | 0.0201          |
| 0.0002        | 273.0 | 43134 | 0.0371          |
| 0.0219        | 274.0 | 43292 | 0.0000          |
| 0.0014        | 275.0 | 43450 | 0.0156          |
| 0.0102        | 276.0 | 43608 | 0.0002          |
| 0.0311        | 277.0 | 43766 | 0.0008          |
| 0.0115        | 278.0 | 43924 | 0.0269          |
| 0.0122        | 279.0 | 44082 | 0.0131          |
| 0.0021        | 280.0 | 44240 | 0.0000          |
| 0.0032        | 281.0 | 44398 | 0.0157          |
| 0.0079        | 282.0 | 44556 | 0.0146          |
| 0.0344        | 283.0 | 44714 | 0.0142          |
| 0.0086        | 284.0 | 44872 | 0.0000          |
| 0.0045        | 285.0 | 45030 | 0.0011          |
| 0.0077        | 286.0 | 45188 | 0.0117          |
| 0.0026        | 287.0 | 45346 | 0.0006          |
| 0.0042        | 288.0 | 45504 | 0.0069          |
| 0.0528        | 289.0 | 45662 | 0.0199          |
| 0.0162        | 290.0 | 45820 | 0.0090          |
| 0.0598        | 291.0 | 45978 | 0.0000          |
| 0.0293        | 292.0 | 46136 | 0.0142          |
| 0.0021        | 293.0 | 46294 | 0.0330          |
| 0.0073        | 294.0 | 46452 | 0.0029          |
| 0.0006        | 295.0 | 46610 | 0.0007          |
| 0.0065        | 296.0 | 46768 | 0.0005          |
| 0.0075        | 297.0 | 46926 | 0.0371          |
| 0.0017        | 298.0 | 47084 | 0.0001          |
| 0.0023        | 299.0 | 47242 | 0.0442          |
| 0.0059        | 300.0 | 47400 | 0.0000          |
| 0.0041        | 301.0 | 47558 | 0.0198          |
| 0.0372        | 302.0 | 47716 | 0.0123          |
| 0.0004        | 303.0 | 47874 | 0.0177          |
| 0.0008        | 304.0 | 48032 | 0.0172          |
| 0.0002        | 305.0 | 48190 | 0.0005          |
| 0.0064        | 306.0 | 48348 | 0.0000          |
| 0.0116        | 307.0 | 48506 | 0.0142          |
| 0.0113        | 308.0 | 48664 | 0.0054          |
| 0.0046        | 309.0 | 48822 | 0.0090          |
| 0.0026        | 310.0 | 48980 | 0.0118          |
| 0.0051        | 311.0 | 49138 | 0.0012          |
| 0.0027        | 312.0 | 49296 | 0.0099          |
| 0.0031        | 313.0 | 49454 | 0.0104          |
| 0.0031        | 314.0 | 49612 | 0.0098          |
| 0.0085        | 315.0 | 49770 | 0.0115          |
| 0.0023        | 316.0 | 49928 | 0.0000          |
| 0.0143        | 317.0 | 50086 | 0.0000          |
| 0.0003        | 318.0 | 50244 | 0.0090          |
| 0.0179        | 319.0 | 50402 | 0.0124          |
| 0.0012        | 320.0 | 50560 | 0.0010          |
| 0.009         | 321.0 | 50718 | 0.0219          |
| 0.0082        | 322.0 | 50876 | 0.0102          |
| 0.0084        | 323.0 | 51034 | 0.0016          |
| 0.0029        | 324.0 | 51192 | 0.0227          |
| 0.0023        | 325.0 | 51350 | 0.0148          |
| 0.0022        | 326.0 | 51508 | 0.0157          |
| 0.0065        | 327.0 | 51666 | 0.0132          |
| 0.0257        | 328.0 | 51824 | 0.0162          |
| 0.0013        | 329.0 | 51982 | 0.0005          |
| 0.0135        | 330.0 | 52140 | 0.0278          |
| 0.0029        | 331.0 | 52298 | 0.0391          |
| 0.0139        | 332.0 | 52456 | 0.0000          |
| 0.0152        | 333.0 | 52614 | 0.0005          |
| 0.0042        | 334.0 | 52772 | 0.0008          |
| 0.0161        | 335.0 | 52930 | 0.0005          |
| 0.0008        | 336.0 | 53088 | 0.0000          |
| 0.0098        | 337.0 | 53246 | 0.0149          |
| 0.0096        | 338.0 | 53404 | 0.0097          |
| 0.0158        | 339.0 | 53562 | 0.0121          |
| 0.0001        | 340.0 | 53720 | 0.0168          |
| 0.0068        | 341.0 | 53878 | 0.0006          |
| 0.0026        | 342.0 | 54036 | 0.0000          |
| 0.0061        | 343.0 | 54194 | 0.0126          |
| 0.0071        | 344.0 | 54352 | 0.0154          |
| 0.0032        | 345.0 | 54510 | 0.0135          |
| 0.0034        | 346.0 | 54668 | 0.0046          |
| 0.003         | 347.0 | 54826 | 0.0137          |
| 0.0028        | 348.0 | 54984 | 0.0012          |
| 0.0157        | 349.0 | 55142 | 0.0193          |
| 0.0046        | 350.0 | 55300 | 0.0006          |
| 0.0023        | 351.0 | 55458 | 0.0137          |
| 0.0           | 352.0 | 55616 | 0.0111          |
| 0.0041        | 353.0 | 55774 | 0.0004          |
| 0.0032        | 354.0 | 55932 | 0.0000          |
| 0.0103        | 355.0 | 56090 | 0.0004          |
| 0.0           | 356.0 | 56248 | 0.0123          |
| 0.0074        | 357.0 | 56406 | 0.0205          |
| 0.0098        | 358.0 | 56564 | 0.0009          |
| 0.004         | 359.0 | 56722 | 0.0005          |
| 0.0018        | 360.0 | 56880 | 0.0134          |
| 0.0032        | 361.0 | 57038 | 0.0000          |
| 0.0063        | 362.0 | 57196 | 0.0230          |
| 0.0015        | 363.0 | 57354 | 0.0087          |
| 0.0041        | 364.0 | 57512 | 0.0110          |
| 0.0021        | 365.0 | 57670 | 0.0075          |
| 0.0019        | 366.0 | 57828 | 0.0000          |
| 0.0021        | 367.0 | 57986 | 0.0109          |
| 0.0018        | 368.0 | 58144 | 0.0228          |
| 0.0027        | 369.0 | 58302 | 0.0183          |
| 0.0036        | 370.0 | 58460 | 0.0116          |
| 0.0042        | 371.0 | 58618 | 0.0015          |
| 0.0029        | 372.0 | 58776 | 0.0010          |
| 0.0206        | 373.0 | 58934 | 0.0000          |
| 0.0071        | 374.0 | 59092 | 0.0000          |
| 0.0004        | 375.0 | 59250 | 0.0151          |
| 0.0009        | 376.0 | 59408 | 0.0000          |
| 0.0037        | 377.0 | 59566 | 0.0341          |
| 0.0097        | 378.0 | 59724 | 0.0239          |
| 0.0032        | 379.0 | 59882 | 0.0147          |
| 0.0041        | 380.0 | 60040 | 0.0006          |
| 0.002         | 381.0 | 60198 | 0.0003          |
| 0.0063        | 382.0 | 60356 | 0.0             |
| 0.0054        | 383.0 | 60514 | 0.0093          |
| 0.0069        | 384.0 | 60672 | 0.0089          |
| 0.0056        | 385.0 | 60830 | 0.0006          |
| 0.0071        | 386.0 | 60988 | 0.0219          |
| 0.0061        | 387.0 | 61146 | 0.0503          |
| 0.0068        | 388.0 | 61304 | 0.0028          |
| 0.002         | 389.0 | 61462 | 0.0011          |
| 0.0024        | 390.0 | 61620 | 0.0075          |
| 0.0028        | 391.0 | 61778 | 0.0009          |
| 0.0045        | 392.0 | 61936 | 0.0015          |
| 0.0009        | 393.0 | 62094 | 0.0350          |
| 0.0064        | 394.0 | 62252 | 0.0103          |
| 0.0003        | 395.0 | 62410 | 0.0118          |
| 0.0112        | 396.0 | 62568 | 0.0000          |
| 0.0046        | 397.0 | 62726 | 0.0118          |
| 0.0064        | 398.0 | 62884 | 0.0482          |
| 0.0021        | 399.0 | 63042 | 0.0228          |
| 0.0083        | 400.0 | 63200 | 0.0009          |
| 0.0008        | 401.0 | 63358 | 0.0005          |
| 0.0061        | 402.0 | 63516 | 0.0000          |
| 0.0038        | 403.0 | 63674 | 0.0031          |
| 0.0022        | 404.0 | 63832 | 0.0198          |
| 0.0074        | 405.0 | 63990 | 0.0000          |
| 0.0024        | 406.0 | 64148 | 0.0             |
| 0.0017        | 407.0 | 64306 | 0.0140          |
| 0.0004        | 408.0 | 64464 | 0.0210          |
| 0.003         | 409.0 | 64622 | 0.0096          |
| 0.0013        | 410.0 | 64780 | 0.0             |
| 0.0166        | 411.0 | 64938 | 0.0285          |
| 0.0029        | 412.0 | 65096 | 0.0000          |
| 0.0133        | 413.0 | 65254 | 0.0000          |
| 0.014         | 414.0 | 65412 | 0.0008          |
| 0.0044        | 415.0 | 65570 | 0.0163          |
| 0.0077        | 416.0 | 65728 | 0.0000          |
| 0.0002        | 417.0 | 65886 | 0.0004          |
| 0.0024        | 418.0 | 66044 | 0.0375          |
| 0.0092        | 419.0 | 66202 | 0.0008          |
| 0.0059        | 420.0 | 66360 | 0.0139          |
| 0.0005        | 421.0 | 66518 | 0.0121          |
| 0.0128        | 422.0 | 66676 | 0.0105          |
| 0.0035        | 423.0 | 66834 | 0.0005          |
| 0.0042        | 424.0 | 66992 | 0.0             |
| 0.0051        | 425.0 | 67150 | 0.0311          |
| 0.0089        | 426.0 | 67308 | 0.0084          |
| 0.0112        | 427.0 | 67466 | 0.0229          |
| 0.0002        | 428.0 | 67624 | 0.0156          |
| 0.0049        | 429.0 | 67782 | 0.0117          |
| 0.0051        | 430.0 | 67940 | 0.0150          |
| 0.0088        | 431.0 | 68098 | 0.0005          |
| 0.01          | 432.0 | 68256 | 0.0136          |
| 0.0032        | 433.0 | 68414 | 0.0             |
| 0.0003        | 434.0 | 68572 | 0.0155          |
| 0.0032        | 435.0 | 68730 | 0.0119          |
| 0.0041        | 436.0 | 68888 | 0.0003          |
| 0.0           | 437.0 | 69046 | 0.0168          |
| 0.0283        | 438.0 | 69204 | 0.0131          |
| 0.0032        | 439.0 | 69362 | 0.0000          |
| 0.007         | 440.0 | 69520 | 0.0000          |
| 0.0079        | 441.0 | 69678 | 0.0251          |
| 0.0121        | 442.0 | 69836 | 0.0003          |
| 0.0067        | 443.0 | 69994 | 0.0005          |
| 0.0017        | 444.0 | 70152 | 0.0000          |
| 0.0027        | 445.0 | 70310 | 0.0138          |
| 0.0013        | 446.0 | 70468 | 0.0322          |
| 0.0086        | 447.0 | 70626 | 0.0000          |
| 0.0066        | 448.0 | 70784 | 0.0112          |
| 0.003         | 449.0 | 70942 | 0.0000          |
| 0.0006        | 450.0 | 71100 | 0.0097          |
| 0.01          | 451.0 | 71258 | 0.0113          |
| 0.0003        | 452.0 | 71416 | 0.0098          |
| 0.0027        | 453.0 | 71574 | 0.0009          |
| 0.0034        | 454.0 | 71732 | 0.0119          |
| 0.0041        | 455.0 | 71890 | 0.0012          |
| 0.01          | 456.0 | 72048 | 0.0222          |
| 0.0032        | 457.0 | 72206 | 0.0111          |
| 0.0027        | 458.0 | 72364 | 0.0000          |
| 0.0064        | 459.0 | 72522 | 0.0149          |
| 0.0034        | 460.0 | 72680 | 0.0097          |
| 0.0083        | 461.0 | 72838 | 0.0146          |
| 0.002         | 462.0 | 72996 | 0.0237          |
| 0.004         | 463.0 | 73154 | 0.0135          |
| 0.0028        | 464.0 | 73312 | 0.0056          |
| 0.0008        | 465.0 | 73470 | 0.0000          |
| 0.0015        | 466.0 | 73628 | 0.0004          |
| 0.0023        | 467.0 | 73786 | 0.0320          |
| 0.008         | 468.0 | 73944 | 0.0108          |
| 0.0052        | 469.0 | 74102 | 0.0283          |
| 0.0088        | 470.0 | 74260 | 0.0009          |
| 0.0037        | 471.0 | 74418 | 0.0251          |
| 0.006         | 472.0 | 74576 | 0.0208          |
| 0.0008        | 473.0 | 74734 | 0.0237          |
| 0.005         | 474.0 | 74892 | 0.0198          |
| 0.0033        | 475.0 | 75050 | 0.0000          |
| 0.0014        | 476.0 | 75208 | 0.0000          |
| 0.0028        | 477.0 | 75366 | 0.0004          |
| 0.0082        | 478.0 | 75524 | 0.0149          |
| 0.0067        | 479.0 | 75682 | 0.0124          |
| 0.0062        | 480.0 | 75840 | 0.0327          |
| 0.0035        | 481.0 | 75998 | 0.0273          |
| 0.0013        | 482.0 | 76156 | 0.0007          |
| 0.0023        | 483.0 | 76314 | 0.0112          |
| 0.0042        | 484.0 | 76472 | 0.0101          |
| 0.0001        | 485.0 | 76630 | 0.0004          |
| 0.0           | 486.0 | 76788 | 0.0000          |
| 0.0069        | 487.0 | 76946 | 0.0278          |
| 0.0002        | 488.0 | 77104 | 0.0000          |
| 0.0045        | 489.0 | 77262 | 0.0163          |
| 0.0005        | 490.0 | 77420 | 0.0000          |
| 0.0005        | 491.0 | 77578 | 0.0272          |
| 0.0014        | 492.0 | 77736 | 0.0000          |
| 0.0088        | 493.0 | 77894 | 0.0215          |
| 0.0033        | 494.0 | 78052 | 0.0009          |
| 0.0031        | 495.0 | 78210 | 0.0261          |
| 0.0038        | 496.0 | 78368 | 0.0120          |
| 0.0091        | 497.0 | 78526 | 0.0151          |
| 0.0062        | 498.0 | 78684 | 0.0009          |
| 0.004         | 499.0 | 78842 | 0.0005          |
| 0.0025        | 500.0 | 79000 | 0.0134          |


### Framework versions

- Transformers 4.31.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.13.1
- Tokenizers 0.11.0