pszemraj autoevaluator HF staff commited on
Commit
19381cd
1 Parent(s): 75364d6

Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator (#9)

Browse files

- Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator (da400766b4b14868f9e1d98aa48da5f443d63e9c)


Co-authored-by: Evaluation Bot <[email protected]>

Files changed (1) hide show
  1. README.md +254 -193
README.md CHANGED
@@ -1,13 +1,13 @@
1
  ---
 
 
 
2
  tags:
3
  - summarization
4
  - summary
5
  - booksum
6
  - long-document
7
  - long-form
8
- license:
9
- - apache-2.0
10
- - bsd-3-clause
11
  datasets:
12
  - kmfoda/booksum
13
  - big_patent
@@ -27,39 +27,38 @@ widget:
27
  deviation of the average recurrence interval, the more specific could be the long
28
  term prediction of a future mainshock.
29
  example_title: earthquakes
30
- - text: " A typical feed-forward neural field algorithm. Spatiotemporal coordinates\
31
- \ are fed into a neural network that predicts values in the reconstructed domain.\
32
- \ Then, this domain is mapped to the sensor domain where sensor measurements are\
33
- \ available as supervision. Class and Section Problems Addressed Generalization\
34
- \ (Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid\
35
- \ Representations (Section 3) Computation & memory efficiency, representation\
36
- \ capacity, editability: Forward Maps (Section 4) Inverse problems Network Architecture\
37
- \ (Section 5) Spectral bias, integration & derivatives. Manipulating Neural Fields\
38
- \ (Section 6) Edit ability, constraints, regularization. Table 2: The five classes\
39
- \ of techniques in the neural field toolbox each addresses problems that arise\
40
- \ in learning, inference, and control. (Section 3). We can supervise reconstruction\
41
- \ via differentiable forward maps that transform Or project our domain (e.g, 3D\
42
- \ reconstruction via 2D images; Section 4) With appropriate network architecture\
43
- \ choices, we can overcome neural network spectral biases (blurriness) and efficiently\
44
- \ compute derivatives and integrals (Section 5). Finally, we can manipulate neural\
45
- \ fields to add constraints and regularizations, and to achieve editable representations\
46
- \ (Section 6). Collectively, these classes constitute a 'toolbox' of techniques\
47
- \ to help solve problems with neural fields There are three components in a conditional\
48
- \ neural field: (1) An encoder or inference function \u20AC that outputs the conditioning\
49
- \ latent variable 2 given an observation 0 E(0) =2. 2 is typically a low-dimensional\
50
- \ vector, and is often referred to aS a latent code Or feature code_ (2) A mapping\
51
- \ function 4 between Z and neural field parameters O: Y(z) = O; (3) The neural\
52
- \ field itself $. The encoder \u20AC finds the most probable z given the observations\
53
- \ O: argmaxz P(2/0). The decoder maximizes the inverse conditional probability\
54
- \ to find the most probable 0 given Z: arg- max P(Olz). We discuss different encoding\
55
- \ schemes with different optimality guarantees (Section 2.1.1), both global and\
56
- \ local conditioning (Section 2.1.2), and different mapping functions Y (Section\
57
- \ 2.1.3) 2. Generalization Suppose we wish to estimate a plausible 3D surface\
58
- \ shape given a partial or noisy point cloud. We need a suitable prior over the\
59
- \ sur- face in its reconstruction domain to generalize to the partial observations.\
60
- \ A neural network expresses a prior via the function space of its architecture\
61
- \ and parameters 0, and generalization is influenced by the inductive bias of\
62
- \ this function space (Section 5)."
63
  example_title: scientific paper
64
  - text: 'Is a else or outside the cob and tree written being of early client rope
65
  and you have is for good reasons. On to the ocean in Orange for time. By''s the
@@ -111,68 +110,82 @@ widget:
111
  the point of you of your model. This hidden data is complete by unseen. In other
112
  words, we solve our problem of validation.'
113
  example_title: transcribed audio - lecture
114
- - text: "Transformer-based models have shown to be very useful for many NLP tasks.\
115
- \ However, a major limitation of transformers-based models is its O(n^2)O(n 2)\
116
- \ time & memory complexity (where nn is sequence length). Hence, it's computationally\
117
- \ very expensive to apply transformer-based models on long sequences n > 512n>512.\
118
- \ Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention\
119
- \ try to remedy this problem by approximating the full attention matrix. You can\
120
- \ checkout \U0001F917's recent blog post in case you are unfamiliar with these\
121
- \ models.\nBigBird (introduced in paper) is one of such recent models to address\
122
- \ this issue. BigBird relies on block sparse attention instead of normal attention\
123
- \ (i.e. BERT's attention) and can handle sequences up to a length of 4096 at a\
124
- \ much lower computational cost compared to BERT. It has achieved SOTA on various\
125
- \ tasks involving very long sequences such as long documents summarization, question-answering\
126
- \ with long contexts.\nBigBird RoBERTa-like model is now available in \U0001F917\
127
- Transformers. The goal of this post is to give the reader an in-depth understanding\
128
- \ of big bird implementation & ease one's life in using BigBird with \U0001F917\
129
- Transformers. But, before going into more depth, it is important to remember that\
130
- \ the BigBird's attention is an approximation of BERT's full attention and therefore\
131
- \ does not strive to be better than BERT's full attention, but rather to be more\
132
- \ efficient. It simply allows to apply transformer-based models to much longer\
133
- \ sequences since BERT's quadratic memory requirement quickly becomes unbearable.\
134
- \ Simply put, if we would have \u221E compute & \u221E time, BERT's attention\
135
- \ would be preferred over block sparse attention (which we are going to discuss\
136
- \ in this post).\nIf you wonder why we need more compute when working with longer\
137
- \ sequences, this blog post is just right for you!\nSome of the main questions\
138
- \ one might have when working with standard BERT-like attention include:\nDo all\
139
- \ tokens really have to attend to all other tokens? Why not compute attention\
140
- \ only over important tokens? How to decide what tokens are important? How to\
141
- \ attend to just a few tokens in a very efficient way? In this blog post, we will\
142
- \ try to answer those questions.\nWhat tokens should be attended to? We will give\
143
- \ a practical example of how attention works by considering the sentence 'BigBird\
144
- \ is now available in HuggingFace for extractive question answering'. In BERT-like\
145
- \ attention, every word would simply attend to all other tokens.\nLet's think\
146
- \ about a sensible choice of key tokens that a queried token actually only should\
147
- \ attend to by writing some pseudo-code. Will will assume that the token available\
148
- \ is queried and build a sensible list of key tokens to attend to.\n>>> # let's\
149
- \ consider following sentence as an example >>> example = ['BigBird', 'is', 'now',\
150
- \ 'available', 'in', 'HuggingFace', 'for', 'extractive', 'question', 'answering']\n\
151
- >>> # further let's assume, we're trying to understand the representation of 'available'\
152
- \ i.e. >>> query_token = 'available' >>> # We will initialize an empty `set` and\
153
- \ fill up the tokens of our interest as we proceed in this section. >>> key_tokens\
154
- \ = [] # => currently 'available' token doesn't have anything to attend Nearby\
155
- \ tokens should be important because, in a sentence (sequence of words), the current\
156
- \ word is highly dependent on neighboring past & future tokens. This intuition\
157
- \ is the idea behind the concept of sliding attention."
 
 
 
 
 
 
 
 
 
 
 
 
158
  example_title: bigbird blog intro
159
- - text: "To be fair, you have to have a very high IQ to understand Rick and Morty.\
160
- \ The humour is extremely subtle, and without a solid grasp of theoretical physics\
161
- \ most of the jokes will go over a typical viewer's head. There's also Rick's\
162
- \ nihilistic outlook, which is deftly woven into his characterisation- his personal\
163
- \ philosophy draws heavily from Narodnaya Volya literature, for instance. The\
164
- \ fans understand this stuff; they have the intellectual capacity to truly appreciate\
165
- \ the depths of these jokes, to realise that they're not just funny- they say\
166
- \ something deep about LIFE. As a consequence people who dislike Rick & Morty\
167
- \ truly ARE idiots- of course they wouldn't appreciate, for instance, the humour\
168
- \ in Rick's existential catchphrase 'Wubba Lubba Dub Dub,' which itself is a cryptic\
169
- \ reference to Turgenev's Russian epic Fathers and Sons. I'm smirking right now\
170
- \ just imagining one of those addlepated simpletons scratching their heads in\
171
- \ confusion as Dan Harmon's genius wit unfolds itself on their television screens.\
172
- \ What fools.. how I pity them. \U0001F602\nAnd yes, by the way, i DO have a Rick\
173
- \ & Morty tattoo. And no, you cannot see it. It's for the ladies' eyes only- and\
174
- \ even then they have to demonstrate that they're within 5 IQ points of my own\
175
- \ (preferably lower) beforehand. Nothin personnel kid \U0001F60E"
 
 
176
  example_title: Richard & Mortimer
177
  parameters:
178
  max_length: 64
@@ -195,30 +208,36 @@ model-index:
195
  config: kmfoda--booksum
196
  split: test
197
  metrics:
198
- - name: ROUGE-1
199
- type: rouge
200
  value: 23.1439
 
201
  verified: true
202
- - name: ROUGE-2
203
- type: rouge
204
  value: 3.2393
 
205
  verified: true
206
- - name: ROUGE-L
207
- type: rouge
208
  value: 12.7038
 
209
  verified: true
210
- - name: ROUGE-LSUM
211
- type: rouge
212
  value: 19.8101
 
213
  verified: true
214
- - name: loss
215
- type: loss
216
  value: 2.766307830810547
 
217
  verified: true
218
- - name: gen_len
219
- type: gen_len
220
  value: 63.4493
 
221
  verified: true
 
222
  - task:
223
  type: summarization
224
  name: Summarization
@@ -228,30 +247,36 @@ model-index:
228
  config: samsum
229
  split: test
230
  metrics:
231
- - name: ROUGE-1
232
- type: rouge
233
  value: 26.8026
 
234
  verified: true
235
- - name: ROUGE-2
236
- type: rouge
237
  value: 6.0656
 
238
  verified: true
239
- - name: ROUGE-L
240
- type: rouge
241
  value: 20.0098
 
242
  verified: true
243
- - name: ROUGE-LSUM
244
- type: rouge
245
  value: 21.9115
 
246
  verified: true
247
- - name: loss
248
- type: loss
249
  value: 2.317471981048584
 
250
  verified: true
251
- - name: gen_len
252
- type: gen_len
253
  value: 19.1111
 
254
  verified: true
 
255
  - task:
256
  type: summarization
257
  name: Summarization
@@ -261,30 +286,36 @@ model-index:
261
  config: default
262
  split: test
263
  metrics:
264
- - name: ROUGE-1
265
- type: rouge
266
  value: 25.2061
 
267
  verified: true
268
- - name: ROUGE-2
269
- type: rouge
270
  value: 4.7048
 
271
  verified: true
272
- - name: ROUGE-L
273
- type: rouge
274
  value: 17.8593
 
275
  verified: true
276
- - name: ROUGE-LSUM
277
- type: rouge
278
  value: 18.0798
 
279
  verified: true
280
- - name: loss
281
- type: loss
282
  value: 3.003053665161133
 
283
  verified: true
284
- - name: gen_len
285
- type: gen_len
286
  value: 27.4815
 
287
  verified: true
 
288
  - task:
289
  type: summarization
290
  name: Summarization
@@ -294,30 +325,36 @@ model-index:
294
  config: 3.0.0
295
  split: test
296
  metrics:
297
- - name: ROUGE-1
298
- type: rouge
299
  value: 27.5692
 
300
  verified: true
301
- - name: ROUGE-2
302
- type: rouge
303
  value: 6.1264
 
304
  verified: true
305
- - name: ROUGE-L
306
- type: rouge
307
  value: 17.1127
 
308
  verified: true
309
- - name: ROUGE-LSUM
310
- type: rouge
311
  value: 23.0066
 
312
  verified: true
313
- - name: loss
314
- type: loss
315
  value: 2.218526601791382
 
316
  verified: true
317
- - name: gen_len
318
- type: gen_len
319
  value: 39.1952
 
320
  verified: true
 
321
  - task:
322
  type: summarization
323
  name: Summarization
@@ -327,30 +364,36 @@ model-index:
327
  config: default
328
  split: test
329
  metrics:
330
- - name: ROUGE-1
331
- type: rouge
332
  value: 28.0632
 
333
  verified: true
334
- - name: ROUGE-2
335
- type: rouge
336
  value: 9.8996
 
337
  verified: true
338
- - name: ROUGE-L
339
- type: rouge
340
  value: 18.25
 
341
  verified: true
342
- - name: ROUGE-LSUM
343
- type: rouge
344
  value: 21.9053
 
345
  verified: true
346
- - name: loss
347
- type: loss
348
  value: 2.032966375350952
 
349
  verified: true
350
- - name: gen_len
351
- type: gen_len
352
  value: 48.5987
 
353
  verified: true
 
354
  - task:
355
  type: summarization
356
  name: Summarization
@@ -360,30 +403,36 @@ model-index:
360
  config: y
361
  split: test
362
  metrics:
363
- - name: ROUGE-1
364
- type: rouge
365
  value: 34.7848
 
366
  verified: true
367
- - name: ROUGE-2
368
- type: rouge
369
  value: 9.7549
 
370
  verified: true
371
- - name: ROUGE-L
372
- type: rouge
373
  value: 22.228
 
374
  verified: true
375
- - name: ROUGE-LSUM
376
- type: rouge
377
  value: 28.0389
 
378
  verified: true
379
- - name: loss
380
- type: loss
381
  value: 1.7787292003631592
 
382
  verified: true
383
- - name: gen_len
384
- type: gen_len
385
  value: 71.6372
 
386
  verified: true
 
387
  - task:
388
  type: summarization
389
  name: Summarization
@@ -393,30 +442,36 @@ model-index:
393
  config: plain_text
394
  split: validation
395
  metrics:
396
- - name: ROUGE-1
397
- type: rouge
398
  value: 23.5925
 
399
  verified: true
400
- - name: ROUGE-2
401
- type: rouge
402
  value: 5.6762
 
403
  verified: true
404
- - name: ROUGE-L
405
- type: rouge
406
  value: 13.8108
 
407
  verified: true
408
- - name: ROUGE-LSUM
409
- type: rouge
410
  value: 20.2437
 
411
  verified: true
412
- - name: loss
413
- type: loss
414
  value: 2.6377077102661133
 
415
  verified: true
416
- - name: gen_len
417
- type: gen_len
418
  value: 64.1807
 
419
  verified: true
 
420
  - task:
421
  type: summarization
422
  name: Summarization
@@ -426,30 +481,36 @@ model-index:
426
  config: plain_text
427
  split: test
428
  metrics:
429
- - name: ROUGE-1
430
- type: rouge
431
  value: 23.7438
 
432
  verified: true
433
- - name: ROUGE-2
434
- type: rouge
435
  value: 5.501
 
436
  verified: true
437
- - name: ROUGE-L
438
- type: rouge
439
  value: 13.8132
 
440
  verified: true
441
- - name: ROUGE-LSUM
442
- type: rouge
443
  value: 20.4615
 
444
  verified: true
445
- - name: loss
446
- type: loss
447
  value: 2.6383883953094482
 
448
  verified: true
449
- - name: gen_len
450
- type: gen_len
451
  value: 64.9085
 
452
  verified: true
 
453
  ---
454
 
455
  # README - long-t5-tglobal-base-16384-booksum-V11-big_patent-V2
 
1
  ---
2
+ license:
3
+ - apache-2.0
4
+ - bsd-3-clause
5
  tags:
6
  - summarization
7
  - summary
8
  - booksum
9
  - long-document
10
  - long-form
 
 
 
11
  datasets:
12
  - kmfoda/booksum
13
  - big_patent
 
27
  deviation of the average recurrence interval, the more specific could be the long
28
  term prediction of a future mainshock.
29
  example_title: earthquakes
30
+ - text: ' A typical feed-forward neural field algorithm. Spatiotemporal coordinates
31
+ are fed into a neural network that predicts values in the reconstructed domain.
32
+ Then, this domain is mapped to the sensor domain where sensor measurements are
33
+ available as supervision. Class and Section Problems Addressed Generalization
34
+ (Section 2) Inverse problems, ill-posed problems, editability; symmetries. Hybrid
35
+ Representations (Section 3) Computation & memory efficiency, representation capacity,
36
+ editability: Forward Maps (Section 4) Inverse problems Network Architecture (Section
37
+ 5) Spectral bias, integration & derivatives. Manipulating Neural Fields (Section
38
+ 6) Edit ability, constraints, regularization. Table 2: The five classes of techniques
39
+ in the neural field toolbox each addresses problems that arise in learning, inference,
40
+ and control. (Section 3). We can supervise reconstruction via differentiable forward
41
+ maps that transform Or project our domain (e.g, 3D reconstruction via 2D images;
42
+ Section 4) With appropriate network architecture choices, we can overcome neural
43
+ network spectral biases (blurriness) and efficiently compute derivatives and integrals
44
+ (Section 5). Finally, we can manipulate neural fields to add constraints and regularizations,
45
+ and to achieve editable representations (Section 6). Collectively, these classes
46
+ constitute a ''toolbox'' of techniques to help solve problems with neural fields
47
+ There are three components in a conditional neural field: (1) An encoder or inference
48
+ function that outputs the conditioning latent variable 2 given an observation
49
+ 0 E(0) =2. 2 is typically a low-dimensional vector, and is often referred to aS
50
+ a latent code Or feature code_ (2) A mapping function 4 between Z and neural field
51
+ parameters O: Y(z) = O; (3) The neural field itself $. The encoder € finds the
52
+ most probable z given the observations O: argmaxz P(2/0). The decoder maximizes
53
+ the inverse conditional probability to find the most probable 0 given Z: arg-
54
+ max P(Olz). We discuss different encoding schemes with different optimality guarantees
55
+ (Section 2.1.1), both global and local conditioning (Section 2.1.2), and different
56
+ mapping functions Y (Section 2.1.3) 2. Generalization Suppose we wish to estimate
57
+ a plausible 3D surface shape given a partial or noisy point cloud. We need a suitable
58
+ prior over the sur- face in its reconstruction domain to generalize to the partial
59
+ observations. A neural network expresses a prior via the function space of its
60
+ architecture and parameters 0, and generalization is influenced by the inductive
61
+ bias of this function space (Section 5).'
 
62
  example_title: scientific paper
63
  - text: 'Is a else or outside the cob and tree written being of early client rope
64
  and you have is for good reasons. On to the ocean in Orange for time. By''s the
 
110
  the point of you of your model. This hidden data is complete by unseen. In other
111
  words, we solve our problem of validation.'
112
  example_title: transcribed audio - lecture
113
+ - text: 'Transformer-based models have shown to be very useful for many NLP tasks.
114
+ However, a major limitation of transformers-based models is its O(n^2)O(n 2) time
115
+ & memory complexity (where nn is sequence length). Hence, it''s computationally
116
+ very expensive to apply transformer-based models on long sequences n > 512n>512.
117
+ Several recent papers, e.g. Longformer, Performer, Reformer, Clustered attention
118
+ try to remedy this problem by approximating the full attention matrix. You can
119
+ checkout 🤗''s recent blog post in case you are unfamiliar with these models.
120
+
121
+ BigBird (introduced in paper) is one of such recent models to address this issue.
122
+ BigBird relies on block sparse attention instead of normal attention (i.e. BERT''s
123
+ attention) and can handle sequences up to a length of 4096 at a much lower computational
124
+ cost compared to BERT. It has achieved SOTA on various tasks involving very long
125
+ sequences such as long documents summarization, question-answering with long contexts.
126
+
127
+ BigBird RoBERTa-like model is now available in 🤗Transformers. The goal of this
128
+ post is to give the reader an in-depth understanding of big bird implementation
129
+ & ease one''s life in using BigBird with 🤗Transformers. But, before going into
130
+ more depth, it is important to remember that the BigBird''s attention is an approximation
131
+ of BERT''s full attention and therefore does not strive to be better than BERT''s
132
+ full attention, but rather to be more efficient. It simply allows to apply transformer-based
133
+ models to much longer sequences since BERT''s quadratic memory requirement quickly
134
+ becomes unbearable. Simply put, if we would have compute & time, BERT''s attention
135
+ would be preferred over block sparse attention (which we are going to discuss
136
+ in this post).
137
+
138
+ If you wonder why we need more compute when working with longer sequences, this
139
+ blog post is just right for you!
140
+
141
+ Some of the main questions one might have when working with standard BERT-like
142
+ attention include:
143
+
144
+ Do all tokens really have to attend to all other tokens? Why not compute attention
145
+ only over important tokens? How to decide what tokens are important? How to attend
146
+ to just a few tokens in a very efficient way? In this blog post, we will try to
147
+ answer those questions.
148
+
149
+ What tokens should be attended to? We will give a practical example of how attention
150
+ works by considering the sentence ''BigBird is now available in HuggingFace for
151
+ extractive question answering''. In BERT-like attention, every word would simply
152
+ attend to all other tokens.
153
+
154
+ Let''s think about a sensible choice of key tokens that a queried token actually
155
+ only should attend to by writing some pseudo-code. Will will assume that the token
156
+ available is queried and build a sensible list of key tokens to attend to.
157
+
158
+ >>> # let''s consider following sentence as an example >>> example = [''BigBird'',
159
+ ''is'', ''now'', ''available'', ''in'', ''HuggingFace'', ''for'', ''extractive'',
160
+ ''question'', ''answering'']
161
+
162
+ >>> # further let''s assume, we''re trying to understand the representation of
163
+ ''available'' i.e. >>> query_token = ''available'' >>> # We will initialize an
164
+ empty `set` and fill up the tokens of our interest as we proceed in this section.
165
+ >>> key_tokens = [] # => currently ''available'' token doesn''t have anything
166
+ to attend Nearby tokens should be important because, in a sentence (sequence of
167
+ words), the current word is highly dependent on neighboring past & future tokens.
168
+ This intuition is the idea behind the concept of sliding attention.'
169
  example_title: bigbird blog intro
170
+ - text: 'To be fair, you have to have a very high IQ to understand Rick and Morty.
171
+ The humour is extremely subtle, and without a solid grasp of theoretical physics
172
+ most of the jokes will go over a typical viewer''s head. There''s also Rick''s
173
+ nihilistic outlook, which is deftly woven into his characterisation- his personal
174
+ philosophy draws heavily from Narodnaya Volya literature, for instance. The fans
175
+ understand this stuff; they have the intellectual capacity to truly appreciate
176
+ the depths of these jokes, to realise that they''re not just funny- they say something
177
+ deep about LIFE. As a consequence people who dislike Rick & Morty truly ARE idiots-
178
+ of course they wouldn''t appreciate, for instance, the humour in Rick''s existential
179
+ catchphrase ''Wubba Lubba Dub Dub,'' which itself is a cryptic reference to Turgenev''s
180
+ Russian epic Fathers and Sons. I''m smirking right now just imagining one of those
181
+ addlepated simpletons scratching their heads in confusion as Dan Harmon''s genius
182
+ wit unfolds itself on their television screens. What fools.. how I pity them.
183
+ 😂
184
+
185
+ And yes, by the way, i DO have a Rick & Morty tattoo. And no, you cannot see it.
186
+ It''s for the ladies'' eyes only- and even then they have to demonstrate that
187
+ they''re within 5 IQ points of my own (preferably lower) beforehand. Nothin personnel
188
+ kid 😎'
189
  example_title: Richard & Mortimer
190
  parameters:
191
  max_length: 64
 
208
  config: kmfoda--booksum
209
  split: test
210
  metrics:
211
+ - type: rouge
 
212
  value: 23.1439
213
+ name: ROUGE-1
214
  verified: true
215
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmQzMDk0MDJlZTJkN2IzODg3NDJhYmY4MzJmOTU4N2FjMDBjODg5NzJlMGFhNDQ2YTFhMzI3YmY5ZWM1MDBkMiIsInZlcnNpb24iOjF9.yoXEV5ircj_cjQhUA_RpWH_8Kaev0sRLwQulYD8wmqxfSEuqamBGedXnIg9X_EcpjvulBhapjGZN2G0s0vz4Dg
216
+ - type: rouge
217
  value: 3.2393
218
+ name: ROUGE-2
219
  verified: true
220
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTkwNzEwYjc5YTZkMmE4NmEwMDE1OTRiNTJmM2VlYmI3NmM2NjIwZWMxM2ZkNjU2MzhjMmQzYjIxODRiYzY4ZiIsInZlcnNpb24iOjF9.CDK_e4fCwERbm3D_Y2tc41SSscIvlZKGTUQ16afpMuH2_HHKbpn7CNgtU9MWiyFZfdgafdUeQPo2CCYI-dCBCg
221
+ - type: rouge
222
  value: 12.7038
223
+ name: ROUGE-L
224
  verified: true
225
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDFkNjcyYmYxYzdlMTY2NTIyY2ZiZDJlZjliYTM1YWZjZGI3YzA5ZDczYjdkMGUzZmUxNmJkMDY0OTk3NWNlMSIsInZlcnNpb24iOjF9.XQmt4GEX0N6y2FNXfLAeLDkB96nJyxhN9dyy-OdBcu5E7Tw0dvIN3feYHxq8MenTShE9lsekIYZy2kieJQfmCg
226
+ - type: rouge
227
  value: 19.8101
228
+ name: ROUGE-LSUM
229
  verified: true
230
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTFhMGNhMzA0YmYyMDhiNzdlMDc2ZDQ3YjFjMDM3ODliMmIxMjQxZWMwYWM0NTM0OGNlZTkzMzVhZDBmMjA1YiIsInZlcnNpb24iOjF9.-YChaP7xwLM9W5jrdLSyLWdb3hAdPbm0mmij3X_pU3nqb3_wuPobjCLGEEQNxAnGq7kE-LI5hgXZ-lGhuKUCCQ
231
+ - type: loss
232
  value: 2.766307830810547
233
+ name: loss
234
  verified: true
235
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODAxYzRhNGM2ZGVkOWRiM2Y4NzNjZDM2MTY2MmM4MzY3ZWM5ZjdmMWUxZGY5Y2E2OTg4ZGEwYzBlMmFiYmQyNSIsInZlcnNpb24iOjF9.VRePqe8Z9dD5l6bsfIRLkFn4mwwVC8G--kOlofQWSiGusRxVrY50fa5MtKTGmuiNs5JDFCPjZmkpGYlSxnOeDw
236
+ - type: gen_len
237
  value: 63.4493
238
+ name: gen_len
239
  verified: true
240
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGY4NWI0MDc3NDk4NTg4YjQ5YzFmN2MyYWFjMzI0MjlkMGZlMWMzYThiMDFlMmM3MmE4ODg0YWExNTMyZjQ5MiIsInZlcnNpb24iOjF9.Ym3jfW0gthJhlLg4CW10jM9YUHUGbAPIdLefE3CTyP0OUrV9yuJAGV6-RDrV-Viwyy1Xaqg4BFa5pX7P2PRRDQ
241
  - task:
242
  type: summarization
243
  name: Summarization
 
247
  config: samsum
248
  split: test
249
  metrics:
250
+ - type: rouge
 
251
  value: 26.8026
252
+ name: ROUGE-1
253
  verified: true
254
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTBhYTQzMGVjZTJjZmE3NjBiNzI2M2FlNTA4Yzk5Njc1Yjk1YTk2NTJiMTRlMzQ3NjU2ZjQxZTNkNDVhNjMzYSIsInZlcnNpb24iOjF9.GyFUubKI3pM5Z8I1jz6Q_f7fSr1nVpwuFluUOVq8aaWfv7L1dZ_5By2FShQM1nwBM-mCiqtFb3a61eR3VEAeBw
255
+ - type: rouge
256
  value: 6.0656
257
+ name: ROUGE-2
258
  verified: true
259
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzEyZTYxYmVlYTc0MzNhMWM1ODgwODRiYWNkN2FjMjIzOTJhNzA0OTFkY2M0ZTJhMWMzNWMzY2E1OGJmYTg5OCIsInZlcnNpb24iOjF9.3U0PamPVFWWE7Nxh6u52mnMP-HpeGPEOLauZthcj32ELSuNx9s260ujguSW_BrJpCXqNNEqIzYTlWf97Ji8vCA
260
+ - type: rouge
261
  value: 20.0098
262
+ name: ROUGE-L
263
  verified: true
264
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOGExYTRmZDgzYzllNWZmMGFlN2FhMDJmZGE1ODkyYTZlNmFhZjZmNGU4YzQwZGZiYTAyZmI1NGJmNjRjODkwYSIsInZlcnNpb24iOjF9.dEON7kZa7dKCHjz7nuuIBdcpwojM5-OxQuEf5n18ZywWdbk9H2LWGY2uvvCRp6cK2JsIzxzTmX9wK7zkWQiCAA
265
+ - type: rouge
266
  value: 21.9115
267
+ name: ROUGE-LSUM
268
  verified: true
269
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2Y4MWE4ZmIyMTA5YWU5YzllYzExMzA1OTc2Mjg3NTYxNjcwMWMxZGI0ZDhmYjJhMGIxNTllY2Q3NDVlNmM2MiIsInZlcnNpb24iOjF9.M8bYXCuNHyVAkA4vBbqvGe8yCgmjCrlhqqliAF6WcmrYRF8CvezQ4S4SWGhhVkcG6v84H-Pa9LzsKmualXdWBw
270
+ - type: loss
271
  value: 2.317471981048584
272
+ name: loss
273
  verified: true
274
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmI1YjNlYzI3OTY4YjY1MDIwYzk3ZDMzZDA4MzQwM2ZhNzY3NDQxZTA2ZThiMmE2MmFmNTg0OGMyYWFhODE5OSIsInZlcnNpb24iOjF9.QpoWo_TLKw72_PbtwknBA1LbUQ8ftls-8VBLuN8_ZhUN2lNNpipU2qMZ1Ga4xAUazkcMhT_TwpqjyGshJFkgAg
275
+ - type: gen_len
276
  value: 19.1111
277
+ name: gen_len
278
  verified: true
279
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTA2MmFiNjI5NzFjOTUzMTEwZTNiYzA1OGY1ZWEyNTE1ZTgzYjMxNDE4YjJkZmIxNWI4MDMyYWUxMWRkODk1NCIsInZlcnNpb24iOjF9.CXy-Dfle9ypabrK3I1GyhOWl46EyRDbf8XlY-D0cNktXcCCbKdgn8DWgJI199GJpH-19mMS_jQt049VJri2EDw
280
  - task:
281
  type: summarization
282
  name: Summarization
 
286
  config: default
287
  split: test
288
  metrics:
289
+ - type: rouge
 
290
  value: 25.2061
291
+ name: ROUGE-1
292
  verified: true
293
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjZmZDRlN2NjZTQyNzkyMmZiYzk1MjJmMmE0MGM4ZjUwOGNmOGFhZjg0MzE0MzM4MmE1Y2EyYTY4ZThmNzUzMiIsInZlcnNpb24iOjF9.pdJWpUnMeqftinZrPkkFRWbCA253BYgt5W-EqbyTVi9BteojJ6yEDbMjE0TyYzlJ28JBcw4IVNL2zaWCgpfRBQ
294
+ - type: rouge
295
  value: 4.7048
296
+ name: ROUGE-2
297
  verified: true
298
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGRjOGUzZTk1ZDc0Zjk5MmE4ZjUzNmZiZjQ2YzE2YzYzODdmYmY3NzMwNDdmYmViNjVkZTUzMmY4YjllOGQ1NCIsInZlcnNpb24iOjF9.nFiT7HhUZSDofK6_UH2-1rzPz_48w7e5j0Q72vqgodSNIwpv2JOlcb1GOlaA9jkvy45PJyDBgP9i6kLVfaNBBw
299
+ - type: rouge
300
  value: 17.8593
301
+ name: ROUGE-L
302
  verified: true
303
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmY5ZjM0ZjdkYTZiMzk0ZWYyM2EzZWNjMjczMjI2MzkwYmNiN2JhNDEzNzdmMmE0NzEwNmVkNGU5YTlkZDAzYyIsInZlcnNpb24iOjF9.C3ZgUsGNNtwZVJFcT90KkBfewrrA3ZXxxVl2u5ykUtzpS4gzoaRuZbPT8WOJAog7kfPPJiG_GZGYy9XTTCdIBw
304
+ - type: rouge
305
  value: 18.0798
306
+ name: ROUGE-LSUM
307
  verified: true
308
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDU4Y2Y3MzExNzNlZTI3NWVmZTNjMmZkNTAxNDBjMzJiZTI5M2E2N2ViODk5OGEwZGU5NzYxZWMzMjMwNmQ2MSIsInZlcnNpb24iOjF9.qDLZsjtftvlw8-3kOoUvanWmemmvaPxUIAxOVh1B18Ihn9kkm0FnZbWxl65YdOLg3dqDcHnDFXvXcS81C8dmBw
309
+ - type: loss
310
  value: 3.003053665161133
311
+ name: loss
312
  verified: true
313
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTM2ODRkMjk5MjczY2ViZGVjMjJjOTFmYTk2NTAyNmUwMTRiZjYwZTllY2NhODFhYWVkZTIzYzQxZjZlOGFkNCIsInZlcnNpb24iOjF9.3SeJzRO0b4cNCTOgsf7c8UrLCLW-6JoOHtNMmMr5DCzNzfqlt2TSJ5ClahzzAYA2_5QhTMhcUYOewH5uZhkpDA
314
+ - type: gen_len
315
  value: 27.4815
316
+ name: gen_len
317
  verified: true
318
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDdiYTVkZGI0NzE0ODcwNjgwNGQ0YmNjZDI1MWQxZWQ0MzNmMDJkYmE4MGM5ZjM4NGViNWZiNTdjNTg2YzBlOSIsInZlcnNpb24iOjF9.VoPyoq8HZq8nbucrPYt52flRFtkD5VAfVD7LykAp-GiN2W6D3cpcagMMrHThP9e8q3qDodxddMcnwY88CGtkAg
319
  - task:
320
  type: summarization
321
  name: Summarization
 
325
  config: 3.0.0
326
  split: test
327
  metrics:
328
+ - type: rouge
 
329
  value: 27.5692
330
+ name: ROUGE-1
331
  verified: true
332
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiM2UzNDRjNDJhNjViYjgxNDY2NzAwODkyYjk1OTllNWFiYmI2MGEyMmM3ZTc1YWZjNjhiZDZkYzAxYzIwYTQzZiIsInZlcnNpb24iOjF9.FEJU7de6nnYa1rhAngf3h0JDSFKXzWKkcHwQtcz6rbPuVV0Jw7u-9PwDXBFh0X8n2PJjOfCqM5hmcrUe0FxkCQ
333
+ - type: rouge
334
  value: 6.1264
335
+ name: ROUGE-2
336
  verified: true
337
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMGIzODA2ZjU2YmM0YmJiZDIxNzQ0ZDI1NGQzZGZlNDg5OTZhYmMwZTQ1ZjVlYzM5ZTQzMjZkMTIyZmY1OGQ2YiIsInZlcnNpb24iOjF9.fN1wSGc_tUvIgYyzKU35PuPxKyTOotKnMCW_u452LduRVyIey9KB8kf8E35vTOVvk7TCiuvRuxXDoAATFktbBQ
338
+ - type: rouge
339
  value: 17.1127
340
+ name: ROUGE-L
341
  verified: true
342
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWRjNTNhZjg1NDVkNTQ5MjkwZjNiNzY0Nzk5ZmM4YjhhZmZiZjQzZGY1YWM1ZGI5MGE0YjNiYzNmNWYyNWI2OSIsInZlcnNpb24iOjF9.KVGdIERnuGTOrxm71i2znI8tdRCgVz7SijP08tsE0H54eUijAYDqQccspfZTXRXeFn0lOUjSHDvHj4ODIRYvAw
343
+ - type: rouge
344
  value: 23.0066
345
+ name: ROUGE-LSUM
346
  verified: true
347
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGUyMzhlODY1YWI4ZDg2NzYwZDYwNmYzZTRhMTQ3NDE2MzUzZGViNzhjMTkzZDRhNTljNDEyMTY4NzAwMjE0OCIsInZlcnNpb24iOjF9.pBz5E_1ffBrv4tDCJhuYFIuBFBk0P3SKxLYoIhOVj_fW0Mj6ZKPcA9ZhdE4U-HsHEgSvFhtBw1UlsGiu145XBw
348
+ - type: loss
349
  value: 2.218526601791382
350
+ name: loss
351
  verified: true
352
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjYxNDk4OWU0M2Y1ZjMxNTA3NjdiNjQ5NWFjYzJiMjVhMjgzMTA3NDhlNTVjMjllZjQ0NWQ2YmYzYjdiMTQ1OCIsInZlcnNpb24iOjF9.SJdyGLltcLnB03U6QxSkZ71Im0aGK-oTbEQDMj2AnEPFThNTb0mMEMpCWpH1lLVeDAh-PE6fCmgt4yPS6n2nBg
353
+ - type: gen_len
354
  value: 39.1952
355
+ name: gen_len
356
  verified: true
357
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTMyY2JiYWVhYTM3OWU2YjhiMDEwZjAwZDgxN2FmNjk2MzZhYmQzMWZiODg2NDY0ZmU4MjdhNjk0MTViMGY1YyIsInZlcnNpb24iOjF9.bsLAi2R8QTrCUj1VW4GQqYauY7CV3mFm2S294zHCJU2ZlAcikutcjxAcgvuSSGiAVJ02Odm5bMTuzx7SYMUSAQ
358
  - task:
359
  type: summarization
360
  name: Summarization
 
364
  config: default
365
  split: test
366
  metrics:
367
+ - type: rouge
 
368
  value: 28.0632
369
+ name: ROUGE-1
370
  verified: true
371
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2RiODA0ZTQxYWU0NDI5YmNjZmYzYTZmY2I5NTczYzVhZjcxOTYwMWI3ZjZiMzRlZmI5ZTA5NjVkY2E4NDFlMyIsInZlcnNpb24iOjF9.POIQUXGryoEzHmdBCeqaBh70uz33XlKVLjfhyRFwhWj7UV15SsDcuumkEk2BXkShFHDRo0CQd1AXD1fFsPCVCQ
372
+ - type: rouge
373
  value: 9.8996
374
+ name: ROUGE-2
375
  verified: true
376
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDBiMDllNTZlZmJiYWI1ZTIxM2JhYmZhYTAzYTQ0NmUzNjcyZjkzMDliYTE5ZjIwY2M0YzU2ZWZlYjNhZDY2YyIsInZlcnNpb24iOjF9.EEJO-ZRVi2EiM-uKMvimaITiHh7wqzNBza6lsIvdyVhVf4UwGhsUaArHzlYR7xn53UBCtIDTucXX7NKFst_4Ag
377
+ - type: rouge
378
  value: 18.25
379
+ name: ROUGE-L
380
  verified: true
381
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTk4ZmJiYWIwYmY4MTBmNGVlMmE1YzA4N2VmYWU3NjRlNTU3YjI2YjBhOGIzNzcwZjczOTZmZGJiNjMyMjYzZiIsInZlcnNpb24iOjF9.Qx-ihTp0UuzhShqHQkiTijODUst1LO5Bi8KaQOCIiVhvywN-2Wt3bmeSNV_C0b5BXsSaHIxrWBTeSRaq5Zp_Bw
382
+ - type: rouge
383
  value: 21.9053
384
+ name: ROUGE-LSUM
385
  verified: true
386
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTIzNGNkNTAyYTkzZjE5ZGZhZjZkYmU3Yjg2ZTVhYjY1NjZhODZjM2NkMWQ5NmJjN2UxNTZlMmJmNDNmOTczZSIsInZlcnNpb24iOjF9.6ZY8rK5bRfOZJkdvhpvOt_gW1xCoA4JsAi0-6No4y-lBaLGUo4LXpGaVcJrrvdN-S7e7yCxnA32jGCdYXzJJBA
387
+ - type: loss
388
  value: 2.032966375350952
389
+ name: loss
390
  verified: true
391
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTM5MmQzMWZhOWIwNjNjNThhNGE4NzFiMzdhNmMzZWM4ZGYyNWE1NmZjMDVjNTBmMGRiNzYzMTc1ZDg2YTYxNCIsInZlcnNpb24iOjF9.Zqrbz7mmljH19mEep_mm4ev5FEsozIqG0eNkj3V5s85OgrHyuKOVkGKhRlqjcWfgkUlsxTpaemZDUVIR84XrBw
392
+ - type: gen_len
393
  value: 48.5987
394
+ name: gen_len
395
  verified: true
396
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODZjNGJiOGUzM2M3NDM3MDRmNmQ1ZjQ3ODUyZTQ2NjEyYmM0YWRhMmU4MDdkOTZmMGNkNTIyNDg3ZmQxMjA4MiIsInZlcnNpb24iOjF9.y91kl4W-QIy6lfQDl0s4h0aeV-v0iH7Y06AJBYRYrddUXRiDw2oSTHEdf14d3Hw-oZNPftzBHUJqAckwEpGFDw
397
  - task:
398
  type: summarization
399
  name: Summarization
 
403
  config: y
404
  split: test
405
  metrics:
406
+ - type: rouge
 
407
  value: 34.7848
408
+ name: ROUGE-1
409
  verified: true
410
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiN2QxOTU1YTUxYWJjOTcwOGYwZjA3MGJlYmRhYWUwMGNhYmQxYjVmMjU5M2E5OGRiY2RmMTNmNGNhZDdmNzc1OCIsInZlcnNpb24iOjF9.bp2K7V-BDMQMd3zk2RY3pILKI7LimWrD5psesXnSF20JiRA3d5-bQdOfPeZGu3ydUqbml3MTswM0lg_6EZTjAw
411
+ - type: rouge
412
  value: 9.7549
413
+ name: ROUGE-2
414
  verified: true
415
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOGY4OWM4MjVmMzhmNGUwYzAxODNjMjY4OTY1YjQ2MjZiYzM2NzgyNGZhMjVlNjllZmI3OTMzNTVhZDA1YzMyOSIsInZlcnNpb24iOjF9.HQ_emvr1RVEfeNfQUdfhfzk2O5BGwzpQKojvRW_w44Ixakn_VrZ4GurxYo0JTF4dDwDBDqjaFnZ4EiYcsrxODQ
416
+ - type: rouge
417
  value: 22.228
418
+ name: ROUGE-L
419
  verified: true
420
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWVkMzc2ODM1ZTg2YzQ4YjMzZjQwMThiODI0YzA5MzJmZjY1ZTJlOGZhOTM1OWEzOTE3M2ExYzFiMjM2NDRlMSIsInZlcnNpb24iOjF9.shmWrR-rNKAYOqEgnnlrgWXaWAWbvrKC_IyvK-fwnqoJcphB9ef2gVX758tQgfe878M1N1sE7StT8rd7FbD8Cw
421
+ - type: rouge
422
  value: 28.0389
423
+ name: ROUGE-LSUM
424
  verified: true
425
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTJmZTczZDc4N2ZlNDk3NmY0Njc2Y2JhNGU2OWJjZGU4YWQ3Y2RjNDU1ZTEyNjFiZDQxZGNhZWFmYTAwOTBiMSIsInZlcnNpb24iOjF9.yOTMgX1vpuhlyPkfCAyNf1k5nSInny0YrbqJeC_MDZlavVIxOQT6qVcMYJpLF2AKRp6UsuFB06PANbQu4Bj6CA
426
+ - type: loss
427
  value: 1.7787292003631592
428
+ name: loss
429
  verified: true
430
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2VlMGU3NDE0NmFiNTI2M2NhZmU2YzRhYjU1ZWNjYmM3YTllMTQxODJhM2JlMjk3NzVjYjQ5M2FlOTk2NjNmZCIsInZlcnNpb24iOjF9.wkkUrosSgGkei41n6CxQH_UwS6fJTMzXLV88EgnI_8Y6Qz2qa9B2cGhpFkP__snnX6u9jhWj68oAfZifqaXnCw
431
+ - type: gen_len
432
  value: 71.6372
433
+ name: gen_len
434
  verified: true
435
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODI0NTcwOTZmNzkwOTFhMTdmYWMxYjI2YTdmMTYwYTBlMTEyOTc3MmFkYmZkNGJmYjc4MTJlYmYwNzIxMjkzMCIsInZlcnNpb24iOjF9.EM9Vh5Mb6H3htv45ohj6FYqBUtmQi1sn0j97brEFWRYp--8N2Y781cR9ktqylEz6PgbbwpuxMYOMD5MctmGLCw
436
  - task:
437
  type: summarization
438
  name: Summarization
 
442
  config: plain_text
443
  split: validation
444
  metrics:
445
+ - type: rouge
 
446
  value: 23.5925
447
+ name: ROUGE-1
448
  verified: true
449
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjk2NWVkM2Y5NTgxYjgyYmY2YjgzYjZiN2VhZGVkOWNiYmRiZGU0ZGYwNzlkN2E3ZDk5ZjQ3ZTQyYjU5YzczYSIsInZlcnNpb24iOjF9.ScWumfXg7-ZeJEZT1IHDVt5LWMDZEihiiCux5chXP2AeRs3pWKhI6xr_D5i3CCEDeyiMzKleCASMBe4sC9LgDQ
450
+ - type: rouge
451
  value: 5.6762
452
+ name: ROUGE-2
453
  verified: true
454
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDU3MGNmMDY3YWQxNDdlMTk5MjM1NGU4M2RmNDNiYzllYmRmNTYzZGFiOGU5MjQ0YWMzYTg1OWFlNmNmMzQ5NiIsInZlcnNpb24iOjF9.9SKt_I8WGKu6bsovBR4mSTDNEaSHB1tN5RyY3JTCHYs2YQNczaKwLNPnyG2i0IbkvaPX_8EOQ7KzwQ5raUVFBg
455
+ - type: rouge
456
  value: 13.8108
457
+ name: ROUGE-L
458
  verified: true
459
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTBiZDVkYjI4ZDBlZGM2NDM4M2U2NzdjNzViNDkzZjE3YTBmYzdlNDNlMTZhZTUxNjA2NmJkODE2ZTk1MTAxMSIsInZlcnNpb24iOjF9.KMTkQsI9BfDfL7FZpwZ9kxTTRA8DNrUEpyBZtloQ0sNfhO0t0Ch1qhktz0HaA0uQfC0WFRfrb9Iz7uMc8XVRBg
460
+ - type: rouge
461
  value: 20.2437
462
+ name: ROUGE-LSUM
463
  verified: true
464
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjBkZGJlYzZjMjQ1Njg4MjQ2NzJhYjY5ZGZlN2Y5Y2M4MDQ0YzQ3YzQzYmY5N2VkNjBiNTEwMDNmZWRlMTAwYyIsInZlcnNpb24iOjF9.AqYAfIMFBY7AIP1yJbjaAbJXYs5VbXxWKpsA_rdW_HWxITvjqoJDK9X3wCueXMy7dSE6L-ysC4yl99Bbc50KBA
465
+ - type: loss
466
  value: 2.6377077102661133
467
+ name: loss
468
  verified: true
469
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGEzMTZhODM0Nzg0ZDY3OTVkYmZmODQ1Y2YzMTY3YmJlYjk2ZGRiMWFkMDQxMTkyYTgwZWNkNmU0NzI0NjA1NCIsInZlcnNpb24iOjF9.ziVXhWBRAml5Xwa-tx9ywwtiJeIzIIclY532L0Mtft3Sc88oGPK9av6nh4kMiO5yWSHJnM3KFQWiuco7w_xNDg
470
+ - type: gen_len
471
  value: 64.1807
472
+ name: gen_len
473
  verified: true
474
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDNhZTRhODgwODI1YmRlODZiY2I3YjFmY2MyZGYyYTY1MzQ5OTgwZGI1NmUwNDMwMmQ0N2Y3YmZmMzcyMTc2NSIsInZlcnNpb24iOjF9.NCVj0Uaq3-diq6pnu8EC0tgwv24NwQCgmWiqpOMvJSN17B_98z_dMbLHRzY8e_tNNVFFagiCnknoE00OqUTjDg
475
  - task:
476
  type: summarization
477
  name: Summarization
 
481
  config: plain_text
482
  split: test
483
  metrics:
484
+ - type: rouge
 
485
  value: 23.7438
486
+ name: ROUGE-1
487
  verified: true
488
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzIwMzZjOGQ5N2U3MTg3NmEwYzZkNjllNDc4NzQ4NWUxN2JmYjdiOGU2MjhkZGJhODY4NDU4N2E5ODU1NTFhMiIsInZlcnNpb24iOjF9.cJoHXGYopFoFVmQXdxu3KrG_usk1ouc0PPR6FS9HrZEbi2T5LtVANntlXmlLTXSvOEaorUyg08yot_j6j1oeCw
489
+ - type: rouge
490
  value: 5.501
491
+ name: ROUGE-2
492
  verified: true
493
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWQ3YmQ5ZTJkNmJhZGEyMTkzYjlkMWZmZGVhNGE5Y2IzYzA5OWM1NTY0NTU0MWUzYTIzNTQ0OGI3ZWZkNjlkMSIsInZlcnNpb24iOjF9.C_SbNoz5qIo0CtVPL_5jqFNZxgmJ1XE43TvVz2reog2jtlhekNfN0rvaHxT4TadAAzIgDZayeBMeNaASgmNCDA
494
+ - type: rouge
495
  value: 13.8132
496
+ name: ROUGE-L
497
  verified: true
498
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTAxODA2NmNlNjkyYTQ4YjEwOTA1ZGMyMjVlZjkzMGI3NzNiMTRkZGRmNDJjZDc2MTYxYzI3NTBlNTVjY2IxNCIsInZlcnNpb24iOjF9.UklkyvqHV3axZ_PalbPb1JZN7rgQjHjJr0ke1yDUzujrY6yBr3XpPxjFhwsEElalc1iiEgdtEZnaCbBhskdGBQ
499
+ - type: rouge
500
  value: 20.4615
501
+ name: ROUGE-LSUM
502
  verified: true
503
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmNhZDI2ODQ4MjBhZDNlZjJkMTQ1NmZjZTdjMDZlMjcwYjE4M2M5ZjIxYzA2M2JmYmJmZDliZTU3NzVkMjdmZiIsInZlcnNpb24iOjF9.m2aRMFUpPFvMSf3sxB7HbKIslWrggFamjiIlOAiPuH5_N8wyLJeHJJw8uvULE8R0GKGWuqXfCCv--lyhZKZkAA
504
+ - type: loss
505
  value: 2.6383883953094482
506
+ name: loss
507
  verified: true
508
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTQzMjFiZWE1NDI1OTFlNWUxMzFiYjJhNzViNDYxMzI3OGU2ZTE1ZDJkNDA3Y2NhODA0ZWM3ZmM3ZTM1NmFlZiIsInZlcnNpb24iOjF9.twTQ94T2Nsq0__TcHLaJ_8HcqozA_FOi6pAiM_IP5qSqKlUXYV1S2-nuS1vs69QB-tSp4XIbqRqhSgKv0VoABw
509
+ - type: gen_len
510
  value: 64.9085
511
+ name: gen_len
512
  verified: true
513
+ verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDk3Njc5ZWM3ZTRkMzk2YjJmMjg1YjFlNDExNTU2NTRhNzRlNjA4NGFkZDg2YmQzN2UzNThhODFmZTNlMjdkZiIsInZlcnNpb24iOjF9.2rXKy4mi2VbZRDewY2mKsVe42KuwxIWcmIzdA39RbSJ7Wg45MfRDUjZweyz7Bnlmy6eCcdv7Ya4oyUwAjNV3AQ
514
  ---
515
 
516
  # README - long-t5-tglobal-base-16384-booksum-V11-big_patent-V2