fyaronskiy commited on
Commit
19d7b6a
1 Parent(s): ce12729

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -19,9 +19,23 @@ metrics:
19
  - recall
20
  ---
21
 
 
22
 
 
 
 
 
 
 
 
 
23
  This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
24
- dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions. Thresholds are selected on validation set by maximizing f1 macro over all labels.
 
 
 
 
 
25
 
26
  # Usage
27
  Using model with Huggingface Transformers:
 
19
  - recall
20
  ---
21
 
22
+ This is the best russian opensource model for detecting all 27 types of emotions:
23
 
24
+ | Model | F1 macro | F1 macro weighted | Precision macro | Recall macro |
25
+ |------------------------------------------------------------------------|----------|-------------------|-----------------|--------------|
26
+ | seara/rubert-tiny2-ru-go-emotions | 0.33 | 0.48 | 0.51 | 0.29 |
27
+ | seara/rubert-base-cased-ru-go-emotions | 0.36 | 0.49 | 0.52 | 0.31 |
28
+ | fyaronskiy/ruRoberta-large-ru-go-emotions default thresholds = 0.5 | 0.41 | 0.52 | **0.58** | 0.36 |
29
+ | fyaronskiy/ruRoberta-large-ru-go-emotions best thresholds | **0.48** | **0.58** | 0.46 | **0.55** |
30
+
31
+ # Summary
32
  This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
33
+ dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions.
34
+ Thresholds are selected on validation set by maximizing f1 macro over all labels.
35
+
36
+ The quality of the model varies greatly across all classes (look at the table with metrics below). There are classes like
37
+ amusement, gratitude, where the model shows high recognition quality, and classes that pose difficulties for the model - grief, relief,
38
+ that do have much fewer examples in the training data.
39
 
40
  # Usage
41
  Using model with Huggingface Transformers: