fyaronskiy
commited on
Commit
•
19d7b6a
1
Parent(s):
ce12729
Update README.md
Browse files
README.md
CHANGED
@@ -19,9 +19,23 @@ metrics:
|
|
19 |
- recall
|
20 |
---
|
21 |
|
|
|
22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
23 |
This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
|
24 |
-
dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions.
|
|
|
|
|
|
|
|
|
|
|
25 |
|
26 |
# Usage
|
27 |
Using model with Huggingface Transformers:
|
|
|
19 |
- recall
|
20 |
---
|
21 |
|
22 |
+
This is the best russian opensource model for detecting all 27 types of emotions:
|
23 |
|
24 |
+
| Model | F1 macro | F1 macro weighted | Precision macro | Recall macro |
|
25 |
+
|------------------------------------------------------------------------|----------|-------------------|-----------------|--------------|
|
26 |
+
| seara/rubert-tiny2-ru-go-emotions | 0.33 | 0.48 | 0.51 | 0.29 |
|
27 |
+
| seara/rubert-base-cased-ru-go-emotions | 0.36 | 0.49 | 0.52 | 0.31 |
|
28 |
+
| fyaronskiy/ruRoberta-large-ru-go-emotions default thresholds = 0.5 | 0.41 | 0.52 | **0.58** | 0.36 |
|
29 |
+
| fyaronskiy/ruRoberta-large-ru-go-emotions best thresholds | **0.48** | **0.58** | 0.46 | **0.55** |
|
30 |
+
|
31 |
+
# Summary
|
32 |
This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
|
33 |
+
dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions.
|
34 |
+
Thresholds are selected on validation set by maximizing f1 macro over all labels.
|
35 |
+
|
36 |
+
The quality of the model varies greatly across all classes (look at the table with metrics below). There are classes like
|
37 |
+
amusement, gratitude, where the model shows high recognition quality, and classes that pose difficulties for the model - grief, relief,
|
38 |
+
that do have much fewer examples in the training data.
|
39 |
|
40 |
# Usage
|
41 |
Using model with Huggingface Transformers:
|