DarshanDeshpande
commited on
Commit
•
ee47592
1
Parent(s):
5303f4b
Update README.md
Browse files
README.md
CHANGED
@@ -17,8 +17,10 @@ pipeline_tag: text-generation
|
|
17 |
|
18 |
# Patronus GLIDER
|
19 |
|
|
|
|
|
20 |
GLIDER is a fine tuned phi-3.5-mini-instruct which can be used as a general purpose evaluation model to judge texts, conversations and RAG setups according to arbitrary, user defined criteria and rubric scale.
|
21 |
-
This model was trained using a combination of synthetic and domain adapted data from popular datasets like Mocha, FinQA, Realtoxicity, etc. The training data for this model covers over 183 metrics and
|
22 |
The maximum sequence length is 8192 tokens but the model can support longer texts as well (tested upto 12,000 tokens).
|
23 |
|
24 |
|
@@ -27,14 +29,14 @@ The maximum sequence length is 8192 tokens but the model can support longer text
|
|
27 |
- **Model Type:** GLIDER is a fine-tuned version of microsoft/Phi-3.5-mini-instruct model.
|
28 |
- **Language:** Primarily English but supports Korean, Kazakh, Hindi, Bengali, Spanish, Indonesian, German, French, Arabic, Russian, Thai, Turkish, Ukraninan, Romainian and more.
|
29 |
- **Developed by:** Patronus AI
|
30 |
-
- **Paper:** [
|
31 |
- **License:** [https://creativecommons.org/licenses/by-nc/4.0/](https://creativecommons.org/licenses/by-nc/4.0/)
|
32 |
|
33 |
### Model Sources
|
34 |
|
35 |
<!-- Provide the basic links for the model. -->
|
36 |
|
37 |
-
- **Repository:** [https://github.com/patronus-ai/
|
38 |
|
39 |
|
40 |
## How to Get Started with the Model
|
@@ -73,6 +75,7 @@ Your output must in the following format:
|
|
73 |
<score>
|
74 |
[The final integer score assigned based on the scoring rubric]
|
75 |
</score>
|
|
|
76 |
```
|
77 |
|
78 |
Since the model supports arbitrary number of inputs and outputs, the data can be structured in any one of the following ways:
|
@@ -134,7 +137,6 @@ Note that these XML tags can be changed according to your convenience and task
|
|
134 |
To run inference, you can use HF pipeline:
|
135 |
|
136 |
```
|
137 |
-
|
138 |
model_name = 'PatronusAI/glider'
|
139 |
pipe = pipeline(
|
140 |
"text-generation",
|
@@ -159,13 +161,21 @@ Since the model is trained in chat format, ensure that you pass the prompt as a
|
|
159 |
|
160 |
The model was evaluated on several popular datasets:
|
161 |
|
162 |
-
<img src="https://i.imgur.com/
|
163 |
|
164 |
## Citation
|
165 |
If you are using the model, cite using
|
166 |
|
167 |
```
|
168 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
169 |
```
|
170 |
|
171 |
## Model Card Contact
|
|
|
17 |
|
18 |
# Patronus GLIDER
|
19 |
|
20 |
+
<img src="https://i.imgur.com/1AbgTJa.png" alt="GLIDER" width="100%"/>
|
21 |
+
|
22 |
GLIDER is a fine tuned phi-3.5-mini-instruct which can be used as a general purpose evaluation model to judge texts, conversations and RAG setups according to arbitrary, user defined criteria and rubric scale.
|
23 |
+
This model was trained using a combination of synthetic and domain adapted data from popular datasets like Mocha, FinQA, Realtoxicity, etc. The training data for this model covers over 183 metrics and 685 domains including finance, medicine, and many more.
|
24 |
The maximum sequence length is 8192 tokens but the model can support longer texts as well (tested upto 12,000 tokens).
|
25 |
|
26 |
|
|
|
29 |
- **Model Type:** GLIDER is a fine-tuned version of microsoft/Phi-3.5-mini-instruct model.
|
30 |
- **Language:** Primarily English but supports Korean, Kazakh, Hindi, Bengali, Spanish, Indonesian, German, French, Arabic, Russian, Thai, Turkish, Ukraninan, Romainian and more.
|
31 |
- **Developed by:** Patronus AI
|
32 |
+
- **Paper:** [https://arxiv.org/abs/2412.14140](https://arxiv.org/abs/2412.14140)
|
33 |
- **License:** [https://creativecommons.org/licenses/by-nc/4.0/](https://creativecommons.org/licenses/by-nc/4.0/)
|
34 |
|
35 |
### Model Sources
|
36 |
|
37 |
<!-- Provide the basic links for the model. -->
|
38 |
|
39 |
+
- **Repository:** [https://github.com/patronus-ai/glider](https://github.com/patronus-ai/glider)
|
40 |
|
41 |
|
42 |
## How to Get Started with the Model
|
|
|
75 |
<score>
|
76 |
[The final integer score assigned based on the scoring rubric]
|
77 |
</score>
|
78 |
+
"""
|
79 |
```
|
80 |
|
81 |
Since the model supports arbitrary number of inputs and outputs, the data can be structured in any one of the following ways:
|
|
|
137 |
To run inference, you can use HF pipeline:
|
138 |
|
139 |
```
|
|
|
140 |
model_name = 'PatronusAI/glider'
|
141 |
pipe = pipeline(
|
142 |
"text-generation",
|
|
|
161 |
|
162 |
The model was evaluated on several popular datasets:
|
163 |
|
164 |
+
<img src="https://i.imgur.com/77lhcwf.png" alt="Results" width="100%"/>
|
165 |
|
166 |
## Citation
|
167 |
If you are using the model, cite using
|
168 |
|
169 |
```
|
170 |
+
@misc{deshpande2024glider,
|
171 |
+
title={GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking},
|
172 |
+
author={Darshan Deshpande and Selvan Sunitha Ravi and Sky CH-Wang and Bartosz Mielczarek and Anand Kannappan and Rebecca Qian},
|
173 |
+
year={2024},
|
174 |
+
eprint={2412.14140},
|
175 |
+
archivePrefix={arXiv},
|
176 |
+
primaryClass={cs.CL},
|
177 |
+
url={https://arxiv.org/abs/2412.14140},
|
178 |
+
}
|
179 |
```
|
180 |
|
181 |
## Model Card Contact
|