DarshanDeshpande commited on
Commit
ee47592
1 Parent(s): 5303f4b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -6
README.md CHANGED
@@ -17,8 +17,10 @@ pipeline_tag: text-generation
17
 
18
  # Patronus GLIDER
19
 
 
 
20
  GLIDER is a fine tuned phi-3.5-mini-instruct which can be used as a general purpose evaluation model to judge texts, conversations and RAG setups according to arbitrary, user defined criteria and rubric scale.
21
- This model was trained using a combination of synthetic and domain adapted data from popular datasets like Mocha, FinQA, Realtoxicity, etc. The training data for this model covers over 183 metrics and 683+ domains including finance, medicine, and many more.
22
  The maximum sequence length is 8192 tokens but the model can support longer texts as well (tested upto 12,000 tokens).
23
 
24
 
@@ -27,14 +29,14 @@ The maximum sequence length is 8192 tokens but the model can support longer text
27
  - **Model Type:** GLIDER is a fine-tuned version of microsoft/Phi-3.5-mini-instruct model.
28
  - **Language:** Primarily English but supports Korean, Kazakh, Hindi, Bengali, Spanish, Indonesian, German, French, Arabic, Russian, Thai, Turkish, Ukraninan, Romainian and more.
29
  - **Developed by:** Patronus AI
30
- - **Paper:** [TBD]
31
  - **License:** [https://creativecommons.org/licenses/by-nc/4.0/](https://creativecommons.org/licenses/by-nc/4.0/)
32
 
33
  ### Model Sources
34
 
35
  <!-- Provide the basic links for the model. -->
36
 
37
- - **Repository:** [https://github.com/patronus-ai/slm-evaluator](https://github.com/patronus-ai/slm-evaluator)
38
 
39
 
40
  ## How to Get Started with the Model
@@ -73,6 +75,7 @@ Your output must in the following format:
73
  <score>
74
  [The final integer score assigned based on the scoring rubric]
75
  </score>
 
76
  ```
77
 
78
  Since the model supports arbitrary number of inputs and outputs, the data can be structured in any one of the following ways:
@@ -134,7 +137,6 @@ Note that these XML tags can be changed according to your convenience and task
134
  To run inference, you can use HF pipeline:
135
 
136
  ```
137
-
138
  model_name = 'PatronusAI/glider'
139
  pipe = pipeline(
140
  "text-generation",
@@ -159,13 +161,21 @@ Since the model is trained in chat format, ensure that you pass the prompt as a
159
 
160
  The model was evaluated on several popular datasets:
161
 
162
- <img src="https://i.imgur.com/PuW1DzP.png" alt="Results" width="70%"/>
163
 
164
  ## Citation
165
  If you are using the model, cite using
166
 
167
  ```
168
- [Paper citation]
 
 
 
 
 
 
 
 
169
  ```
170
 
171
  ## Model Card Contact
 
17
 
18
  # Patronus GLIDER
19
 
20
+ <img src="https://i.imgur.com/1AbgTJa.png" alt="GLIDER" width="100%"/>
21
+
22
  GLIDER is a fine tuned phi-3.5-mini-instruct which can be used as a general purpose evaluation model to judge texts, conversations and RAG setups according to arbitrary, user defined criteria and rubric scale.
23
+ This model was trained using a combination of synthetic and domain adapted data from popular datasets like Mocha, FinQA, Realtoxicity, etc. The training data for this model covers over 183 metrics and 685 domains including finance, medicine, and many more.
24
  The maximum sequence length is 8192 tokens but the model can support longer texts as well (tested upto 12,000 tokens).
25
 
26
 
 
29
  - **Model Type:** GLIDER is a fine-tuned version of microsoft/Phi-3.5-mini-instruct model.
30
  - **Language:** Primarily English but supports Korean, Kazakh, Hindi, Bengali, Spanish, Indonesian, German, French, Arabic, Russian, Thai, Turkish, Ukraninan, Romainian and more.
31
  - **Developed by:** Patronus AI
32
+ - **Paper:** [https://arxiv.org/abs/2412.14140](https://arxiv.org/abs/2412.14140)
33
  - **License:** [https://creativecommons.org/licenses/by-nc/4.0/](https://creativecommons.org/licenses/by-nc/4.0/)
34
 
35
  ### Model Sources
36
 
37
  <!-- Provide the basic links for the model. -->
38
 
39
+ - **Repository:** [https://github.com/patronus-ai/glider](https://github.com/patronus-ai/glider)
40
 
41
 
42
  ## How to Get Started with the Model
 
75
  <score>
76
  [The final integer score assigned based on the scoring rubric]
77
  </score>
78
+ """
79
  ```
80
 
81
  Since the model supports arbitrary number of inputs and outputs, the data can be structured in any one of the following ways:
 
137
  To run inference, you can use HF pipeline:
138
 
139
  ```
 
140
  model_name = 'PatronusAI/glider'
141
  pipe = pipeline(
142
  "text-generation",
 
161
 
162
  The model was evaluated on several popular datasets:
163
 
164
+ <img src="https://i.imgur.com/77lhcwf.png" alt="Results" width="100%"/>
165
 
166
  ## Citation
167
  If you are using the model, cite using
168
 
169
  ```
170
+ @misc{deshpande2024glider,
171
+ title={GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking},
172
+ author={Darshan Deshpande and Selvan Sunitha Ravi and Sky CH-Wang and Bartosz Mielczarek and Anand Kannappan and Rebecca Qian},
173
+ year={2024},
174
+ eprint={2412.14140},
175
+ archivePrefix={arXiv},
176
+ primaryClass={cs.CL},
177
+ url={https://arxiv.org/abs/2412.14140},
178
+ }
179
  ```
180
 
181
  ## Model Card Contact