Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
Avelina 
posted an update Aug 12
Post
2160
Hey HF. I just released a new reward modelling dataset: Avelina/UltraSteer-v0

UltraSteer-V0 is a massive collection of single- and multi-turn dialogue with fine-grained reward labels produced by Nvidia's nvidia/Llama2-13B-SteerLM-RM reward model. We have a total of 2.3M labelled sequences taken from high quality datasets with a total of 2.8M labelled turns each containing 9 attributes produced as is from the reward model.

This is still very much an early version of the dataset (but it's fully usable!) and an updated version will be on the way with a full paper.

I would really appreciate if people could take a look at the dataset and suggest any improvements (e.g. more data sources, different cleaning approaches, different label schema, etc) in the community section.
mmoy

Like the fact you kept the profanity within the dataset so folks have that option to learn, leverage or reject that type of language based on the application.

What were upper bounds for values in each of the categories?

·
Avelina

Each attribute should be in the range zero to four, however the included labels are given as is by the reward model which means some values may be outside this range (although only slightly) so it is recommended that you clamp all attributes between zero and four.

We included the unclamped versions because you may want the exact outputs given by the reward model for some specific reason, and if we had clamped these values in the dataset you would be unable to recover them.

In this post