Anthropic/hh-rlhf
Viewer
•
Updated
•
169k
•
10.6k
•
1.23k
Awesome feedback datasets
A curated list of datasets with human or AI feedback. Useful for training reward models or applying techniques like DPO.
Note The OG of open preference data. Not super great quality
berkeley-nest/Nectar
Viewer • Updated • 183k • 381 • 281Note NC
-
openbmb/UltraFeedback
Viewer • Updated • 64k • 1.44k • 343
Intel/orca_dpo_pairs
Viewer • Updated • 12.9k • 1.65k • 291Note A simple idea to just generate GPT-4 responses and treat them as preferred response.
Hello-SimpleAI/HC3
Viewer • Updated • 48.6k • 1.07k • 184Note Some parts are NC
-
lvwerra/stack-exchange-paired
Viewer • Updated • 31.3M • 2.04k • 142
argilla/ultrafeedback-binarized-preferences-cleaned
Viewer • Updated • 60.9k • 8.96k • 127Note Computes the mean preference score instead of relying on the overall score from GPT-4. Also removes contamination from TruthfulQA prompts
nvidia/HelpSteer
Viewer • Updated • 37.1k • 1.8k • 227Note The dataset behind NVIDIA's SteerLM alignment method
-
jondurbin/truthy-dpo-v0.1
Viewer • Updated • 1.02k • 410 • 132 -
lmsys/chatbot_arena_conversations
Viewer • Updated • 33k • 1.24k • 355 -
hbXNov/sparse_feedback
Updated • 46 • 5 -
Unified-Language-Model-Alignment/Anthropic_HH_Golden
Viewer • Updated • 44.8k • 452 • 31 -
neovalle/H4rmony
Viewer • Updated • 2.02k • 71 • 15 -
PKU-Alignment/PKU-SafeRLHF
Viewer • Updated • 164k • 3.79k • 120 -
peiyi9979/Math-Shepherd
Viewer • Updated • 445k • 682 • 79 -
m-a-p/Code-Feedback
Viewer • Updated • 66.4k • 199 • 199 -
introspector/unimath
Updated • 4.42k • 7 -
abacusai/MetaMath_DPO_FewShot
Viewer • Updated • 395k • 98 • 25 -
interstellarninja/tool-calls-dpo
Viewer • Updated • 235 • 66 • 7