Reward Model,Preference Datasets Used
RLHFlow/ArmoRM-Llama3-8B-v0.1,"HelpSteer, UltraFeedback, BeaverTails, Argilla-Capybara, Argilla-Math-Preferences, CodeUltraFeedback, Argilla-OpenOrca"
RLHFlow/pair-preference-model-LLaMA3-8B,"Filtered HH-RLHF, SHP, HelpSteer, SafeRLHF-30k, UltraFeedback, UltraInteract, CodeUltraFeedback, Argilla-Math, OpenOrca, Capybara"
sfairXC/FsfairX-LLaMA3-RM-v0.1,"Filtered HH-RLHF, SHP, HelpSteer, SafeRLHF-30k, UltraFeedback, UltraInteract, CodeUltraFeedback, Argilla-Math, OpenOrca, Capybara"
openbmb/Eurus-RM-7b,"UltraInteract, UltraFeedback, UltrSafety"
Nexusflow/Starling-RM-34B,Nectar
weqweasdas/RM-Mistral-7B,"HH-RLHF, Capybara, Orca, SHP, UltraFeedback, HelpSetter, PKU-SafeRLHF, PKU-SafeRLHF-30k"
hendrydong/Mistral-RM-for-RAFT-GSHF-v0,Undisclosed
stabilityai/stablelm-2-12b-chat,"HH-RLHF, argilla/dpo-mix-7k, and other Undisclosed"
Ray2333/reward-model-Mistral-7B-instruct...,"Summarize, WebGPT, Dahoas/instruct-synthetic-prompt-responses, HH-RLHF, ChatBotArena Conversations, UltraFeedback, Nectar"
allenai/tulu-2-dpo-70b,UltraFeedback
meta-llama/Meta-Llama-3-70B-Instruct,Undisclosed
prometheus-eval/prometheus-8x7b-v2.0,Preference Collction (relabeled mix)
NousResearch/Nous-Hermes-2-Mistral-7B-DPO,Undisclosed
mistralai/Mixtral-8x7B-Instruct-v0.1,Undisclosed
upstage/SOLAR-10.7B-Instruct-v1.0,"OpenOrca, Intel-Orca, UltraFeedback"
HuggingFaceH4/zephyr-7b-alpha,UltraFeedback
allenai/tulu-2-dpo-13b,UltraFeedback
0-hero/Matter-0.1-7B-boost-DPO-preview,Undisclosed
prometheus-eval/prometheus-7b-v2.0,Preference Collction (relabeled mix)
HuggingFaceH4/starchat2-15b-v0.1,"UltraFeedback, Orca"
HuggingFaceH4/zephyr-7b-beta,UltraFeedback
allenai/tulu-2-dpo-7b,UltraFeedback
jondurbin/bagel-dpo-34b-v0.5,"Airoboros 3.2, Contextual DPO, HelpSteer, Orca, Gutenberg-DPO, Python DPO, Toxic DPO, Truthy, UltraFeedback"
berkeley-nest/Starling-RM-7B-alpha,Nectar
NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO,Undisclosed
0-hero/Matter-0.1-7B-DPO-preview,Undisclosed
stabilityai/stablelm-zephyr-3b,"UltraFeedback, Orca"
Qwen/Qwen1.5-14B-Chat,Undisclosed
CohereForAI/c4ai-command-r-plus,Undisclosed
OpenAssistant/oasst-rm-2.1-pythia-1.4b-epoch-2.5,"WebGPT, HH-RLHF, SHP, WebGPT, Summarize"
Qwen/Qwen1.5-7B-Chat,Undisclosed
weqweasdas/RM-Gemma-7B,"HH-RLHF, SHP, UltraFeedback, Capybara, HelpSteer, Orca"
openbmb/Eurus-7b-kto,"UltraInteract, UltraFeedback"
Qwen/Qwen1.5-72B-Chat,Undisclosed
openbmb/UltraRM-13b,"UltraFeedback, HH-RLHF, SHP, Summarize"