Spaces:
Running
on
CPU Upgrade
Arabic MMLU
Hi,
There are two versions of MMLU: one is the Arabic MMLU one generated by arabic speakers (https://huggingface.co./datasets/OALL/Arabic_MMLU) and the other one is the translated version of MMLU (https://huggingface.co./datasets/OALL/AlGhafa-Arabic-LLM-Benchmark-Translated). I wonder which one is used for the evaluation part of OALL.
Best,
Malikeh
Hey there,
Well first this one (https://huggingface.co./datasets/OALL/Arabic_MMLU) is not generated by arabic speakers but translated also by AceGPT team ! The MMLU reported in the leaderboard is that one (https://huggingface.co./datasets/OALL/Arabic_MMLU). While we use other subsets from (https://huggingface.co./datasets/OALL/AlGhafa-Arabic-LLM-Benchmark-Translated) but MMLU
Thank you