Spaces:
Running
on
CPU Upgrade
Same eval error?
Hi, I'm here with same reason like previously https://huggingface.co./spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1089
All my mistral 7b v0.2 models failed like previous.(problem started a month ago, even model that previously was evaluated was getting error with new try)
https://huggingface.co./datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.88-07.02-RP-dpo-merged_16bit-4_eval_request_False_float16_Original.json
https://huggingface.co./datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.89-13.02-RP_eval_request_False_bfloat16_Original.json
https://huggingface.co./datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.90-14.02-RP_eval_request_False_bfloat16_Original.json
https://huggingface.co./datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.91-14.02-RP_eval_request_False_bfloat16_Original.json
Thx for looking at it.
Tried icefog72/Ice0.91-14.02-RP and it failed again.
Maybe this is a problem?
Tried eval with max_position_embeddings 4096 and still fail
Already evaluated model failed(I don't need re eval it again, it's just to show that eval is broken).