Spaces:

open-llm-leaderboard
/

open_llm_leaderboard

Running on CPU Upgrade

App Files Files Community

1014

[BUG] Gemma2-9b-it evaluation

#849

by DeepMount00 - opened Jul 18

Discussion

DeepMount00

Jul 18

In my opinion, there is a bug in the evaluation of gemma-9b-it. The reported scores are very low, whereas the scores of gemma-9b-it should be much higher.

danielus

Jul 18

Yeah, i use Gemma 2 9b every day, for tasks like summarization, also in italian, a really good model

alozowski

Open LLM Leaderboard org Jul 18

Hi everyone!

Yes, we are looking into this issue right now, and this problem is being discussed here – please, feel free to write there in case of any questions

alozowski changed discussion status to closed Jul 18

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment