Reproduce results on AI2D and ChartQA

#15
by NimrodShabtay1986 - opened

Hi,
Thanks for a wonderful work.
I am trying to reproduce the results for chartQA test and AI2D and I am getting lower results than I expected.
I took the generation code from the model card, and it would be very helpful if you can share the prompt and other (if any) parameters so I can reproduce the results.

Thanks in advance

We now have code to reproduce the results, see: https://github.com/allenai/molmo?tab=readme-ov-file#evaluation

chrisc36 changed discussion status to closed

Sign up or log in to comment