🚩 Report: Ethical issue(s)
#142
by
Impulse2000
- opened
- The Evals are not reproducible (https://x.com/ArtificialAnlys/status/1832457791010959539/photo/1)
- Even the provider HyperBolic and all other providers have stopped hosting the model (Lack of Adoption means its not effective)
- Matt schumer did not disclose he is an investor in Glaive (the platform he promoted)
- Huge Claims made about the model (being the best model in the world)
- HF evals are not very good. it scores a 30.74 on average on the HF 2 leaderboard. the original llama3.1 70b model scores 41.74 on average.
I don't post this report lightly, i waited a while, gathering information on this.
The action i think is appropriate, is getting the model author to correct their model README.md, lowering the claims made, and showing the real benchmarks, which have been pushed to this repo probably a hundred times by now, but Matt Schumer refuses to accept the real benchmarks.
If they fail to do this within a reasonably time period, like 30 days. i would reccomend you remove the model from the platform or put a disclaimer at the top of the model card, saying its claims are proven false.
Thanks for reading,
James Clarke.
Founder of Novora LLC.