Spaces:
Running
on
CPU Upgrade
🚩 Report: Legal issue(s)
The Open Medical LLM Leaderboard is enabling the incompliant and illegal placement on the market. In USA and EU this breaches various effective and valid regulations as US FDA 21cfr820 and EU MDR 2017/745. The board, HuggingFace and the LLMs are in parallel reported to US FDA and EU27 local health authorities.
"Effective Placement on the market" and - for Huggingface - "putting into service" according to the regulation has been fulfilled by publishing the models, their files and the making the site public. The site has been saved according to the rules of legalized and notarized screenshots
US FDA Database for approved AI in Healthcare and EUDAMED database have been searched and no model or Huggingface as distributor (official role according to the regulation) are registered or approved.
There is no disclaimer on the website or any other remark that the models are NOT regulatory approved for use and for research they are not approved too as there is no entry for any valid clinicaltrial on the respective international clinical trial websites.
Applicable and breached regulation (copied from linkedin):
EU MDR 2017/745 for AI
https://lnkd.in/eRxS5AJr
US FDA for #AI as #SaMD
https://lnkd.in/e--8gYvz
https://lnkd.in/esKdMwGv
https://lnkd.in/evKPHg8j
Health Canada | Santé Canada
https://lnkd.in/ew5ynUZR
Therapeutic Goods Administration in Australia for #AI as #SaMD
https://lnkd.in/epZ--BNi
And special guidance for #AI, #ML and #LLM:
https://lnkd.in/eSNQFdKT
@adhocon The leaderboard is purely evaluative and does not facilitate the distribution or deployment of the models, therefore the concerns about breaching US FDA 21 CFR 820 and EU MDR 2017/745 are not relevant.
Since the leaderboard does not involve the actual placement of the models on the market or their deployment in a clinical setting, the issue of "effective placement on the market" or "putting into service" is also irrelevant. The primary role of the leaderboard is to assess and compare the performance of the models, not to distribute them.
The regulations and guidance documents that you mention primarily apply to the deployment and use of AI models in medical settings. If the leaderboard does not facilitate such use, the direct applicability of these regulations doesn't apply.
cringe (copied from linkedin)