Spaces:
Running
on
CPU Upgrade
One user.. dozens of evals.. unintentional abuse?
A user should only have at most 1 slot for running eval indistinctly of the queue length for evaluation.. I understand this is not intentional abusing.. but indeed seems unreasonable for 1 user to push 30 evals of trash and clog the entire pipeline..
I personally disagree on using HF Eval pipeline to push garbage to be evaluated.. instead I prefer to do not push to eval things that I havent tested locally first.. it feels like more civic..
@clefourrier ^^ IMHO tbh.. 1 user 1 run at a time, many queues, but just one slot consumed by HF user handler?
I feel the same way. Currently, FINGU's LLM models are a bottleneck for evaluation.
I feel the same way. Currently, FINGU's LLM models are a bottleneck for evaluation.
Appreciated the fact that you step forward, and understanding the problem in a civic manner. I strongly believe this is not an intentional abuse, is just how things are.. a reasonable QoS/FairQueueing could be in place to "share is care" the evaluation pipe that HF sponsors.