Join the community of Machine Learners and AI enthusiasts.
Sign Up
The Serverless Inference API was significantly degraded a few months ago, making it almost unusable unless it is labeled Warm.
The conditions under which it will be Warm are unknown, but it is safe to say that it is impossible to aim for.
If you create a Spaces with Gradio, it may work, so you can try that.
https://www.gradio.app/guides/using-hugging-face-integrations
thanks
You can use Modal Labs to run inference. GPU's take ~ 30 seconds to provision. Here's a quickstart on the matter: https://modal.com/docs/examples/vllm_inference
