# You can use any model that available to you and deployed on Hugging Face with compatible API # X_NAME variables are optional for HuggingFace API you can use them for your convenience # Make sure your key has permission to use all models # Set up you key here: https://huggingface.co./docs/api-inference/en/quicktour#get-your-api-token #HF_API_KEY=os.getenv('HF_Key') #TTS_KEY=os.getenv('HF_Key') # For example you can try public Inference API endpoint for Meta-Llama-3-70B-Instruct model # This model quiality is comparable with GPT-4 # But public API has strict limit for output tokens, so it is very hard to use it for this usecase # You can use your private API endpoint for this model # Or use any other Hugging Face model that supports Messages API # Don't forget to add '/v1' to the end of the URL LLM_URL=https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-70B-Instruct/v1 LLM_TYPE=HF_API LLM_NAME=Meta-Llama-3-70B-Instruct # If you want to use any other model serving provider the configuration will be similar # Below is the example for Groq # GROQ_API_KEY=gsk_YOUR_GROQ_API_KEY # LLM_URL=https://api.groq.com/openai/v1 # LLM_TYPE=GROQ_API # LLM_NAME=llama3-70b-8192 # The Open AI whisper family with more models is available on HuggingFace: # https://huggingface.co./collections/openai/whisper-release-6501bba2cf999715fd953013 # You can also use any other compatible STT model from HuggingFace STT_URL=https://api-inference.huggingface.co/models/openai/whisper-tiny.en STT_TYPE=HF_API STT_NAME=whisper-tiny.en # You can use compatible TTS model from HuggingFace # For example you can try public Inference API endpoint for Facebook MMS-TTS model # In my experience OS TTS models from HF sound much more robotic than OpenAI TTS models TTS_URL=https://api-inference.huggingface.co/models/microsoft/speecht5_tts TTS_TYPE=HF_API TTS_NAME=Facebook-mms-tts-eng