diff --git a/docs/my-website/docs/anthropic_unified.md b/docs/my-website/docs/anthropic_unified.md index d4660bf070..03ba8a6884 100644 --- a/docs/my-website/docs/anthropic_unified.md +++ b/docs/my-website/docs/anthropic_unified.md @@ -1,7 +1,7 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# /v1/messages [BETA] +# /v1/messages Use LiteLLM to call all your LLM APIs in the Anthropic `v1/messages` format. diff --git a/docs/my-website/docs/audio_transcription.md b/docs/my-website/docs/audio_transcription.md index 22517f68e4..8cbc567180 100644 --- a/docs/my-website/docs/audio_transcription.md +++ b/docs/my-website/docs/audio_transcription.md @@ -3,13 +3,22 @@ import TabItem from '@theme/TabItem'; # /audio/transcriptions -Use this to loadbalance across Azure + OpenAI. +## Overview + +| Feature | Supported | Notes | +|-------|-------|-------| +| Cost Tracking | ✅ | | +| Logging | ✅ | works across all integrations | +| End-user Tracking | ✅ | | +| Fallbacks | ✅ | between supported models | +| Loadbalancing | ✅ | between supported models | +| Support llm providers | `openai`, `azure`, `vertex_ai`, `gemini`, `deepgram`, `groq`, `fireworks_ai` | | ## Quick Start ### LiteLLM Python SDK -```python showLineNumbers +```python showLineNumbers title="Python SDK Example" from litellm import transcription import os @@ -30,7 +39,7 @@ print(f"response: {response}") -```yaml showLineNumbers +```yaml showLineNumbers title="OpenAI Configuration" model_list: - model_name: whisper litellm_params: @@ -45,7 +54,7 @@ general_settings: -```yaml showLineNumbers +```yaml showLineNumbers title="OpenAI + Azure Configuration" model_list: - model_name: whisper litellm_params: @@ -71,7 +80,7 @@ general_settings: ### Start proxy -```bash +```bash showLineNumbers title="Start Proxy Server" litellm --config /path/to/config.yaml # RUNNING on http://0.0.0.0:8000 @@ -82,7 +91,7 @@ litellm --config /path/to/config.yaml -```bash +```bash showLineNumbers title="Test with cURL" curl --location 'http://0.0.0.0:8000/v1/audio/transcriptions' \ --header 'Authorization: Bearer sk-1234' \ --form 'file=@"/Users/krrishdholakia/Downloads/gettysburg.wav"' \ @@ -92,7 +101,7 @@ curl --location 'http://0.0.0.0:8000/v1/audio/transcriptions' \ -```python showLineNumbers +```python showLineNumbers title="Test with OpenAI Python SDK" from openai import OpenAI client = openai.OpenAI( api_key="sk-1234", @@ -115,4 +124,82 @@ transcript = client.audio.transcriptions.create( - Azure - [Fireworks AI](./providers/fireworks_ai.md#audio-transcription) - [Groq](./providers/groq.md#speech-to-text---whisper) -- [Deepgram](./providers/deepgram.md) \ No newline at end of file +- [Deepgram](./providers/deepgram.md) + +--- + +## Fallbacks + +You can configure fallbacks for audio transcription to automatically retry with different models if the primary model fails. + + + + +```bash showLineNumbers title="Test with cURL and Fallbacks" +curl --location 'http://0.0.0.0:4000/v1/audio/transcriptions' \ +--header 'Authorization: Bearer sk-1234' \ +--form 'file=@"gettysburg.wav"' \ +--form 'model="groq/whisper-large-v3"' \ +--form 'fallbacks[]="openai/whisper-1"' +``` + + + + +```python showLineNumbers title="Test with OpenAI Python SDK and Fallbacks" +from openai import OpenAI +client = OpenAI( + api_key="sk-1234", + base_url="http://0.0.0.0:4000" +) + +audio_file = open("gettysburg.wav", "rb") +transcript = client.audio.transcriptions.create( + model="groq/whisper-large-v3", + file=audio_file, + extra_body={ + "fallbacks": ["openai/whisper-1"] + } +) +``` + + + +### Testing Fallbacks + +You can test your fallback configuration using `mock_testing_fallbacks=true` to simulate failures: + + + + +```bash showLineNumbers title="Test Fallbacks with Mock Testing" +curl --location 'http://0.0.0.0:4000/v1/audio/transcriptions' \ +--header 'Authorization: Bearer sk-1234' \ +--form 'file=@"gettysburg.wav"' \ +--form 'model="groq/whisper-large-v3"' \ +--form 'fallbacks[]="openai/whisper-1"' \ +--form 'mock_testing_fallbacks=true' +``` + + + + +```python showLineNumbers title="Test Fallbacks with Mock Testing" +from openai import OpenAI +client = OpenAI( + api_key="sk-1234", + base_url="http://0.0.0.0:4000" +) + +audio_file = open("gettysburg.wav", "rb") +transcript = client.audio.transcriptions.create( + model="groq/whisper-large-v3", + file=audio_file, + extra_body={ + "fallbacks": ["openai/whisper-1"], + "mock_testing_fallbacks": True + } +) +``` + + \ No newline at end of file diff --git a/litellm/proxy/proxy_config.yaml b/litellm/proxy/proxy_config.yaml index 498cf4dcf7..13724177c0 100644 --- a/litellm/proxy/proxy_config.yaml +++ b/litellm/proxy/proxy_config.yaml @@ -1,19 +1,9 @@ model_list: - - model_name: anthropic/* + - model_name: groq/* litellm_params: - model: anthropic/* + model: groq/* + api_key: bad - model_name: openai/* litellm_params: model: openai/* -router_settings: - model_group_alias: { - "gemini-2.5-pro": "anthropic/claude-sonnet-4-20250514", - "gemini-2.5-flash": "anthropic/claude-sonnet-4-20250514", - # "gemini-2.5-pro": "openai/gpt-4o-mini" - } - -mcp_servers: - deepwiki_mcp: - url: "https://mcp.deepwiki.com/mcp" - transport: "http"