diff --git a/docs/my-website/docs/routing.md b/docs/my-website/docs/routing.md index 1ad2a23133..1ecac52b75 100644 --- a/docs/my-website/docs/routing.md +++ b/docs/my-website/docs/routing.md @@ -798,6 +798,9 @@ There are 3 types of fallbacks: - `fallbacks`: For all remaining errors - e.g. litellm.RateLimitError **Content Policy Violation Fallback** + + + ```python from litellm import Router @@ -830,9 +833,32 @@ response = router.completion( messages=[{"role": "user", "content": "Hey, how's it going?"}], ) ``` + + + +In your proxy config.yaml just add this line 👇 + +```yaml +router_settings: + content_policy_fallbacks=[{"claude-2": ["my-fallback-model"]}] +``` + +Start proxy + +```bash +litellm --config /path/to/config.yaml + +# RUNNING on http://0.0.0.0:4000 +``` + + + **Context Window Exceeded Fallback** + + + ```python from litellm import Router @@ -865,9 +891,33 @@ response = router.completion( messages=[{"role": "user", "content": "Hey, how's it going?"}], ) ``` + + + +In your proxy config.yaml just add this line 👇 + +```yaml +router_settings: + context_window_fallbacks=[{"claude-2": ["my-fallback-model"]}] +``` + +Start proxy + +```bash +litellm --config /path/to/config.yaml + +# RUNNING on http://0.0.0.0:4000 +``` + + + **Regular Fallbacks** + + + + ```python from litellm import Router @@ -900,6 +950,26 @@ response = router.completion( messages=[{"role": "user", "content": "Hey, how's it going?"}], ) ``` + + + +In your proxy config.yaml just add this line 👇 + +```yaml +router_settings: + fallbacks=[{"claude-2": ["my-fallback-model"]}] +``` + +Start proxy + +```bash +litellm --config /path/to/config.yaml + +# RUNNING on http://0.0.0.0:4000 +``` + + + ### Caching