mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-29 12:20:24 +00:00
51876292a0
* feat(router): integrate allowed_fails_policy into health check failures (#24988) * feat(router): integrate allowed_fails_policy into health check failures Health check failures now increment the same per-deployment failure counters used by allowed_fails_policy, so users can control how many health check failures of each error type are required before a deployment enters cooldown. - ahealth_check() preserves the original exception in its return dict - run_with_timeout() returns a litellm.Timeout on health check timeout - _perform_health_check() propagates exceptions to unhealthy endpoints - _write_health_state_to_router_cache() calls _set_cooldown_deployments for each unhealthy endpoint that has an exception - When allowed_fails_policy is set, the binary health check filter is bypassed so cooldown is the sole routing exclusion mechanism - Safety net: if all deployments are in cooldown with enable_health_check_routing=True, the cooldown filter is bypassed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(router): add health_check_ignore_transient_errors flag When enabled, health check failures with 429 (rate limit) or 408 (timeout) status codes are skipped from the cooldown pipeline. These are transient load issues, not broken deployments. Auth errors (401), 404, and 5xx errors still increment counters and trigger cooldown as before. Config (general_settings): health_check_ignore_transient_errors: true Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(router): also exclude 429/408 from health state cache when ignore_transient_errors set The previous fix only skipped cooldown counter increments. The health state cache was still marking 429/408 endpoints as is_healthy=False, causing the binary health check filter to exclude them from routing. Now, when health_check_ignore_transient_errors=True, 429/408 endpoints are also excluded from the unhealthy list passed to build_deployment_health_states(), so the binary filter treats them as unaffected (not unhealthy). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(router): add health check driven routing guide New standalone page covering the full health check routing feature: allowed_fails_policy integration, health_check_ignore_transient_errors, architecture SVG, step-by-step setup, and gotchas (TTL, AllowedFails semantics). Replaces the inline section in health.md with a link to the new page. Added to the Routing & Load Balancing sidebar. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): fix three CI failures - Add "exception" to ILLEGAL_DISPLAY_PARAMS in health_check.py so the exception object is stripped before the health endpoint serializes results to JSON (fixes TypeError: 'URL' object is not iterable) - Add allowed_fails_policy = None to FakeRouter stubs in test_router_health_check_routing.py (fixes AttributeError) - Add health_check_ignore_transient_errors to config_settings.md router settings reference table (fixes documentation test) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix litellm/tests/proxy_unit_tests/test_proxy_server.py * fix(router): address greptile review comments - Narrow cooldown safety-net bypass: only fires when allowed_fails_policy is set (cooldown is health-check driven). Without a policy, cooldowns are from real request failures and must not be bypassed. - Restore cooldown deployments DEBUG log that was accidentally removed. - Fix test_health TypeError: move exception extraction to a separate exceptions_by_model_id dict returned alongside endpoints, so exception objects never appear in the endpoint dicts that get JSON-serialized by the /health response. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): properly isolate exceptions from health response Return exceptions_by_model_id as a separate third value from _perform_health_check / perform_health_check so exception objects (which contain non-JSON-serializable httpx URL types) never appear in the endpoint dicts that get serialized by the /health response. Callers updated: _health_endpoints.py, shared_health_check_manager.py, proxy_server.py background loop. All use the exceptions dict only for cooldown integration, not for display. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(shared-health-check): fix remaining 2-value return sites and update type annotation * fix(health-check-routing): fix P0 cooldown integration never firing The cooldown loop was reading endpoint.get("exception") which is always None because exceptions are now returned via exceptions_by_model_id, not stored in endpoint dicts. Fixed to use _exceptions.get(model_id). Also fixes the transient-error filter to use _exceptions instead of endpoint.get("exception"), and fixes all remaining 2-value return sites in shared_health_check_manager.py. Tests updated to pass exceptions via exceptions_by_model_id parameter instead of endpoint dicts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): fix P1 transient-error filter broken on cache hits When SharedHealthCheckManager returns cached results, exceptions_by_model_id is always {} so the transient-error filter defaulted to status 500 for all endpoints, incorrectly marking 429/408 endpoints as unhealthy. Fix: store integer exception_status on each unhealthy endpoint dict in _perform_health_check. _get_endpoint_exception_status() uses the live exception object when available (direct path) and falls back to the stored integer (cache-hit path). The integer is JSON-serializable and survives the shared cache round-trip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): gate cooldown loop behind allowed_fails_policy Without the policy, cooldown is not the routing exclusion mechanism. Firing _set_cooldown_deployments for all enable_health_check_routing users was a backwards-incompatible change — 401s would immediately cooldown deployments that the binary filter would have recovered on the next cycle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: undo allowed_fails_policy gate on cooldown loop Cooldown integration via health checks is intentional for all enable_health_check_routing users, not just those with allowed_fails_policy. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(docs+tests): fix health_check_ignore_transient_errors doc section and test coverage - Move health_check_ignore_transient_errors from router_settings to general_settings in config_settings.md (code reads it from general_settings) - Remove duplicate enable_health_check_routing / health_check_staleness_threshold entries that were incorrectly listed under router_settings - Replace TestHealthCheckEndpointExceptionPropagation tests with ones that exercise the real _perform_health_check code path via mocked ahealth_check, verifying exceptions appear in exceptions_by_model_id and NOT in endpoint dicts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tests+docs): fix tuple unpacking and docs test failures - Update test mocks that return (healthy, unhealthy) to return (healthy, unhealthy, {}) to match the new 3-value signature - Update test unpackings of perform_shared_health_check to use healthy, unhealthy, _ = ... - Add health_check_ignore_transient_errors to router_settings section in config_settings.md (it is a Router constructor param, so the doc test requires it there; it also lives in general_settings for proxy use) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix CodeQL errors * fix(tests): fix 2-value unpackings of _perform_health_check in test_health_check.py * fix(tests): fix mock _perform_health_check returning 2-tuple instead of 3 * fix team routing --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add distributed lock for key rotation job (#23364) * fix: add distributed lock for key rotation job * fix: address Greptile review feedback on key rotation lock (#23834) * fix: address Greptile review feedback on key rotation lock * fix req changes greptile * feat(proxy): Optional on_error for guardrail pipeline (API / technical failures) (#24831) * guardrails fallback * docs * docs: add LITELLM_KEY_ROTATION_LOCK_TTL_SECONDS to environment variables reference * fix(mypy): accept Union[Dict, Any] in _get_deployment_order and use typed list to fix min() type error * fix(mypy): use Optional[str] for api_base in PydanticAI provider to match superclass signature --------- Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com> Co-authored-by: Shivam Rawat <shivam@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai>
1473 lines
44 KiB
JavaScript
1473 lines
44 KiB
JavaScript
/**
|
|
* Creating a sidebar enables you to:
|
|
- create an ordered group of docs
|
|
- render a sidebar for each doc of that group
|
|
- provide next/previous navigation
|
|
|
|
The sidebars can be generated from the filesystem, or explicitly defined here.
|
|
|
|
Create as many sidebars as you want.
|
|
*/
|
|
|
|
// @ts-check
|
|
|
|
/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
|
|
const sidebars = {
|
|
// // By default, Docusaurus generates a sidebar from the docs folder structure
|
|
integrationsSidebar: [
|
|
{ type: "doc", id: "integrations/index" },
|
|
{ type: "doc", id: "integrations/community" },
|
|
{
|
|
type: "category",
|
|
label: "Observability",
|
|
link: { type: "doc", id: "integrations/observability_index" },
|
|
items: [
|
|
{
|
|
type: "category",
|
|
label: "Contributing to Integrations",
|
|
items: [
|
|
{
|
|
type: "autogenerated",
|
|
dirName: "contribute_integration"
|
|
}
|
|
],
|
|
},
|
|
{
|
|
type: "autogenerated",
|
|
dirName: "observability"
|
|
},
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Guardrail Providers",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Guardrail Providers",
|
|
description: "Add safety and content filtering to LLM calls",
|
|
slug: "/guardrail_providers"
|
|
},
|
|
items: [
|
|
{
|
|
type: "category",
|
|
label: "Contributing to Guardrails",
|
|
items: [
|
|
"adding_provider/generic_guardrail_api",
|
|
"adding_provider/simple_guardrail_tutorial",
|
|
"adding_provider/adding_guardrail_support",
|
|
]
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "proxy/guardrails/team_based_guardrails",
|
|
label: "Team Bring-Your-Own Guardrails",
|
|
},
|
|
...[
|
|
"proxy/guardrails/qualifire",
|
|
"proxy/guardrails/aim_security",
|
|
"proxy/guardrails/onyx_security",
|
|
"proxy/guardrails/aporia_api",
|
|
"proxy/guardrails/azure_content_guardrail",
|
|
"proxy/guardrails/bedrock",
|
|
"proxy/guardrails/crowdstrike_aidr",
|
|
"proxy/guardrails/enkryptai",
|
|
"proxy/guardrails/ibm_guardrails",
|
|
"proxy/guardrails/grayswan",
|
|
"proxy/guardrails/hiddenlayer",
|
|
"proxy/guardrails/lasso_security",
|
|
"proxy/guardrails/guardrails_ai",
|
|
"proxy/guardrails/lakera_ai",
|
|
"proxy/guardrails/model_armor",
|
|
"proxy/guardrails/noma_security",
|
|
"proxy/guardrails/dynamoai",
|
|
"proxy/guardrails/openai_moderation",
|
|
"proxy/guardrails/pangea",
|
|
"proxy/guardrails/pillar_security",
|
|
"proxy/guardrails/pii_masking_v2",
|
|
"proxy/guardrails/panw_prisma_airs",
|
|
"proxy/guardrails/secret_detection",
|
|
"proxy/guardrails/custom_guardrail",
|
|
"proxy/guardrails/custom_code_guardrail",
|
|
"proxy/guardrails/prompt_injection",
|
|
"proxy/guardrails/tool_permission",
|
|
"proxy/guardrails/zscaler_ai_guard",
|
|
"proxy/guardrails/javelin"
|
|
].sort(),
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Alerting & Monitoring",
|
|
items: [
|
|
"proxy/alerting",
|
|
"proxy/pagerduty",
|
|
"proxy/prometheus",
|
|
"proxy/pyroscope_profiling"
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "[Beta] Prompt Management",
|
|
items: [
|
|
{
|
|
type: "category",
|
|
label: "Contributing to Prompt Management",
|
|
items: [
|
|
"adding_provider/generic_prompt_management_api",
|
|
]
|
|
},
|
|
"proxy/litellm_prompt_management",
|
|
"proxy/custom_prompt_management",
|
|
"proxy/native_litellm_prompt",
|
|
"proxy/prompt_management",
|
|
"proxy/arize_phoenix_prompts"
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "AI Tools",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "AI Tools",
|
|
description: "Integrate LiteLLM with AI tools like OpenWebUI, Claude Code, and more",
|
|
slug: "/ai_tools"
|
|
},
|
|
items: [
|
|
"tutorials/openweb_ui",
|
|
{
|
|
type: "category",
|
|
label: "Claude Code",
|
|
items: [
|
|
"tutorials/claude_responses_api",
|
|
"tutorials/claude_code_max_subscription",
|
|
"tutorials/claude_code_byok",
|
|
"tutorials/claude_code_customer_tracking",
|
|
"tutorials/claude_code_prompt_cache_routing",
|
|
"tutorials/claude_code_websearch",
|
|
"tutorials/claude_mcp",
|
|
"tutorials/claude_non_anthropic_models",
|
|
"tutorials/claude_code_plugin_marketplace",
|
|
"tutorials/claude_code_beta_headers",
|
|
]
|
|
},
|
|
"tutorials/opencode_integration",
|
|
"tutorials/openclaw_integration",
|
|
"tutorials/cursor_integration",
|
|
"tutorials/github_copilot_integration",
|
|
"tutorials/litellm_gemini_cli",
|
|
"tutorials/litellm_qwen_code_cli",
|
|
"tutorials/openai_codex",
|
|
"tutorials/retool_assist",
|
|
"tutorials/cost_tracking_coding"
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Agent SDKs",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Agent SDKs",
|
|
description: "Use LiteLLM with agent frameworks and SDKs",
|
|
slug: "/agent_sdks"
|
|
},
|
|
items: [
|
|
"tutorials/openai_agents_sdk",
|
|
"tutorials/claude_agent_sdk",
|
|
"tutorials/copilotkit_sdk",
|
|
"tutorials/google_adk",
|
|
"tutorials/google_genai_sdk",
|
|
"tutorials/livekit_xai_realtime",
|
|
"integrations/letta",
|
|
{ type: "doc", id: "tutorials/instructor", label: "Instructor with LiteLLM" },
|
|
{ type: "doc", id: "langchain/langchain", label: "LangChain with LiteLLM" },
|
|
"projects/openai-agents"
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Manage with AI Agents",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Manage with AI Agents",
|
|
description: "Use AI agents to manage your LiteLLM deployment — create users, teams, keys, models, and more via natural language.",
|
|
slug: "/manage_with_ai_agents"
|
|
},
|
|
items: [
|
|
"tutorials/claude_code_skills",
|
|
]
|
|
},
|
|
|
|
],
|
|
// But you can create a sidebar manually
|
|
tutorialSidebar: [
|
|
// ════════════════════════════════════════════════════════════
|
|
// GET STARTED
|
|
// ════════════════════════════════════════════════════════════
|
|
{
|
|
type: "category",
|
|
label: "Get Started",
|
|
collapsible: false,
|
|
collapsed: false,
|
|
items: [
|
|
{ type: "doc", id: "index", label: "Quickstart" },
|
|
{ type: "link", label: "Models & Pricing", href: "https://models.litellm.ai" },
|
|
{ type: "link", label: "Changelog", href: "/release_notes" },
|
|
],
|
|
},
|
|
|
|
{
|
|
type: "category",
|
|
label: "LiteLLM Python SDK",
|
|
items: [
|
|
{
|
|
type: "link",
|
|
label: "Quick Start",
|
|
href: "/docs/#litellm-python-sdk",
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "SDK Functions",
|
|
items: [
|
|
{
|
|
type: "doc",
|
|
id: "completion/input",
|
|
label: "completion()",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "embedding/supported_embedding",
|
|
label: "embedding()",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "response_api",
|
|
label: "responses()",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "text_completion",
|
|
label: "text_completion()",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "image_generation",
|
|
label: "image_generation()",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "audio_transcription",
|
|
label: "transcription()",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "text_to_speech",
|
|
label: "speech()",
|
|
},
|
|
{
|
|
type: "link",
|
|
label: "All Supported Endpoints →",
|
|
href: "https://docs.litellm.ai/docs/supported_endpoints",
|
|
},
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Configuration",
|
|
items: [
|
|
"set_keys",
|
|
"proxy_auth",
|
|
"caching/all_caches",
|
|
],
|
|
},
|
|
"completion/token_usage",
|
|
"exception_mapping",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "LiteLLM AI Gateway (Proxy)",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "LiteLLM AI Gateway (LLM Proxy)",
|
|
description: `OpenAI Proxy Server (LLM Gateway) to call 100+ LLMs in a unified interface & track spend, set budgets per virtual key/user`,
|
|
slug: "/simple_proxy",
|
|
},
|
|
items: [
|
|
{ type: "doc", id: "proxy/docker_quick_start", label: "Getting Started Tutorial" },
|
|
{
|
|
type: "category",
|
|
label: "Agent & MCP Gateway",
|
|
items: [
|
|
{
|
|
type: "category",
|
|
label: "A2A Agent Gateway",
|
|
items: [
|
|
"a2a",
|
|
"a2a_invoking_agents",
|
|
"a2a_agent_headers",
|
|
"a2a_cost_tracking",
|
|
"a2a_agent_permissions",
|
|
"a2a_iteration_budgets",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "MCP Gateway",
|
|
items: [
|
|
"mcp",
|
|
"mcp_usage",
|
|
"mcp_openapi",
|
|
"mcp_oauth",
|
|
"mcp_aws_sigv4",
|
|
"mcp_zero_trust",
|
|
"mcp_public_internet",
|
|
"mcp_semantic_filter",
|
|
"mcp_control",
|
|
"mcp_cost",
|
|
"mcp_guardrail",
|
|
{
|
|
type: "link",
|
|
label: "MCP Troubleshooting Guide",
|
|
href: "/docs/mcp_troubleshoot"
|
|
},
|
|
],
|
|
},
|
|
],
|
|
},
|
|
{
|
|
"type": "category",
|
|
"label": "Config.yaml",
|
|
"items": ["proxy/configs", "proxy/config_management", "proxy/config_settings"]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Setup & Deployment",
|
|
items: [
|
|
"proxy/quick_start",
|
|
"proxy/cli",
|
|
"proxy/debugging",
|
|
"proxy/error_diagnosis",
|
|
"proxy/deploy",
|
|
"proxy/health",
|
|
"proxy/master_key_rotations",
|
|
"proxy/model_management",
|
|
"proxy/prod",
|
|
"proxy/worker_startup_hooks",
|
|
"proxy/release_cycle",
|
|
],
|
|
},
|
|
{
|
|
"type": "link",
|
|
"label": "Demo LiteLLM Cloud",
|
|
"href": "https://www.litellm.ai/cloud"
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Admin UI",
|
|
items: [
|
|
"proxy/ui",
|
|
{
|
|
type: "category",
|
|
label: "Setup & SSO",
|
|
items: [
|
|
"proxy/admin_ui_sso",
|
|
"proxy/ui/ui_edit_logo",
|
|
"proxy/custom_sso",
|
|
"proxy/custom_root_ui",
|
|
"tutorials/scim_litellm",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Models",
|
|
items: [
|
|
"proxy/ui_credentials",
|
|
"proxy/ai_hub",
|
|
"proxy/model_compare_ui",
|
|
"proxy/ui_store_model_db_setting",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Teams & Organizations",
|
|
items: [
|
|
{
|
|
type: "link",
|
|
label: "Role-based Access Controls (RBAC) →",
|
|
href: "/docs/proxy/access_control"
|
|
},
|
|
"proxy/self_serve",
|
|
"proxy/public_teams",
|
|
"proxy/ui_project_management",
|
|
"proxy/ui/bulk_edit_users",
|
|
"proxy/ui/page_visibility",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Observability: Usage",
|
|
items: [
|
|
"proxy/customer_usage",
|
|
"proxy/endpoint_activity",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Logs",
|
|
items: [
|
|
"proxy/ui_logs",
|
|
"proxy/ui_spend_log_settings",
|
|
"proxy/ui_logs_sessions",
|
|
"proxy/deleted_keys_teams",
|
|
]
|
|
}
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Architecture",
|
|
items: [
|
|
"proxy/architecture",
|
|
"proxy/multi_tenant_architecture",
|
|
"proxy/control_plane_and_data_plane",
|
|
"proxy/high_availability_control_plane",
|
|
"proxy/db_deadlocks",
|
|
"proxy/db_info",
|
|
"proxy/image_handling",
|
|
"proxy/jwt_auth_arch",
|
|
"proxy/spend_logs_deletion",
|
|
"proxy/user_management_heirarchy",
|
|
"router_architecture"
|
|
],
|
|
},
|
|
{
|
|
type: "link",
|
|
label: "All Endpoints (Swagger)",
|
|
href: "https://litellm-api.up.railway.app/",
|
|
},
|
|
"proxy/enterprise",
|
|
{
|
|
type: "category",
|
|
label: "Authentication",
|
|
items: [
|
|
"proxy/virtual_keys",
|
|
"proxy/token_auth",
|
|
"proxy/jwt_key_mapping",
|
|
"proxy/service_accounts",
|
|
"proxy/access_control",
|
|
"proxy/cli_sso",
|
|
"proxy/custom_auth",
|
|
"proxy/ip_address",
|
|
"proxy/multiple_admins",
|
|
"proxy/public_routes",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Budgets + Rate Limits",
|
|
items: [
|
|
"proxy/users",
|
|
"proxy/team_budgets",
|
|
"proxy/project_management",
|
|
"proxy/ui_team_soft_budget_alerts",
|
|
"proxy/tag_budgets",
|
|
"proxy/customers",
|
|
"proxy/dynamic_rate_limit",
|
|
"proxy/rate_limit_tiers",
|
|
"proxy/temporary_budget_increase",
|
|
"proxy/budget_reset_and_tz",
|
|
],
|
|
},
|
|
"proxy/caching",
|
|
{
|
|
type: "category",
|
|
label: "Guardrails",
|
|
items: [
|
|
"proxy/guardrails/quick_start",
|
|
"proxy/guardrails/team_based_guardrails",
|
|
"proxy/guardrails/guardrail_load_balancing",
|
|
"proxy/guardrails/test_playground",
|
|
"proxy/guardrails/litellm_content_filter",
|
|
"proxy/guardrails/realtime_guardrails",
|
|
{
|
|
type: "link",
|
|
label: "Providers →",
|
|
href: "/docs/guardrail_providers",
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Contributing to Guardrails",
|
|
items: [
|
|
"adding_provider/generic_guardrail_api",
|
|
"adding_provider/simple_guardrail_tutorial",
|
|
"adding_provider/adding_guardrail_support",
|
|
]
|
|
},
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Policies",
|
|
items: [
|
|
"proxy/guardrails/guardrail_policies",
|
|
"proxy/guardrails/policy_flow_builder",
|
|
"proxy/guardrails/policy_templates",
|
|
"proxy/guardrails/policy_tags",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Create Custom Plugins",
|
|
description: "Modify requests, responses, and more",
|
|
items: [
|
|
"proxy/call_hooks",
|
|
"proxy/rules",
|
|
]
|
|
},
|
|
"proxy/management_cli",
|
|
{
|
|
type: "link",
|
|
label: "Load Balancing, Routing, Fallbacks",
|
|
href: "https://docs.litellm.ai/docs/routing-load-balancing",
|
|
},
|
|
"traffic_mirroring",
|
|
{
|
|
type: "category",
|
|
label: "Logging, Alerting, Metrics",
|
|
items: [
|
|
"proxy/dynamic_logging",
|
|
"proxy/logging",
|
|
"proxy/logging_spec",
|
|
"proxy/team_logging",
|
|
"proxy/email",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Making LLM Requests",
|
|
items: [
|
|
"proxy/user_keys",
|
|
"proxy/clientside_auth",
|
|
"proxy/request_headers",
|
|
"proxy/response_headers",
|
|
"proxy/forward_client_headers",
|
|
"proxy/model_discovery",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Model Access",
|
|
items: [
|
|
"proxy/model_access_guide",
|
|
"proxy/model_access",
|
|
"proxy/model_access_groups",
|
|
"proxy/access_groups",
|
|
"proxy/team_model_add"
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Secret Managers",
|
|
items: [
|
|
"secret_managers/overview",
|
|
"secret_managers/aws_secret_manager",
|
|
"secret_managers/aws_kms",
|
|
"secret_managers/azure_key_vault",
|
|
"secret_managers/cyberark",
|
|
"secret_managers/google_secret_manager",
|
|
"secret_managers/google_kms",
|
|
"secret_managers/hashicorp_vault",
|
|
"secret_managers/custom_secret_manager",
|
|
"oidc"
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Spend Tracking",
|
|
items: [
|
|
"proxy/cost_tracking",
|
|
"tutorials/vertex_ai_pay_go",
|
|
"proxy/request_tags",
|
|
"proxy/custom_pricing",
|
|
"proxy/pricing_calculator",
|
|
"proxy/provider_margins",
|
|
"proxy/provider_discounts",
|
|
"proxy/sync_models_github",
|
|
"proxy/billing",
|
|
],
|
|
},
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Supported Endpoints",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Supported Endpoints",
|
|
description:
|
|
"Learn how to deploy + call models from different providers on LiteLLM",
|
|
slug: "/supported_endpoints",
|
|
},
|
|
items: [
|
|
{
|
|
type: "link",
|
|
label: "/a2a - A2A Agent Gateway",
|
|
href: "/docs/a2a",
|
|
},
|
|
"assistants",
|
|
"audio_transcription",
|
|
"text_to_speech",
|
|
{
|
|
type: "category",
|
|
label: "/batches",
|
|
items: [
|
|
"batches",
|
|
"proxy/managed_batches",
|
|
]
|
|
},
|
|
"containers",
|
|
"container_files",
|
|
{
|
|
type: "category",
|
|
label: "/chat/completions",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Chat Completions",
|
|
description: "Details on the completion() function",
|
|
slug: "/completion",
|
|
},
|
|
items: [
|
|
"completion/input",
|
|
"completion/output",
|
|
"completion/usage",
|
|
"completion/http_handler_config",
|
|
],
|
|
},
|
|
"text_completion",
|
|
"bedrock_converse",
|
|
"embedding/supported_embedding",
|
|
{
|
|
type: "category",
|
|
label: "/files",
|
|
items: [
|
|
"files_endpoints",
|
|
"proxy/litellm_managed_files",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "/fine_tuning",
|
|
items: [
|
|
"fine_tuning",
|
|
"proxy/managed_finetuning",
|
|
]
|
|
},
|
|
"evals_api",
|
|
"generateContent",
|
|
"apply_guardrail",
|
|
"bedrock_invoke",
|
|
"interactions",
|
|
"image_edits",
|
|
"image_generation",
|
|
"image_variations",
|
|
"videos",
|
|
"vector_store_files",
|
|
"vector_stores/create",
|
|
"vector_stores/search",
|
|
{
|
|
type: "category",
|
|
label: "/mcp - Model Context Protocol",
|
|
items: [
|
|
"mcp",
|
|
"mcp_usage",
|
|
"mcp_openapi",
|
|
"mcp_oauth",
|
|
"mcp_aws_sigv4",
|
|
"mcp_zero_trust",
|
|
"mcp_public_internet",
|
|
"mcp_semantic_filter",
|
|
"mcp_control",
|
|
"mcp_cost",
|
|
"mcp_guardrail",
|
|
"mcp_zero_trust",
|
|
"mcp_troubleshoot",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "/v1/messages",
|
|
items: [
|
|
"anthropic_unified/index",
|
|
"anthropic_unified/structured_output",
|
|
"anthropic_unified/messages_to_responses_mapping",
|
|
]
|
|
},
|
|
"anthropic_count_tokens",
|
|
"moderation",
|
|
"ocr",
|
|
{
|
|
type: "category",
|
|
label: "Pass-through Endpoints (Anthropic SDK, etc.)",
|
|
items: [
|
|
"pass_through/intro",
|
|
"pass_through/anthropic_completion",
|
|
"pass_through/assembly_ai",
|
|
"pass_through/bedrock",
|
|
"pass_through/azure_passthrough",
|
|
"pass_through/cohere",
|
|
"pass_through/cursor",
|
|
"pass_through/google_ai_studio",
|
|
"pass_through/langfuse",
|
|
"pass_through/mistral",
|
|
"pass_through/openai_passthrough",
|
|
{
|
|
type: "category",
|
|
label: "Vertex AI",
|
|
items: [
|
|
"pass_through/vertex_ai",
|
|
"pass_through/vertex_ai_live_websocket",
|
|
"pass_through/vertex_ai_search_datastores",
|
|
]
|
|
},
|
|
"pass_through/vllm",
|
|
"proxy/pass_through",
|
|
"proxy/pass_through_guardrails"
|
|
]
|
|
},
|
|
"rag_ingest",
|
|
"rag_query",
|
|
"realtime",
|
|
"proxy/realtime_webrtc",
|
|
"rerank",
|
|
"response_api",
|
|
"prompt_management",
|
|
"response_api_compact",
|
|
{
|
|
type: "category",
|
|
label: "/search",
|
|
items: [
|
|
"search/index",
|
|
"search/perplexity",
|
|
"search/tavily",
|
|
"search/exa_ai",
|
|
"search/brave",
|
|
"search/parallel_ai",
|
|
"search/google_pse",
|
|
"search/dataforseo",
|
|
"search/firecrawl",
|
|
"search/searxng",
|
|
"search/linkup",
|
|
"search/serper",
|
|
]
|
|
},
|
|
"skills",
|
|
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Supported Models & Providers",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Providers",
|
|
description:
|
|
"Learn how to deploy + call models from different providers on LiteLLM",
|
|
slug: "/providers",
|
|
},
|
|
items: [
|
|
{
|
|
type: "doc",
|
|
id: "provider_registration/index",
|
|
label: "Integrate as a Model Provider",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "contributing/adding_openai_compatible_providers",
|
|
label: "Add OpenAI-Compatible Provider (JSON)",
|
|
},
|
|
{
|
|
type: "doc",
|
|
id: "provider_registration/add_model_pricing",
|
|
label: "Add Model Pricing & Context Window",
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "OpenAI",
|
|
items: [
|
|
"providers/openai",
|
|
"providers/openai/responses_api",
|
|
"providers/openai/text_to_speech",
|
|
"providers/openai/videos",
|
|
]
|
|
},
|
|
"providers/text_completion_openai",
|
|
"providers/openai_compatible",
|
|
{
|
|
type: "category",
|
|
label: "Azure OpenAI",
|
|
items: [
|
|
"providers/azure/azure",
|
|
"providers/azure/azure_responses",
|
|
"providers/azure/azure_embedding",
|
|
"providers/azure/azure_speech",
|
|
"providers/azure/videos",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Azure AI",
|
|
items: [
|
|
"providers/azure_ai",
|
|
"providers/azure_ai/azure_model_router",
|
|
"providers/azure_ai_agents",
|
|
"providers/azure_ocr",
|
|
"providers/azure_document_intelligence",
|
|
"providers/azure_ai_speech",
|
|
"providers/azure_ai_img",
|
|
"providers/azure_ai_vector_stores",
|
|
"providers/azure_ai/azure_ai_vector_stores_passthrough",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Vertex AI",
|
|
items: [
|
|
"providers/vertex",
|
|
"providers/vertex_ai/videos",
|
|
"providers/vertex_partner",
|
|
"providers/vertex_self_deployed",
|
|
"providers/vertex_embedding",
|
|
"providers/vertex_image",
|
|
"providers/vertex_speech",
|
|
"providers/vertex_batch",
|
|
"providers/vertex_ocr",
|
|
"providers/vertex_ai_agent_engine",
|
|
"providers/vertex_realtime",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Google AI Studio",
|
|
items: [
|
|
"providers/gemini",
|
|
"providers/gemini/videos",
|
|
"providers/gemini/music",
|
|
"providers/google_ai_studio/files",
|
|
"providers/google_ai_studio/image_gen",
|
|
"providers/google_ai_studio/realtime",
|
|
]
|
|
},
|
|
"providers/anthropic",
|
|
"providers/aws_sagemaker",
|
|
{
|
|
type: "category",
|
|
label: "Bedrock",
|
|
items: [
|
|
"providers/bedrock",
|
|
"providers/bedrock_embedding",
|
|
"providers/bedrock_imported",
|
|
"providers/bedrock_image_gen",
|
|
"providers/bedrock_rerank",
|
|
"providers/bedrock_agentcore",
|
|
"providers/bedrock_agents",
|
|
"providers/bedrock_writer",
|
|
"providers/bedrock_batches",
|
|
"providers/bedrock_realtime_with_audio",
|
|
"providers/aws_polly",
|
|
"providers/bedrock_vector_store",
|
|
"providers/bedrock_mantle",
|
|
]
|
|
},
|
|
"providers/litellm_proxy",
|
|
"providers/abliteration",
|
|
"providers/ai21",
|
|
"providers/aiml",
|
|
"providers/aleph_alpha",
|
|
"providers/amazon_nova",
|
|
"providers/anyscale",
|
|
"providers/apertis",
|
|
"providers/baseten",
|
|
"providers/black_forest_labs",
|
|
"providers/black_forest_labs_img_edit",
|
|
"providers/bytez",
|
|
"providers/cerebras",
|
|
"providers/chutes",
|
|
"providers/clarifai",
|
|
"providers/cloudflare_workers",
|
|
"providers/codestral",
|
|
"providers/cohere",
|
|
"providers/cometapi",
|
|
"providers/compactifai",
|
|
"providers/custom_llm_server",
|
|
"providers/dashscope",
|
|
"providers/databricks",
|
|
"providers/datarobot",
|
|
"providers/deepgram",
|
|
"providers/deepinfra",
|
|
"providers/deepseek",
|
|
"providers/docker_model_runner",
|
|
"providers/elevenlabs",
|
|
"providers/fal_ai",
|
|
"providers/featherless_ai",
|
|
"providers/fireworks_ai",
|
|
"providers/friendliai",
|
|
"providers/galadriel",
|
|
"providers/github",
|
|
"providers/github_copilot",
|
|
"providers/gmi",
|
|
"providers/chatgpt",
|
|
"providers/gradient_ai",
|
|
"providers/groq",
|
|
"providers/helicone",
|
|
"providers/heroku",
|
|
{
|
|
type: "category",
|
|
label: "HuggingFace",
|
|
items: [
|
|
"providers/huggingface",
|
|
"providers/huggingface_rerank",
|
|
]
|
|
},
|
|
"providers/hyperbolic",
|
|
"providers/infinity",
|
|
"providers/jina_ai",
|
|
"providers/lambda_ai",
|
|
"providers/langgraph",
|
|
"providers/lemonade",
|
|
"providers/llamafile",
|
|
"providers/llamagate",
|
|
"providers/lm_studio",
|
|
"providers/manus",
|
|
"providers/meta_llama",
|
|
"providers/milvus_vector_stores",
|
|
"providers/mistral",
|
|
"providers/minimax",
|
|
"providers/moonshot",
|
|
"providers/morph",
|
|
"providers/nebius",
|
|
"providers/nlp_cloud",
|
|
"providers/nano-gpt",
|
|
"providers/novita",
|
|
{ type: "doc", id: "providers/nscale", label: "Nscale (EU Sovereign)" },
|
|
{
|
|
type: "category",
|
|
label: "Nvidia NIM",
|
|
items: [
|
|
"providers/nvidia_nim",
|
|
"providers/nvidia_nim_rerank",
|
|
]
|
|
},
|
|
"providers/oci",
|
|
"providers/ollama",
|
|
"providers/openrouter",
|
|
"providers/sarvam",
|
|
"providers/ovhcloud",
|
|
{
|
|
type: "category",
|
|
label: "Perplexity AI",
|
|
items: [
|
|
"providers/perplexity",
|
|
"providers/perplexity_embedding",
|
|
]
|
|
},
|
|
"providers/petals",
|
|
"providers/poe",
|
|
"providers/publicai",
|
|
"providers/predibase",
|
|
"providers/pydantic_ai_agent",
|
|
"providers/ragflow",
|
|
"providers/recraft",
|
|
"providers/replicate",
|
|
{
|
|
type: "category",
|
|
label: "RunwayML",
|
|
items: [
|
|
"providers/runwayml/images",
|
|
"providers/runwayml/videos",
|
|
]
|
|
},
|
|
"providers/sambanova",
|
|
"providers/sap",
|
|
"providers/scaleway",
|
|
"providers/stability",
|
|
"providers/synthetic",
|
|
"providers/snowflake",
|
|
"providers/togetherai",
|
|
"providers/topaz",
|
|
"providers/triton-inference-server",
|
|
"providers/v0",
|
|
"providers/vercel_ai_gateway",
|
|
{
|
|
type: "category",
|
|
label: "vLLM",
|
|
items: [
|
|
"providers/vllm",
|
|
"providers/vllm_batches",
|
|
]
|
|
},
|
|
"providers/volcano",
|
|
"providers/voyage",
|
|
"providers/wandb_inference",
|
|
{
|
|
type: "category",
|
|
label: "WatsonX",
|
|
items: [
|
|
"providers/watsonx/index",
|
|
"providers/watsonx/audio_transcription",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "xAI",
|
|
items: [
|
|
"providers/xai",
|
|
"providers/xai_realtime",
|
|
]
|
|
},
|
|
"providers/xiaomi_mimo",
|
|
"providers/xinference",
|
|
"providers/zai",
|
|
],
|
|
},
|
|
|
|
|
|
{
|
|
type: "category",
|
|
label: "Routing & Load Balancing",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Routing & Load Balancing",
|
|
description: "Learn how to load balance, route, and set fallbacks for your LLM requests",
|
|
slug: "/routing-load-balancing",
|
|
},
|
|
items: [
|
|
"routing",
|
|
"scheduler",
|
|
"proxy/auto_routing",
|
|
"proxy/load_balancing",
|
|
"proxy/keys_teams_router_settings",
|
|
"proxy/provider_budget_routing",
|
|
"proxy/reliability",
|
|
"proxy/fallback_management",
|
|
"proxy/tag_routing",
|
|
"proxy/timeout",
|
|
"wildcard_routing",
|
|
"proxy/health_check_routing"
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Load Testing",
|
|
items: [
|
|
"benchmarks",
|
|
"load_test_advanced",
|
|
"load_test_sdk",
|
|
"load_test_rpm",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Contributing",
|
|
items: [
|
|
"extras/contributing_code",
|
|
{
|
|
type: "category",
|
|
label: "Adding Providers",
|
|
items: [
|
|
"contributing/adding_openai_compatible_providers",
|
|
"adding_provider/directory_structure",
|
|
"adding_provider/new_rerank_provider",
|
|
]
|
|
},
|
|
"extras/contributing",
|
|
"contributing",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Extras",
|
|
items: [
|
|
"sdk_custom_pricing",
|
|
"migration",
|
|
"data_security",
|
|
"data_retention",
|
|
"proxy/security_encryption_faq",
|
|
"migration_policy",
|
|
{
|
|
type: "category",
|
|
label: "❤️ 🚅 Projects built on LiteLLM",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Projects built on LiteLLM",
|
|
description:
|
|
"Learn how to deploy + call models from different providers on LiteLLM",
|
|
slug: "/project",
|
|
},
|
|
items: [
|
|
"projects/smolagents",
|
|
"projects/mini-swe-agent",
|
|
"projects/openai-agents",
|
|
"projects/Google ADK",
|
|
"projects/Agent Lightning",
|
|
"projects/Harbor",
|
|
"projects/GraphRAG",
|
|
"projects/Docq.AI",
|
|
"projects/PDL",
|
|
"projects/OpenInterpreter",
|
|
"projects/Elroy",
|
|
"projects/dbally",
|
|
"projects/FastREPL",
|
|
"projects/PROMPTMETHEUS",
|
|
"projects/Codium PR Agent",
|
|
"projects/Prompt2Model",
|
|
"projects/SalesGPT",
|
|
"projects/Softgen",
|
|
"projects/Quivr",
|
|
"projects/Langstream",
|
|
"projects/Otter",
|
|
"projects/YiVal",
|
|
"projects/llm_cord",
|
|
"projects/pgai",
|
|
"projects/GPTLocalhost",
|
|
"projects/HolmesGPT",
|
|
"projects/Railtracks",
|
|
],
|
|
},
|
|
"extras/code_quality",
|
|
"rules",
|
|
"proxy/team_based_routing",
|
|
"proxy/customer_routing",
|
|
"proxy_server",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Troubleshooting",
|
|
items: [
|
|
"troubleshoot/ui_issues",
|
|
"mcp_troubleshoot",
|
|
{
|
|
type: "category",
|
|
label: "Performance / Latency",
|
|
items: [
|
|
"troubleshoot/latency_overhead",
|
|
"troubleshoot/cpu_issues",
|
|
"troubleshoot/memory_issues",
|
|
"troubleshoot/spend_queue_warnings",
|
|
"troubleshoot/max_callbacks",
|
|
"troubleshoot/prisma_migrations",
|
|
],
|
|
},
|
|
"troubleshoot/pip_venv_upgrade",
|
|
"troubleshoot/rollback",
|
|
"troubleshoot",
|
|
],
|
|
},
|
|
],
|
|
};
|
|
|
|
const learnSidebar = {
|
|
learnSidebar: [
|
|
// ── Landing page ──────────────────────────────────────────────────
|
|
{ type: "doc", id: "learn/index", label: "Learn" },
|
|
{
|
|
type: "category",
|
|
label: "Start Here",
|
|
collapsible: true,
|
|
collapsed: false,
|
|
items: [
|
|
"learn/sdk_quickstart",
|
|
"learn/gateway_quickstart",
|
|
],
|
|
},
|
|
|
|
// ── Guides ────────────────────────────────────────────────────────
|
|
{
|
|
type: "category",
|
|
label: "Guides",
|
|
collapsible: true,
|
|
collapsed: false,
|
|
link: { type: "doc", id: "guides/index" },
|
|
items: [
|
|
{
|
|
type: "category",
|
|
label: "Core Requests",
|
|
collapsible: true,
|
|
collapsed: false,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Core Requests",
|
|
description: "Streaming, batching, structured outputs, and reasoning behavior",
|
|
slug: "/guides/core_request_response_patterns"
|
|
},
|
|
items: [
|
|
"completion/stream",
|
|
"completion/batching",
|
|
"completion/json_mode",
|
|
"reasoning_content",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Tool Calling",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Tool Calling",
|
|
description: "Function calling, web tools, interception patterns, computer use, code interpreter, and tool-call hygiene",
|
|
slug: "/guides/tools_integrations"
|
|
},
|
|
items: [
|
|
"completion/function_call",
|
|
"completion/web_search",
|
|
{
|
|
type: "doc",
|
|
id: "integrations/websearch_interception",
|
|
label: "Web Search Interception",
|
|
},
|
|
"completion/web_fetch",
|
|
"completion/computer_use",
|
|
"guides/code_interpreter",
|
|
"completion/message_sanitization",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Multimodal I/O",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Multimodal I/O",
|
|
description: "Vision, audio, PDFs, image generation, and video generation",
|
|
slug: "/guides/multimodal_io"
|
|
},
|
|
items: [
|
|
"completion/vision",
|
|
"completion/audio",
|
|
"completion/document_understanding",
|
|
"completion/image_generation_chat",
|
|
"proxy/veo_video_generation",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Retrieval & Knowledge",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Retrieval & Knowledge",
|
|
description: "Vector stores, file search, citations, and knowledge-base routing",
|
|
slug: "/guides/retrieval_knowledge"
|
|
},
|
|
items: [
|
|
"completion/knowledgebase",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Prompts & Context",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Prompts & Context",
|
|
description: "Prompt caching, trimming, formatting, assistant prefill, and predicted outputs",
|
|
slug: "/guides/prompts_context"
|
|
},
|
|
items: [
|
|
"completion/prefix",
|
|
"completion/predict_outputs",
|
|
"completion/message_trimming",
|
|
"completion/prompt_caching",
|
|
"completion/prompt_formatting",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Compatibility & Extensibility",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Compatibility & Extensibility",
|
|
description: "Provider-specific params, model aliases, fine-tuned models, and adapters",
|
|
slug: "/guides/compatibility_extensibility"
|
|
},
|
|
items: [
|
|
"completion/provider_specific_params",
|
|
"completion/drop_params",
|
|
"completion/model_alias",
|
|
"guides/finetuned_models",
|
|
"extras/creating_adapters",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Reliability, Testing & Spend",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Reliability, Testing & Spend",
|
|
description: "Retries, fallbacks, mock responses, and budget controls",
|
|
slug: "/guides/reliability_testing_spend"
|
|
},
|
|
items: [
|
|
"completion/mock_requests",
|
|
"completion/reliable_completions",
|
|
"budget_manager",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Security & Network",
|
|
collapsible: true,
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Security & Network",
|
|
description: "SSL, custom CA bundles, HTTP proxy settings, and per-service verification",
|
|
slug: "/guides/security_network"
|
|
},
|
|
items: [
|
|
"guides/security_settings",
|
|
],
|
|
},
|
|
],
|
|
},
|
|
|
|
// ── Tutorials ─────────────────────────────────────────────────────
|
|
{
|
|
type: "category",
|
|
label: "Tutorials",
|
|
collapsible: true,
|
|
collapsed: false,
|
|
link: { type: "doc", id: "tutorials/index" },
|
|
items: [
|
|
{
|
|
type: "category",
|
|
label: "Getting Started",
|
|
collapsed: false,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Getting Started",
|
|
description: "Installation, playground, text completion, and mock completions",
|
|
slug: "/tutorials/getting_started"
|
|
},
|
|
items: [
|
|
"tutorials/installation",
|
|
"tutorials/first_playground",
|
|
"tutorials/text_completion",
|
|
"tutorials/mock_completion",
|
|
],
|
|
},
|
|
{
|
|
type: "link",
|
|
label: "Agent SDKs & Frameworks",
|
|
href: "/docs/agent_sdks",
|
|
},
|
|
{
|
|
type: "link",
|
|
label: "AI Coding Tools",
|
|
href: "/docs/ai_tools",
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Python SDK",
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Python SDK",
|
|
description: "Tutorials using only the Python SDK — no proxy server required",
|
|
slug: "/tutorials/python_sdk"
|
|
},
|
|
items: [
|
|
"tutorials/gradio_integration",
|
|
"tutorials/provider_specific_params",
|
|
"tutorials/model_fallbacks",
|
|
"tutorials/fallbacks",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Provider Setup",
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Provider Setup",
|
|
description: "Connect LiteLLM to Azure OpenAI, HuggingFace, TogetherAI, local models, and more",
|
|
slug: "/tutorials/provider_tutorials"
|
|
},
|
|
items: [
|
|
"tutorials/azure_openai",
|
|
"tutorials/TogetherAI_liteLLM",
|
|
"tutorials/huggingface_tutorial",
|
|
"tutorials/huggingface_codellama",
|
|
"tutorials/finetuned_chat_gpt",
|
|
"tutorials/oobabooga",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Proxy: Admin & Access",
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Proxy: Admin & Access",
|
|
description: "User and team management, SSO, SCIM, and routing rules",
|
|
slug: "/tutorials/proxy_admin_access"
|
|
},
|
|
items: [
|
|
"tutorials/default_team_self_serve",
|
|
"tutorials/msft_sso",
|
|
"tutorials/scim_litellm",
|
|
"tutorials/tag_management",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Proxy: Features & Safety",
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Proxy: Features & Safety",
|
|
description: "Prompt caching, passthrough APIs, realtime, guardrails, and PII masking",
|
|
slug: "/tutorials/proxy_features_safety"
|
|
},
|
|
items: [
|
|
"tutorials/prompt_caching",
|
|
"tutorials/file_search_responses_api",
|
|
"tutorials/anthropic_file_usage",
|
|
"tutorials/gemini_realtime_with_audio",
|
|
"tutorials/litellm_proxy_aporia",
|
|
"tutorials/presidio_pii_masking",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Observability & Evaluation",
|
|
collapsed: true,
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Observability & Evaluation",
|
|
description: "Logging, monitoring, benchmarking, and evaluation suites",
|
|
slug: "/tutorials/observability_evaluation"
|
|
},
|
|
items: [
|
|
"tutorials/elasticsearch_logging",
|
|
"tutorials/compare_llms",
|
|
"tutorials/litellm_Test_Multiple_Providers",
|
|
"tutorials/eval_suites",
|
|
"tutorials/lm_evaluation_harness",
|
|
],
|
|
},
|
|
],
|
|
},
|
|
],
|
|
};
|
|
|
|
module.exports = { ...sidebars, ...learnSidebar };
|