Commit Graph

3983 Commits

Author SHA1 Message Date
Ishaan Jaff b5048e2b2a docs AZURE_CERTIFICATE_PASSWORD 2025-07-29 14:25:14 -07:00
Max Rabin 918e46d025 Remove extraneous s in docs (#13079) 2025-07-29 08:39:42 -07:00
Ishaan Jaff 33510120fd docs 2025-07-28 22:02:35 -07:00
Cole McIntosh 3a347922c8 docs: add Qwen Code CLI tutorial (#12915)
- Add new tutorial for integrating Qwen Code CLI with LiteLLM Proxy
- Update sidebar to include Qwen Code CLI in both AI Tools and main Tutorials sections
- Document environment variables for OpenAI-compatible configuration
- Include examples for routing to various providers (Anthropic, OpenAI, Bedrock)
2025-07-28 21:39:02 -07:00
Ishaan Jaff a8371d2cb1 [Feat] Add Google AI Studio Imagen4 model family (#13065)
* add gemini

* add init files

* add get_gemini_image_generation_config

* refactor transform

* TestGoogleImageGen

* fix transform

* fix transform

* add gemini_image_cost_calculator

* add cost tracking for gemini/imagen models

* docs image gen

* docs image gen

* test_get_model_info_gemini
2025-07-28 21:25:40 -07:00
Jugal D. Bhatt ba95541571 [MCP gateway] add url namespacing docs (#13063)
* added the url docs

* Added url change
2025-07-28 17:28:55 -07:00
Alex Strick van Linschoten 75ae43e667 feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration (#12956)
* feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration

This commit brings the langfuse_otel integration to feature parity with the vanilla Langfuse integration by adding support for all metadata fields.

Changes:
- Extended LangfuseSpanAttributes enum with all supported metadata fields:
  - Generation-level: generation_name, generation_id, parent_observation_id, version, mask_input/output
  - Trace-level: trace_user_id, session_id, tags, trace_name, trace_id, trace_metadata, trace_version, trace_release, existing_trace_id, update_trace_keys
  - Debug: debug_langfuse

- Implemented metadata extraction and mapping in langfuse_otel.py:
  - Added _extract_langfuse_metadata() helper to extract metadata from kwargs
  - Support for header-based metadata (langfuse_* headers) via proxy
  - Enhanced _set_langfuse_specific_attributes() to map all metadata to OTEL attributes
  - JSON serialization for complex types (lists, dicts) for OTEL compatibility

- Updated documentation:
  - Added 'Metadata Support' section explaining all fields are now supported
  - Provided usage example showing how to pass metadata
  - Clarified that traces are viewed in Langfuse UI (not generic OTEL backends)
  - Added opentelemetry-exporter-otlp to required dependencies

This allows users to pass metadata like:
metadata={
    'generation_name': 'my-generation',
    'trace_id': 'trace-123',
    'session_id': 'session-456',
    'tags': ['prod', 'v1'],
    'trace_metadata': {'user_type': 'premium'}
}

All metadata is exported as OpenTelemetry span attributes with 'langfuse.*' prefix for easy filtering and analysis in the Langfuse UI.

* Fix ruff linting error

* test(langfuse-otel): Fix failing test and add comprehensive metadata tests

- Fix test_set_langfuse_environment_attribute to use positional arguments
  instead of keyword arguments when asserting safe_set_attribute calls
- Add test_extract_langfuse_metadata_basic to verify metadata extraction
  from litellm_params
- Add test_extract_langfuse_metadata_with_header_enrichment to test
  integration with header-based metadata using a stubbed LangFuseLogger
- Add test_set_langfuse_specific_attributes_full_mapping to comprehensively
  test all metadata field mappings and JSON serialization of complex types

These tests ensure full coverage of the langfuse_otel metadata features
added in commit ab1dbe355 and fix the CI test failure.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-07-28 16:53:36 -07:00
Richard Tweed 3abf7cc871 chore: Improve docs for cost tracking (#12976) 2025-07-28 16:49:14 -07:00
Ishaan Jaff d5c61cc526 docs - openweb show how to include reasoning content (#13060) 2025-07-28 15:25:47 -07:00
Krish Dholakia e8a374fc47 Litellm release notes 07 27 2025 p1 (#13027)
* docs(index.md): initial commit for v1.74.9-stable release note

* docs(index.md): add more cost tracking models

* docs(index.md): add new llm api endpoints + mcp gateway features

* docs: add logging/guardrail improvements

* docs(index.md): complete initial draft

* build(model_prices_and_context_window.json): fix or pricing

* build(model_prices_and_context_window.json): fix or pricing
2025-07-27 09:46:25 -07:00
Jugal D. Bhatt b921e22594 [MCP Gateway] Litellm mcp multi header propagation (#13003)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* add multi server header support

* add and fix tests

* fix tests

* fix tests

* add a common util

* ruff fix

* fix ruff

* fix tests

* fix migration

* mypy fix

* change server py
2025-07-26 11:45:14 -07:00
Jugal D. Bhatt 10595ea077 [MCP Gateway] add Litellm mcp alias for prefixing (#12994)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* fix tests

* add a common util

* ruff fix

* fix migration
2025-07-25 17:57:52 -07:00
Jugal D. Bhatt d1b63566ac remove features from enterprise (#12988) 2025-07-25 10:55:28 -07:00
Ishaan Jaff 0f015a2f68 docs fix link 2025-07-25 09:08:18 -07:00
Ishaan Jaff 1a15bd4885 docs auto router 2025-07-25 09:04:27 -07:00
Ishaan Jaff f2e9834dc6 docs - auto router on litellm proxy 2025-07-25 08:58:54 -07:00
Ishaan Jaff 7bb48ffaf9 docs auto routing 2025-07-25 08:50:16 -07:00
Ishaan Jaff 50574c0c3e docs - auto routing 2025-07-25 07:31:39 -07:00
Ishaan Jaff ecfafd1243 docs update 2025-07-25 07:19:32 -07:00
Ishaan Jaff 7745f61fb1 docs update 2025-07-25 07:19:20 -07:00
Ishaan Jaff 9c76ccd92c docs Health Check Server 2025-07-25 07:17:26 -07:00
Dmitriy Alergant 0cc320cc0a GuardrailsAI: use validatedOutput to allow usage of "fix" guards. Previously "fix" guards had no effect in llmOutput mode. (#12891) 2025-07-24 22:07:06 -07:00
Murad Khafizov 650ae0ef88 Added documentation about metadata exposed over the /v1/models endpoint (#12942) 2025-07-24 21:51:16 -07:00
Ishaan Jaff 2bf0c1187c v1.74.7-stable 2025-07-24 18:44:21 -07:00
Ishaan Jaff 095ddea0dc docs index 2025-07-24 18:39:11 -07:00
Ishaan Jaff 0a07698d9d docs update 2025-07-24 11:26:30 -07:00
superpoussin22 ca9712c587 Update control_plane_and_data_plane.md 2025-07-24 16:15:55 +02:00
Ishaan Jaff 99ad24c683 [Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero (#12908)
* add cz init

* add DataAnalyzer

* add CZRNGenerator -> LiteLLM resource generator

* add CloudZeroStreamer

* add CBFTransformer

* add cloudzero DB connection

* add config

* add cli.py

* cleanup

* add CloudZeroLogger

* add CloudZeroLogger

* update CloudZeroLogger

* add cloudzero_router to litellm proxy

* add CloudZeroInitRequest

* add cloudzero spend endpoints

* simple dry run endpoint

* refactor dir structure

* add well types CBFRecord

* TestCBFTransformer

* TestCloudZeroStreamer

* ruff fix

* add polars pip + docker requirements.txt

* _group_by_date

* update code qa check

* docs for CZ params
2025-07-23 13:00:58 -07:00
Ishaan Jaff 5616aaee6d docs pillar guard 2025-07-23 07:15:02 -07:00
Krrish Dholakia 663dea37a8 docs: fix docs 2025-07-22 23:30:50 -07:00
Igal Boxerman d118cc7098 feat: Add Pillar Security guardrail integration (#12791)
* feat: Add Pillar Security guardrail integration

Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.

* fix: Resolve MyPy type error in Pillar guardrail config

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.

* fix: Resolve MyPy type error in Pillar guardrail config

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.

* fix docs

* fix docs

* improved docs

* fix examples, READY
2025-07-22 23:17:33 -07:00
Jugal D. Bhatt 03d5102044 [Add health check] add architecture diagram (#12879)
* add architecture diagram

* add image path

* add new image
2025-07-22 23:12:19 -07:00
Jugal D. Bhatt 405eb7f243 [Docs ]Litellm mcp access group doc (#12883)
* add example for config

* fix typo
2025-07-22 23:11:53 -07:00
Krish Dholakia d26f74ade5 Request Headers - support x-litellm-num-retries + Usage - support usage by model group (#12890)
* feat(litellm_pre_call_utils.py): add num_retries to litellm data for backend call

allow user to pass in num retries via request headers

* test(test_litellm_pre_call_utils.py): add unit test

* docs(request_headers.md): document new request header

* fix(common_daily_activity.py): show spend breakdown by model group

Partial fix for https://github.com/BerriAI/litellm/issues/12887

* feat(new_usage.tsx): new tab switcher for viewing usage by model group vs. received model

Closes https://github.com/BerriAI/litellm/issues/12887
2025-07-22 23:09:36 -07:00
Ishaan Jaff bf300f8ca7 Revert "Litellm dev 07 21 2025 p1 (#12848)"
This reverts commit e4e10aa4ed.
2025-07-22 18:28:36 -07:00
Ishaan Jaff d5ee93aa0c [Feat] Add Recraft API - Image Edits Support (#12874)
* test_recraft_image_edit_api

* add RecraftImageEditConfig

* complete RecraftImageEditConfig

* add RecraftImageEditRequestParams in types

* update RecraftImageEditRequestParams

* working

* transform_image_edit_request

* Image Edit docs recraft

* working transform_image_edit_request

* TestRecraftImageEditTransformation
2025-07-22 15:03:08 -07:00
dependabot[bot] d4900d7dc0 build(deps): bump form-data from 4.0.3 to 4.0.4 in /docs/my-website (#12867)
---
updated-dependencies:
- dependency-name: form-data
  dependency-version: 4.0.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-22 11:23:41 -07:00
Ishaan Jaff 6f0c211788 [Feat] MCP Gateway - allow using MCPs with all LLM APIs when using /responses with LiteLLM (#12546)
* add MCPResponsesAPIHelper

* rename LiteLLM_Proxy_MCP_Handler

* aresponses_api_with_mcp

* mock_responses_api_response

* test response with litellm proxy MCP

* add _should_use_litellm_mcp_gateway

* fix transform_mcp_tool_to_openai_responses_api_tool

* use correct _transform_mcp_tools_to_openai

* fix config.yaml

* fixes for native MCP handling

* docs MCP with litellm proxy

* aresponses_api_with_mcp

* fix linting

* fix mypy

* fix linting

* test_aresponses_api_with_mcp_mock_integration

* docs How it works when server_url="litellm_proxy"
2025-07-22 10:22:45 -07:00
Krrish Dholakia 964c3b7a3d docs(index.md): cleanup 2025-07-22 08:47:52 -07:00
Krrish Dholakia b4469dafd4 docs(index.md): cleanup 2025-07-22 08:47:26 -07:00
Krrish Dholakia 581e96a768 docs(index.md): cleanup 2025-07-22 08:43:33 -07:00
Ishaan Jaff 03baf23ad1 [Feat] Add Recraft Image Generation API Support - New LLM Provider (#12832)
* add recraft

* init RecraftImageGenerationConfig

* add get_complete_url + validate_environment

* add image_generation_handler in llm http clas

* fixes for transform

* working recraft request

* fixed img gen transform

* fixes for llm http handler

* test: TestRecraftImageGeneration

* fixes for llm_http_handler

* fix RecraftImageGenerationConfig

* TestRecraftImageGenerationTransformation

* add recraft API

* docs recraft API

* fix code QA

* map_openai_params

* fix recraft

* cost tracking for recraft/recraftv3

* fix code qa check
2025-07-21 22:19:58 -07:00
Krish Dholakia e4e10aa4ed Litellm dev 07 21 2025 p1 (#12848)
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang

* test(tests/): update tests
2025-07-21 22:09:39 -07:00
Krish Dholakia 70ffdcd210 Passthrough Auth - make Auth checks OSS + Anthropic - only show 'reasoning_effort' for supported models (#12847)
* fix(anthropic/chat/transformation.py): move 'reasoning_effort' to inside 'supports_reasoning' check

Fixes https://github.com/BerriAI/litellm/issues/12833

* fix(proxy_settings_endpoints.py): change order of checks - don't add allowed ip unless db is connected

Fixes https://github.com/BerriAI/litellm/issues/12815

* feat(user_api_key_auth.py): make passthrough auth checks OSS

Fixes https://github.com/BerriAI/litellm/issues/12789

* docs(enterprise.md): clarify on docs
2025-07-21 21:55:41 -07:00
Krrish Dholakia 68a5415a51 docs(config_settings.md): document new flags 2025-07-21 20:19:26 -07:00
Cole McIntosh ff22aed1ea Merge pull request #12826 from colesmcintosh/feature/add-hyperbolic-provider 2025-07-21 20:10:45 -06:00
Ishaan Jaff 83463e3ea5 fix docusaurus package lock 2025-07-21 18:29:17 -07:00
Ishaan Jaff 4ca3e8e617 Docs - litellm benchmarks (#12842)
* docs litellm overhead

* In these tests the baseline latency characteristics
2025-07-21 18:10:30 -07:00
Ishaan Jaff 133c26c015 [Azure OpenAI Feature] - Support DefaultAzureCredential without hard-coded environment variables (#12841)
* DefaultAzureCredential

* update get_azure_ad_token_provider

* fixes for get_azure_ad_token_provider

* test_get_azure_ad_token_provider_with_default_azure_credential

* test_get_azure_ad_token_fallback_to_default_azure_credential

* docs DefaultAzureCredential

* fix linting
2025-07-21 18:04:16 -07:00
Cole McIntosh 34ccda10ed Merge upstream/main - resolve conflicts to include both hyperbolic and recraft providers 2025-07-21 17:38:17 -06:00