Commit Graph

4012 Commits

Author SHA1 Message Date
Krrish Dholakia bfecab2cc5 docs(index.md): cleanup tag 2025-08-04 14:18:04 -07:00
Krrish Dholakia 98102e2804 docs(index.md): cleanup 2025-08-02 23:06:46 -07:00
Ishaan Jaff 4c217c66f5 docs User Agent Activity Tracking 2025-08-02 17:26:40 -07:00
Ishaan Jaff 2ee4e84406 docs fix 2025-08-02 16:47:44 -07:00
Ishaan Jaff 06856b4d37 docs fix 2025-08-02 15:47:09 -07:00
Ishaan Jaff 0f9f5f7a6c docs fix 2025-08-02 15:44:59 -07:00
Ishaan Jaff 69a360429c agent 4.png 2025-08-02 15:34:14 -07:00
Ishaan Jaff e32169dc37 docs cost tracking coding 2025-08-02 15:29:44 -07:00
Ishaan Jaff 340b64a46a docs - Track Usage for Coding Tools 2025-08-02 15:18:04 -07:00
Ishaan Jaff 9b029c35be docs RC 2025-08-02 15:06:12 -07:00
Ishaan Jaff 8d6b333909 docs computer use 2025-08-02 15:00:48 -07:00
Ishaan Jaff e306fb6eee [docs release notes] (#13237)
* docs release notes

* docs release notes

* docs rnotes

* docs api version

* fixes docs

* docs rn
2025-08-02 14:15:10 -07:00
Ishaan Jaff 5dfc88473f fixes MCP gateway docs 2025-08-02 13:30:51 -07:00
Ishaan Jaff 6929767be2 docs release notes 2025-08-02 12:56:00 -07:00
Jugal D. Bhatt a4c11600a9 [LLM] fix model reload on model update (#13216)
* fix model reload on model update

* remove the flag
2025-08-01 18:08:02 -07:00
Krrish Dholakia 3d02d3602e docs(index.md): cleanup 2025-08-01 15:09:30 -07:00
Krish Dholakia 23dfab3737 Index.md - cleanup docs (#13215)
* docs: add highlights

* docs(index.md): add model-level guardrails

* docs(index.md): cleanup notes

* docs: fix docs

* docs: add more details

* docs(index.md): cleanup doc
2025-08-01 14:57:32 -07:00
Krrish Dholakia 3501ee7e82 docs(index.md): cleanup 2025-08-01 13:56:52 -07:00
Dmitry Tyumentsev 9955521194 allow redifine base api url in pass trough (#13134) 2025-07-31 22:38:33 -07:00
Ishaan Jaff 547c46cd02 add When to Use Each Endpoint (#13193) 2025-07-31 17:16:46 -07:00
Ishaan Jaff ee70d593c1 [Feat] Allow redacting message / response content for specific logging integrations - DD LLM Observability (#13158)
* fix redact_standard_logging_payload

* add StandardCustomLoggerInitParams

* allow defining DatadogLLMObsInitParams

* fix init DataDogLLMObsLogger

* fix import

* update redact_standard_logging_payload_from_model_call_details

* test_dd_llms_obs_redaction

* docs DD logging

* docs DD

* docs DD

* Redacting Messages, Response docs DD LLM Obs

* fix redaction logic

* fix create_llm_obs_payload

* fix logging response

* fixes

* ruff fix

* fix test

* test_dd_llms_obs_redaction

* test_create_llm_obs_payload

* redact_standard_logging_payload_from_model_call_details

* img - dd_llm_obs

* docs DD

* fix linting

* fix linting

* fix mypy

* test_create_llm_obs_payload

* test_create_llm_obs_payload

* fix mock_env_vars

* fix _handle_anthropic_messages_response_logging
2025-07-31 16:44:16 -07:00
Ishaan Jaff 79be436c2b [Feat] Background Health Checks - Allow disabling background health checks for a specific (#13186)
* disable background health checks for specific models

* test_background_health_check_skip_disabled_models

* Disable Background Health Checks For Specific Models
2025-07-31 13:48:35 -07:00
Krrish Dholakia 68177686e3 docs: cleanup 2025-07-30 10:21:45 -07:00
Krrish Dholakia 8c70025a63 docs: cleanup 2025-07-30 08:02:00 -07:00
Ishaan Jaff 5fa2b00c3f [Feat] MLFlow Logging - Allow adding tags for ML Flow logging requests (#13108)
* add mlflow tags

* fixes config

* add litellm mlflow

* test_mlflow_request_tags_functionality

* docs ML flow litellm proxy

* docs ml flow

* docs mlflow
2025-07-29 16:51:27 -07:00
Cole McIntosh 8826e02a98 feat: Add dot notation support for all JWT fields (#13013)
* feat: Add dot notation support for all JWT fields

- Updated all JWT field access methods to use get_nested_value for dot notation support
- Enhanced get_team_id to properly handle team_id_default fallback with nested fields
- Added comprehensive unit tests for nested JWT field access and edge cases
- Updated documentation to reflect dot notation support across all JWT fields
- Maintains full backward compatibility with existing flat field configurations

Supported fields with dot notation:
- team_id_jwt_field, team_ids_jwt_field, user_id_jwt_field
- user_email_jwt_field, org_id_jwt_field, object_id_jwt_field
- end_user_id_jwt_field (roles_jwt_field was already supported)

Example: user_id_jwt_field: 'user.sub' accesses token['user']['sub']

* fix: Add type annotations to resolve mypy errors

- Add explicit type annotation for team_ids variable in get_team_ids_from_jwt
- Add type ignore comment for sentinel object return in get_team_id
- Resolves mypy errors while maintaining functionality

* fix: Resolve mypy type error in get_team_ids_from_jwt

- Remove explicit List[str] type annotation that conflicts with get_nested_value return type
- Simplify return logic to use 'team_ids or []' ensuring always returns List[str]
- Fixes: Incompatible types in assignment (expression has type 'list[str] | None', variable has type 'list[str]')

* fix: Add proper type annotation for team_ids variable

- Use Optional[List[str]] type annotation to satisfy mypy requirements
- Resolves: Need type annotation for 'team_ids' [var-annotated]
- Maintains functionality while ensuring type safety

* refactor: remove outdated JWT unit tests and consolidate JWT-related functionality

- Deleted the test_jwt.py file as it contained outdated and redundant tests.
- Consolidated JWT-related tests into test_handle_jwt.py for better organization and maintainability.
- Updated tests to ensure proper functionality of JWT handling, including token validation and role mapping.
- Enhanced test coverage for JWT field access and nested claims handling.

* test: add comprehensive unit tests for JWT authentication

- Introduced a new test file `test_jwt.py` containing unit tests for JWT authentication.
- Implemented tests for loading configuration with custom role names, validating tokens, and handling team tokens.
- Enhanced coverage for JWT field access, nested claims, and role-based access control.
- Added fixtures for Prisma client and public JWT key generation to support testing.
- Ensured proper handling of valid and invalid tokens, including user and team scenarios.

* revert test_handle_jwt.py

* rename file

* test: remove outdated JWT nesting tests and add new nested field access tests

- Deleted the `test_jwt_nesting.py` file as it contained outdated tests.
- Introduced new tests in `test_handle_jwt.py` to verify nested JWT field access.
- Enhanced coverage for accessing nested values using dot notation and ensured backward compatibility with flat field names.
- Added tests for handling missing nested paths and appropriate default values.
- Improved handling of metadata prefixes in nested field access.

* restore file
2025-07-29 16:51:17 -07:00
Krrish Dholakia 8e3caf4306 docs(index.md): cleanup 2025-07-29 16:31:19 -07:00
Krrish Dholakia b35d20fd93 docs(index.md): add rc docker tag 2025-07-29 16:30:43 -07:00
Krish Dholakia b7c5073d28 Custom Auth - bubble up custom exceptions (#13093)
* fix(enterprise/litellm_enterprise/proxy/auth/user_api_key_auth.py): bubble up exception if type is ProxyException

* docs(custom_auth.md): doc on bubbling up custom exceptions
2025-07-29 16:28:25 -07:00
Ishaan Jaff b5048e2b2a docs AZURE_CERTIFICATE_PASSWORD 2025-07-29 14:25:14 -07:00
Max Rabin 918e46d025 Remove extraneous s in docs (#13079) 2025-07-29 08:39:42 -07:00
Ishaan Jaff 33510120fd docs 2025-07-28 22:02:35 -07:00
Cole McIntosh 3a347922c8 docs: add Qwen Code CLI tutorial (#12915)
- Add new tutorial for integrating Qwen Code CLI with LiteLLM Proxy
- Update sidebar to include Qwen Code CLI in both AI Tools and main Tutorials sections
- Document environment variables for OpenAI-compatible configuration
- Include examples for routing to various providers (Anthropic, OpenAI, Bedrock)
2025-07-28 21:39:02 -07:00
Ishaan Jaff a8371d2cb1 [Feat] Add Google AI Studio Imagen4 model family (#13065)
* add gemini

* add init files

* add get_gemini_image_generation_config

* refactor transform

* TestGoogleImageGen

* fix transform

* fix transform

* add gemini_image_cost_calculator

* add cost tracking for gemini/imagen models

* docs image gen

* docs image gen

* test_get_model_info_gemini
2025-07-28 21:25:40 -07:00
Jugal D. Bhatt ba95541571 [MCP gateway] add url namespacing docs (#13063)
* added the url docs

* Added url change
2025-07-28 17:28:55 -07:00
Alex Strick van Linschoten 75ae43e667 feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration (#12956)
* feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration

This commit brings the langfuse_otel integration to feature parity with the vanilla Langfuse integration by adding support for all metadata fields.

Changes:
- Extended LangfuseSpanAttributes enum with all supported metadata fields:
  - Generation-level: generation_name, generation_id, parent_observation_id, version, mask_input/output
  - Trace-level: trace_user_id, session_id, tags, trace_name, trace_id, trace_metadata, trace_version, trace_release, existing_trace_id, update_trace_keys
  - Debug: debug_langfuse

- Implemented metadata extraction and mapping in langfuse_otel.py:
  - Added _extract_langfuse_metadata() helper to extract metadata from kwargs
  - Support for header-based metadata (langfuse_* headers) via proxy
  - Enhanced _set_langfuse_specific_attributes() to map all metadata to OTEL attributes
  - JSON serialization for complex types (lists, dicts) for OTEL compatibility

- Updated documentation:
  - Added 'Metadata Support' section explaining all fields are now supported
  - Provided usage example showing how to pass metadata
  - Clarified that traces are viewed in Langfuse UI (not generic OTEL backends)
  - Added opentelemetry-exporter-otlp to required dependencies

This allows users to pass metadata like:
metadata={
    'generation_name': 'my-generation',
    'trace_id': 'trace-123',
    'session_id': 'session-456',
    'tags': ['prod', 'v1'],
    'trace_metadata': {'user_type': 'premium'}
}

All metadata is exported as OpenTelemetry span attributes with 'langfuse.*' prefix for easy filtering and analysis in the Langfuse UI.

* Fix ruff linting error

* test(langfuse-otel): Fix failing test and add comprehensive metadata tests

- Fix test_set_langfuse_environment_attribute to use positional arguments
  instead of keyword arguments when asserting safe_set_attribute calls
- Add test_extract_langfuse_metadata_basic to verify metadata extraction
  from litellm_params
- Add test_extract_langfuse_metadata_with_header_enrichment to test
  integration with header-based metadata using a stubbed LangFuseLogger
- Add test_set_langfuse_specific_attributes_full_mapping to comprehensively
  test all metadata field mappings and JSON serialization of complex types

These tests ensure full coverage of the langfuse_otel metadata features
added in commit ab1dbe355 and fix the CI test failure.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-07-28 16:53:36 -07:00
Richard Tweed 3abf7cc871 chore: Improve docs for cost tracking (#12976) 2025-07-28 16:49:14 -07:00
Ishaan Jaff d5c61cc526 docs - openweb show how to include reasoning content (#13060) 2025-07-28 15:25:47 -07:00
Krish Dholakia e8a374fc47 Litellm release notes 07 27 2025 p1 (#13027)
* docs(index.md): initial commit for v1.74.9-stable release note

* docs(index.md): add more cost tracking models

* docs(index.md): add new llm api endpoints + mcp gateway features

* docs: add logging/guardrail improvements

* docs(index.md): complete initial draft

* build(model_prices_and_context_window.json): fix or pricing

* build(model_prices_and_context_window.json): fix or pricing
2025-07-27 09:46:25 -07:00
Jugal D. Bhatt b921e22594 [MCP Gateway] Litellm mcp multi header propagation (#13003)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* add multi server header support

* add and fix tests

* fix tests

* fix tests

* add a common util

* ruff fix

* fix ruff

* fix tests

* fix migration

* mypy fix

* change server py
2025-07-26 11:45:14 -07:00
Jugal D. Bhatt 10595ea077 [MCP Gateway] add Litellm mcp alias for prefixing (#12994)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* fix tests

* add a common util

* ruff fix

* fix migration
2025-07-25 17:57:52 -07:00
Jugal D. Bhatt d1b63566ac remove features from enterprise (#12988) 2025-07-25 10:55:28 -07:00
Ishaan Jaff 0f015a2f68 docs fix link 2025-07-25 09:08:18 -07:00
Ishaan Jaff 1a15bd4885 docs auto router 2025-07-25 09:04:27 -07:00
Ishaan Jaff f2e9834dc6 docs - auto router on litellm proxy 2025-07-25 08:58:54 -07:00
Ishaan Jaff 7bb48ffaf9 docs auto routing 2025-07-25 08:50:16 -07:00
Ishaan Jaff 50574c0c3e docs - auto routing 2025-07-25 07:31:39 -07:00
Ishaan Jaff ecfafd1243 docs update 2025-07-25 07:19:32 -07:00
Ishaan Jaff 7745f61fb1 docs update 2025-07-25 07:19:20 -07:00
Ishaan Jaff 9c76ccd92c docs Health Check Server 2025-07-25 07:17:26 -07:00