Commit Graph
22648 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub 87ae2cf3d7 SSO - Allow passing additional headers + Spend Tags - automatically track spend by user agent (allows cost tracking for claude code) (#11781)
* feat(ui_sso.py): allow admin to specify additional headers for sso provider

some sso providers require special headers to return a json response

* test(test_ui_sso.py): add unit tests to ensure custom headers are respect3ed

* docs(config_settings.md): document new header param

* fix(litellm_pre_call_utils.py): add spend tag tracking by user agent

allows checking spend for cli tools like claude code

* feat(litellm_pre_call_utils.py): track spend by user agent part if user agent contains "/"

allows tracking spend across user agent versions

Better cost tracking for claude cod

* test(test_litellm_pre_call_utils.py): add testing for pre call utils, user agent parsing

* fix: fix linting check
2025-06-16 21:53:40 -07:00
Ishaan JaffandGitHub 80501b8268 [Feat] Day-0 Support for OpenAI Re-usable prompts Responses API (#11782)
* add prompt to responses params

* add OpenAI PromptObject

* add prompt param to responses api

* test_get_optional_params_responses_api

* test_openai_responses_litellm_router_with_prompt

* docs Reusable Prompts
2025-06-16 21:28:50 -07:00
Abinand PandGitHub 99c2a7fb70 feat: update the feature of ollama_embeddings to work on a sync api (#11746)
* feat: update the feature of ollama_embeddings to work on a sync api

Signed-off-by: Abinand P <abinand0911@gmail.com>

* lint:fixing of the lint file

Signed-off-by: Abinand P <abinand0911@gmail.com>

* fix:test

Signed-off-by: Abinand P <abinand0911@gmail.com>

* chore: added test for ollama embedding and refactored handler

Signed-off-by: Abinand P <abinand0911@gmail.com>

* fix:lint error

Signed-off-by: Abinand P <abinand0911@gmail.com>

---------

Signed-off-by: Abinand P <abinand0911@gmail.com>
2025-06-16 19:07:33 -07:00
Krrish Dholakia bb907b5ecc test: fix test 2025-06-16 18:49:41 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
4e313e517b build(deps): bump next from 14.2.26 to 14.2.30 in /ui/litellm-dashboard (#11720)
Bumps [next](https://github.com/vercel/next.js) from 14.2.26 to 14.2.30.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.2.26...v14.2.30)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 14.2.30
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-16 17:33:11 -07:00
MadsRCandGitHub 2224fd96be AWS credentials no longer mandatory (#11765) 2025-06-16 17:32:10 -07:00
Ishaan JaffandGitHub 49b957d14c fix aiohttp version (#11777) 2025-06-16 17:31:14 -07:00
35d74af5a0 [Fix] Pass through - Langfuse don't log request to Langfuse passthrough on Langfuse (#11768)
* dont log lf pass through request

* test_is_langfuse_route

* Update litellm/proxy/pass_through_endpoints/success_handler.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-16 17:30:23 -07:00
Ishaan JaffandGitHub 7f11b5b76c [Security] - Add Trivy Security Scan for UI + Docs folder - remove all vulnerabilities (#11778)
* docs - sec scan

* ui - fix 0 security issues

* fix - 0 medium vuln on docs

* add MEDIUM scan for Docs

* Run Trivy scan on LiteLLM UI
2025-06-16 17:13:19 -07:00
Ishaan JaffandGitHub 261d56b214 [Security] Fixes for docs (#11776)
* fix - docs

* docs fixes

* fixes for docs

* fixes docs
2025-06-16 16:20:55 -07:00
Ishaan Jaff 2f322237ca docs - mcp with litellm config.yaml 2025-06-16 12:35:51 -07:00
shagunb-acnandGitHub 441d579d1b #11748: Added Mistral Small to BEDROCK_CONVERSE_MODELS for Converse API support as it supports natively (#11760) 2025-06-16 12:27:26 -07:00
tanjiroandGitHub bc9f111ee8 move buttons to the left (#11771) 2025-06-16 12:27:04 -07:00
Emerson GomesandGitHub b21f4a3f74 Add Vertex Imagen-4 models (#11767) 2025-06-16 10:08:51 -07:00
Cole McIntoshandGitHub 6d373876d2 fix(prometheus.py): always initialize label_filters for PrometheusLogger (#11764) 2025-06-16 09:20:57 -07:00
Krrish Dholakia 06519e3f03 docs: specify when users can expect prod version 2025-06-15 01:39:14 -07:00
Krrish Dholakia 48845065f5 docs: specify rc docker version on release notes 2025-06-15 01:38:25 -07:00
Krrish Dholakia 48ac5a940f docs: cleanup 2025-06-15 01:33:49 -07:00
Krrish Dholakia 30827d0522 docs: cleanup docs 2025-06-15 01:33:06 -07:00
Krish DholakiaandGitHub 7a128e2017 VertexAI Anthropic - streaming passthrough cost tracking (#11734)
* feat(vertex_passthrough_logging_handler.py): initial anthropic passthrough streaming cost tracking support

* fix: fix linting errors

* test: update test
2025-06-15 01:16:43 -07:00
Krish DholakiaandGitHub bb256c6d83 Release note updates + Responses API Bridge improvements (#11740)
* docs: track which items need docs

* docs(anthropic.md): add tool_choice="none" to docs

* docs: add docs for new anthropic + perplexity features

* docs: cleanup mistral reasoning docs

* docs: add links to docs

* docs(index.md): update docs

* docs: refactor to add a new 'integrations' tab to docs

* refactor(docs/): create separate tab for integrations

make it easier to highlight new integrations

* docs: sort sidebar

* docs: update

* feat: working claude code with openai codex mini

* docs: add responses api to docs

* feat(index.md): update docs

* fix: fix linting error
2025-06-15 01:13:10 -07:00
Krish DholakiaandGitHub d15664c8e6 Litellm stable docs 06 14 2025 p2 (#11738)
* docs: track which items need docs

* docs(anthropic.md): add tool_choice="none" to docs

* docs: add docs for new anthropic + perplexity features

* docs: cleanup mistral reasoning docs

* docs: add links to docs

* docs(index.md): update docs

* docs: refactor to add a new 'integrations' tab to docs

* refactor(docs/): create separate tab for integrations

make it easier to highlight new integrations

* docs: sort sidebar

* docs: update
2025-06-14 19:38:46 -07:00
Ishaan Jaff 4efdfc00ab docs MCP 2025-06-14 18:13:29 -07:00
Ishaan Jaff 55af010d26 docs mcp 2025-06-14 18:10:37 -07:00
Ishaan Jaff 515e008974 docs mcp 2025-06-14 17:59:28 -07:00
Ishaan Jaff 45a7d1adfe docs mcp 2025-06-14 17:51:45 -07:00
Karthick JandGitHub ac4c29c352 feat(azure): Make Azure AD scope configurable (#11621) 2025-06-14 17:43:01 -07:00
Ishaan Jaff fc57a53a60 docs - cleanup mcp 2025-06-14 17:31:55 -07:00
Ishaan Jaff a4530b49d8 clean up mcp docs 2025-06-14 17:31:22 -07:00
Ishaan Jaff 001e27aede fix typo 2025-06-14 17:29:50 -07:00
Krrish Dholakia e50647627c build(ghcr_deploy.yml): add rc to all docker images v1.72.6.rc v1.72.6.dev1 2025-06-14 17:16:46 -07:00
Krrish Dholakia a5157978aa build(ghrc_deploy.yml): add 'rc' release type 2025-06-14 17:15:29 -07:00
Krish DholakiaandGitHub 0908618a19 Litellm stable release 06 14 2025 (#11737)
* docs: initial commit with stable release changelog notes

* docs: style updates

* docs(index.md): updated changelog

* docs(index.md): cleanup

* docs(index.md): add general proxy improvements

* docs: index.md

cleanup
2025-06-14 16:56:29 -07:00
Ishaan Jaff 327868fff6 speed up llm_translation_testing 2025-06-14 16:56:02 -07:00
Ishaan Jaff 355e6118d8 def test_text_completion_stream(): 2025-06-14 16:46:09 -07:00
Ishaan Jaff 363d6b6858 test_xai_check_for_stop_in_supported_params 2025-06-14 16:33:38 -07:00
Ishaan Jaff d477c1915d fix - speed up litellm_proxy_unit_testing 2025-06-14 16:30:28 -07:00
Ishaan Jaff 248f450767 ui new build 2025-06-14 16:28:55 -07:00
Krish DholakiaandGitHub c92b6c175c Prometheus - fix request increment + add route tracking for streaming requests (#11731)
* fix(prometheus.py): remove request increment from inside the log success event

it's only done on post-call success/failure

* fix(litellm_logging.py): add additional validation step for checking if 'stream' is true

prevent double counting on non-stream requests

* test: add unit testing to ensure stream is not incorrectly set to true

* feat(litellm_logging.py): emit request route in standard logging payload

used by prometheus streaming metrics for route

* fix: fix otel test

* fix: fix linting errors

* test: update test

* fix: fix linting error
2025-06-14 16:26:48 -07:00
Ishaan Jaff c96325aefa fix code qa check 2025-06-14 15:56:00 -07:00
Ishaan JaffandGitHub c233e86e45 rollback dd trace (#11736) 2025-06-14 15:52:29 -07:00
Ishaan JaffandGitHub c40580f892 [Fix] JWT - Fix error when team member already part of team (#11735)
* fix _check_member_duplication

* fix map_user_to_teams

* test_map_user_to_teams_handles_already_in_team_exception

* test_team_endpoints.py
2025-06-14 15:50:16 -07:00
Ishaan Jaff 1c1e41c51d Revert "Implement health check backend API and storage functionality (#11678)"
This reverts commit 5f34ceea1a.
2025-06-14 15:45:33 -07:00
Ishaan Jaff ad82792c4b fix lf OTEL 2025-06-14 15:43:38 -07:00
Ishaan Jaff c59772bae9 fix install on python 2025-06-14 15:42:06 -07:00
Ishaan JaffandGitHub d4107dfd68 [Bug Fix] Ensure /redis/info works on GCP Redis (#11732)
* fix cache_redis_info

* mock_redis_client_list_restricted
2025-06-14 15:35:09 -07:00
Jason DaiandGitHub 4100d1f97d fix: re-export get_valid_models utils in init file (#11721)
* fix: re-export get_valid_models utils in init file

* move it to the correct utils
2025-06-14 15:13:38 -07:00
Ishaan Jaff 6a6ea33ca8 get_custom_url 2025-06-14 14:29:24 -07:00
Cole McIntoshandGitHub 5f34ceea1a Implement health check backend API and storage functionality (#11678)
* feat: Add health check functionality and endpoints

- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.

* test: Add unit tests for health check functionality

- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.

* Refactor health endpoint model ID handling and improve logging

- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.

* Refactor utility functions in proxy module for improved readability and error handling

- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.

* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness

- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.

* Fix disable_spend_updates method to handle None return value gracefully

- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.

* Refactor join_paths function in utils.py for improved path handling

- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.

* Enhance health check functionality and improve error handling

- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.

* Add health check methods for database interaction

- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.

* Refactor health check result saving to use typed arguments

- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.

* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.

* Implement response time validation and details cleaning in health check result saving

- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.

* Add health check utility functions and refactor existing endpoints

- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.

* Refactor health check tests for improved clarity and coverage

- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.

* Implement helper function for health check and database saving

- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.
2025-06-14 14:27:11 -07:00
nevinandGitHub b7cb66ee8f Fixed grok-3-mini to not use stop tokens (#11563)
* fixed grok-3-mini to not use stop tokens

* added xai config test
2025-06-14 14:26:43 -07:00