Commit Graph

5306 Commits

Author SHA1 Message Date
Krrish Dholakia 330cd4f2a9 docs(self_serve.md): update self serve docs to include default team / member budget information 2025-06-24 09:31:33 -07:00
Krish Dholakia a89397a798 Litellm dev 06 23 2025 p1 (#11989)
* fix(litellm_logging.py): fix using router model id for logging calls

Fixes https://github.com/BerriAI/litellm/issues/11975#issuecomment-2995882238

* test(test_litellm_logging.py): add unit test for custom price tracking

* fix(vertex_ai/): don't send invalid format parameter to vertex

causes calls to fail

* fix(vertex_ai_context_caching.py): if cached content present and tools in message, cache tools as well

gemini throws errors if tools passed in alongside cached content

* test: add unit tests

* fix: fix linting errors

* test: test_vertex_ai_common_utils.py

update test

* fix(streaming_handler.py): unset response cost when creating model response
2025-06-23 22:33:06 -07:00
Ishaan Jaff eec0e4d2be docs - web search 2025-06-23 21:25:35 -07:00
Ishaan Jaff b5c48c8c22 [Feat] Add List Callbacks API Endpoint (#11987)
* add get_callbacks_by_type

* add list_callbacks

* fix _get_callback_string

* add callback_management_endpoints_router

* fix proxy config.yaml

* fixes list callbacks

* TestCallbackManagementEndpoints

* update docs

* docs Response Fields

* docs header format

* docs Dynamic Callback Management
2025-06-23 15:34:25 -07:00
Ishaan Jaff 8c5fb6f539 [Feat] Enterprise - Allow dynamically disabling callbacks in request headers (#11985)
* Add support for disabling callbacks via x-litellm-disable-callbacks header

* add _is_callback_disabled_via_headers

* add get_proxy_server_request_headers

* _is_callback_disabled_via_headers

* X_LITELLM_DISABLE_CALLBACKS

* add EnterpriseCallbackControls

* use EnterpriseCallbackControls

* use CustomLoggerRegistry

* use CustomLoggerRegistry

* CustomLoggerRegistry

* EnterpriseCallbackControls

* TestEnterpriseCallbackControls

* docs clean up

* docs dynamic callbacks

* doc fixes

* fix code qa checks

* fix CustomLoggerRegistry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-23 14:32:05 -07:00
Mukesh Sai Kumar a44ff7c486 Fix markdown table not rendering properly (#11969) 2025-06-23 09:28:51 -07:00
Ishaan Jaff 05b95eb702 docs - Recommended Machine Specifications (#11980) 2025-06-23 08:26:29 -07:00
Cole McIntosh 3c22d52d6c Merge branch 'BerriAI:main' into docs/elasticsearch-logging-tutorial 2025-06-22 12:51:26 -06:00
Ishaan Jaff b36dbb870c fix ghcr 2025-06-22 09:06:58 -07:00
Ishaan Jaff 9e5c97aabd fix pip pkg 2025-06-22 09:05:45 -07:00
Krrish Dholakia c0de1320fc docs(index.md): update release notes with default team for new users 2025-06-21 23:09:47 -07:00
Krrish Dholakia 10cf7d52eb docs: add further reading link in self serve docs 2025-06-21 17:03:25 -07:00
Krish Dholakia 3224b0f4fe (Tutorial) Onboard Users for AI Exploration (#11955)
* fix: allow setting no-default-models and unsetting max budget

* docs(sso_self_serve.md): add e2e tutorial of onboarding users for ai exploration

* docs: rename doc
2025-06-21 16:54:23 -07:00
Ishaan Jaff 9867432ef4 docs pass through 2025-06-21 16:41:04 -07:00
Ishaan Jaff 6cd799a78c docs release notes 2025-06-21 16:37:35 -07:00
Ishaan Jaff a3bfd29104 fixes release notes 2025-06-21 15:38:54 -07:00
Ishaan Jaff 30d6301a94 fix release notes 2025-06-21 15:37:12 -07:00
Ishaan Jaff 9aef2cdea6 docs pass through 2025-06-21 15:36:34 -07:00
Ishaan Jaff db219ffca0 docs pass through 2025-06-21 15:24:17 -07:00
Ishaan Jaff c7972b55ce docs - pass through 2025-06-21 15:18:15 -07:00
Ishaan Jaff 0bed17068b docs add imgs 2025-06-21 15:11:58 -07:00
Ishaan Jaff af3c49def3 [Docs] [Pre-Release] v1.73.0-stable (#11950)
* draft 1.73.0

* fixes

* docs pass through

* clean up

* docs fix

* docs fixes

* docs fix

* docs - Logging / Guardrails Integrations

* v1.73.0

* docs fixes

* fixes v2 heath check

* docs fix

* docs vertex img gen

* docs fix

* docs link

* docs azure responses

* azure codex

* docs fixes

* fixes release notes

* fix docs

* docs pre release
2025-06-21 11:38:37 -07:00
Cole McIntosh c7f51c57a6 Merge branch 'BerriAI:main' into docs/elasticsearch-logging-tutorial 2025-06-20 08:39:22 -06:00
Ishaan Jaff b90d3ca8d0 [Feat] MCP - Allow connecting to MCP with authentication headers + Allow clients to specify MCP headers (#11890) (#11891)
* initial mcp auth with special header (#11890)

Co-authored-by: wagnerjt <wagnerjt@github.com>

* add mcp auth header

* fixes MCP client for litellm proxy

* fixes loc of MCP types

* fixes use MCP client for auth to MCPs

* fix organization

* fix mcp auth header

* add MCP auth header to litellm auth

* fixes for MCP auth

* Add MCP auth to list tools

* fix MCP call tool

* fixes for MCP auth header

* tests for MCP transport

* TestMCPClientUnitTests

* docs MCP auth

* fix types

* docs fix

* fix MCP auth import

* fix code qa check

* test fix mcp auth token check

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>
2025-06-19 20:07:08 -07:00
Ishaan Jaff 08b2b4f5f5 [Feat] Enable Tool Calling for meta_llama (#11895)
* Enable Tool Calling for `meta_llama` (#11825)

* feat: enable tools and function_call features

* fix: ignore pydantic warnings for StreamingChoices from llama-api

* docs: add tool calling examples

* docs: change default models to Maverick

* docs: fix output of tool use

* test_map_openai_params

---------

Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
2025-06-19 13:44:22 -07:00
Ishaan Jaff b30bd155d3 fix pip 2025-06-19 12:29:21 -07:00
Ishaan Jaff e90473c40e v1.72.6-stable 2025-06-19 12:21:16 -07:00
Krrish Dholakia b55e1daff9 docs(self_serve.md): document team param 2025-06-18 23:26:05 -07:00
Krrish Dholakia 38eb7c370f docs(sidebars.js): add section for ai tools on integrations tab
make it easier to discover
2025-06-18 22:26:32 -07:00
Krrish Dholakia 6bf66ce919 docs(index.md): add tag 2025-06-18 22:15:30 -07:00
Cole McIntosh e1fbdde289 Fix boto3 tracer wrapping for observability (#11869)
* feat: add summarize parameter to /spend/logs endpoint for individual transaction logs

- Introduced a new `summarize` parameter to control data format when querying spend logs.
- `summarize=true` (default) returns aggregated data, while `summarize=false` provides individual transaction logs.
- Updated documentation and added tests to validate the new functionality.

* fix: wrap boto3.Session() with tracer for observability - Add tracer.trace wrapper around boto3.Session() call in _get_aws_region_name method - Ensures all boto3 initializations in base_aws_llm.py are properly instrumented - Fixes test_boto3_init_tracer_wrapping test failure - Maintains consistency with other boto3 calls in the same file
2025-06-18 15:47:55 -07:00
Shankyg 58dce17414 fix #11857 (#11859) 2025-06-18 14:05:12 -07:00
karen-veigas 5848056c49 Fix #11856: Update billing.md docs to call the new GPT-4o model (#11858)
* Fix: #11853
Updated alerting.md to call the latest model for adding metadata.

* Fix: #11856
Updated billing docs to call newer GPT model for Langchain and CURL
2025-06-18 08:19:36 -07:00
Krrish Dholakia 01af7fe1a8 docs(index.md): update release note with rc patch 2025-06-17 22:55:50 -07:00
Ishaan Jaff 5a5c34c469 MCP docs fix 2025-06-17 20:59:12 -07:00
Nicholas Couture 8bcf163b83 Update bedrock guardrail docs (#11826)
Added the `aws_region_name` and `aws_role_name` keys to the definition of a guardrail config. 

Without these you will get the error `{\"message\":\"The guardrail identifier or version provided in the request does not exist.\"}`
2025-06-17 17:41:41 -07:00
Krrish Dholakia 3c7399e7ec docs(document_understanding.md): clarify support for openai models 2025-06-17 17:40:49 -07:00
Cole McIntosh 3a5b1a404f Update Elasticsearch version and logging exporter configuration in the Elasticsearch logging tutorial 2025-06-17 14:32:59 -06:00
Cole McIntosh 0ae5e4b15b Merge branch 'BerriAI:main' into docs/elasticsearch-logging-tutorial 2025-06-17 13:10:53 -06:00
Cole McIntosh 36d964a758 Fix JSX syntax error in documentation - Fixed unclosed <token> tag in config_settings.md that was causing Vercel deployment to fail (#11818) 2025-06-17 11:39:57 -07:00
Krish Dholakia 30b431681e JWT Auth - correctly return user email + UI Model Update - Allow editing model access group for existing model (#11783)
* fix(handle_jwt.py): check user object, if jwt user is proxy admin

correctly return user role - if jwt user has role updated in UI

* test(test_handle_jwt.py): add unit test for passing correct user role

* feat(model_info_view.tsx): separate UI component for updating edit model component

* feat(model_info_view.tsx): allow updating model access group on UI

show all available access groups in ui component

* docs: minor fixes
2025-06-16 22:11:04 -07:00
Krish Dholakia 87ae2cf3d7 SSO - Allow passing additional headers + Spend Tags - automatically track spend by user agent (allows cost tracking for claude code) (#11781)
* feat(ui_sso.py): allow admin to specify additional headers for sso provider

some sso providers require special headers to return a json response

* test(test_ui_sso.py): add unit tests to ensure custom headers are respect3ed

* docs(config_settings.md): document new header param

* fix(litellm_pre_call_utils.py): add spend tag tracking by user agent

allows checking spend for cli tools like claude code

* feat(litellm_pre_call_utils.py): track spend by user agent part if user agent contains "/"

allows tracking spend across user agent versions

Better cost tracking for claude cod

* test(test_litellm_pre_call_utils.py): add testing for pre call utils, user agent parsing

* fix: fix linting check
2025-06-16 21:53:40 -07:00
Ishaan Jaff 80501b8268 [Feat] Day-0 Support for OpenAI Re-usable prompts Responses API (#11782)
* add prompt to responses params

* add OpenAI PromptObject

* add prompt param to responses api

* test_get_optional_params_responses_api

* test_openai_responses_litellm_router_with_prompt

* docs Reusable Prompts
2025-06-16 21:28:50 -07:00
Ishaan Jaff 7f11b5b76c [Security] - Add Trivy Security Scan for UI + Docs folder - remove all vulnerabilities (#11778)
* docs - sec scan

* ui - fix 0 security issues

* fix - 0 medium vuln on docs

* add MEDIUM scan for Docs

* Run Trivy scan on LiteLLM UI
2025-06-16 17:13:19 -07:00
Ishaan Jaff 261d56b214 [Security] Fixes for docs (#11776)
* fix - docs

* docs fixes

* fixes for docs

* fixes docs
2025-06-16 16:20:55 -07:00
Ishaan Jaff 2f322237ca docs - mcp with litellm config.yaml 2025-06-16 12:35:51 -07:00
Cole McIntosh 11396b936a docs: remove analytics examples from Elasticsearch logging tutorial 2025-06-16 12:01:30 -06:00
Cole McIntosh 05514681cc docs: remove sections from Elasticsearch logging tutorial 2025-06-16 11:48:44 -06:00
Cole McIntosh 631f9a8952 docs: update Elasticsearch logging tutorial to include OpenTelemetry setup and configuration 2025-06-16 10:50:17 -06:00
Cole McIntosh 6fbc79f403 docs: add Elasticsearch logging tutorial and update sidebar 2025-06-16 07:16:18 -06:00