Commit Graph

3723 Commits

Author SHA1 Message Date
Ishaan Jaff 08b2b4f5f5 [Feat] Enable Tool Calling for meta_llama (#11895)
* Enable Tool Calling for `meta_llama` (#11825)

* feat: enable tools and function_call features

* fix: ignore pydantic warnings for StreamingChoices from llama-api

* docs: add tool calling examples

* docs: change default models to Maverick

* docs: fix output of tool use

* test_map_openai_params

---------

Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
2025-06-19 13:44:22 -07:00
Ishaan Jaff b30bd155d3 fix pip 2025-06-19 12:29:21 -07:00
Ishaan Jaff e90473c40e v1.72.6-stable 2025-06-19 12:21:16 -07:00
Krrish Dholakia b55e1daff9 docs(self_serve.md): document team param 2025-06-18 23:26:05 -07:00
Krrish Dholakia 38eb7c370f docs(sidebars.js): add section for ai tools on integrations tab
make it easier to discover
2025-06-18 22:26:32 -07:00
Krrish Dholakia 6bf66ce919 docs(index.md): add tag 2025-06-18 22:15:30 -07:00
Cole McIntosh e1fbdde289 Fix boto3 tracer wrapping for observability (#11869)
* feat: add summarize parameter to /spend/logs endpoint for individual transaction logs

- Introduced a new `summarize` parameter to control data format when querying spend logs.
- `summarize=true` (default) returns aggregated data, while `summarize=false` provides individual transaction logs.
- Updated documentation and added tests to validate the new functionality.

* fix: wrap boto3.Session() with tracer for observability - Add tracer.trace wrapper around boto3.Session() call in _get_aws_region_name method - Ensures all boto3 initializations in base_aws_llm.py are properly instrumented - Fixes test_boto3_init_tracer_wrapping test failure - Maintains consistency with other boto3 calls in the same file
2025-06-18 15:47:55 -07:00
Shankyg 58dce17414 fix #11857 (#11859) 2025-06-18 14:05:12 -07:00
karen-veigas 5848056c49 Fix #11856: Update billing.md docs to call the new GPT-4o model (#11858)
* Fix: #11853
Updated alerting.md to call the latest model for adding metadata.

* Fix: #11856
Updated billing docs to call newer GPT model for Langchain and CURL
2025-06-18 08:19:36 -07:00
Krrish Dholakia 01af7fe1a8 docs(index.md): update release note with rc patch 2025-06-17 22:55:50 -07:00
Ishaan Jaff 5a5c34c469 MCP docs fix 2025-06-17 20:59:12 -07:00
Nicholas Couture 8bcf163b83 Update bedrock guardrail docs (#11826)
Added the `aws_region_name` and `aws_role_name` keys to the definition of a guardrail config. 

Without these you will get the error `{\"message\":\"The guardrail identifier or version provided in the request does not exist.\"}`
2025-06-17 17:41:41 -07:00
Krrish Dholakia 3c7399e7ec docs(document_understanding.md): clarify support for openai models 2025-06-17 17:40:49 -07:00
Cole McIntosh 36d964a758 Fix JSX syntax error in documentation - Fixed unclosed <token> tag in config_settings.md that was causing Vercel deployment to fail (#11818) 2025-06-17 11:39:57 -07:00
Krish Dholakia 30b431681e JWT Auth - correctly return user email + UI Model Update - Allow editing model access group for existing model (#11783)
* fix(handle_jwt.py): check user object, if jwt user is proxy admin

correctly return user role - if jwt user has role updated in UI

* test(test_handle_jwt.py): add unit test for passing correct user role

* feat(model_info_view.tsx): separate UI component for updating edit model component

* feat(model_info_view.tsx): allow updating model access group on UI

show all available access groups in ui component

* docs: minor fixes
2025-06-16 22:11:04 -07:00
Krish Dholakia 87ae2cf3d7 SSO - Allow passing additional headers + Spend Tags - automatically track spend by user agent (allows cost tracking for claude code) (#11781)
* feat(ui_sso.py): allow admin to specify additional headers for sso provider

some sso providers require special headers to return a json response

* test(test_ui_sso.py): add unit tests to ensure custom headers are respect3ed

* docs(config_settings.md): document new header param

* fix(litellm_pre_call_utils.py): add spend tag tracking by user agent

allows checking spend for cli tools like claude code

* feat(litellm_pre_call_utils.py): track spend by user agent part if user agent contains "/"

allows tracking spend across user agent versions

Better cost tracking for claude cod

* test(test_litellm_pre_call_utils.py): add testing for pre call utils, user agent parsing

* fix: fix linting check
2025-06-16 21:53:40 -07:00
Ishaan Jaff 80501b8268 [Feat] Day-0 Support for OpenAI Re-usable prompts Responses API (#11782)
* add prompt to responses params

* add OpenAI PromptObject

* add prompt param to responses api

* test_get_optional_params_responses_api

* test_openai_responses_litellm_router_with_prompt

* docs Reusable Prompts
2025-06-16 21:28:50 -07:00
Ishaan Jaff 7f11b5b76c [Security] - Add Trivy Security Scan for UI + Docs folder - remove all vulnerabilities (#11778)
* docs - sec scan

* ui - fix 0 security issues

* fix - 0 medium vuln on docs

* add MEDIUM scan for Docs

* Run Trivy scan on LiteLLM UI
2025-06-16 17:13:19 -07:00
Ishaan Jaff 261d56b214 [Security] Fixes for docs (#11776)
* fix - docs

* docs fixes

* fixes for docs

* fixes docs
2025-06-16 16:20:55 -07:00
Ishaan Jaff 2f322237ca docs - mcp with litellm config.yaml 2025-06-16 12:35:51 -07:00
Krrish Dholakia 06519e3f03 docs: specify when users can expect prod version 2025-06-15 01:39:14 -07:00
Krrish Dholakia 48845065f5 docs: specify rc docker version on release notes 2025-06-15 01:38:25 -07:00
Krrish Dholakia 48ac5a940f docs: cleanup 2025-06-15 01:33:49 -07:00
Krrish Dholakia 30827d0522 docs: cleanup docs 2025-06-15 01:33:06 -07:00
Krish Dholakia bb256c6d83 Release note updates + Responses API Bridge improvements (#11740)
* docs: track which items need docs

* docs(anthropic.md): add tool_choice="none" to docs

* docs: add docs for new anthropic + perplexity features

* docs: cleanup mistral reasoning docs

* docs: add links to docs

* docs(index.md): update docs

* docs: refactor to add a new 'integrations' tab to docs

* refactor(docs/): create separate tab for integrations

make it easier to highlight new integrations

* docs: sort sidebar

* docs: update

* feat: working claude code with openai codex mini

* docs: add responses api to docs

* feat(index.md): update docs

* fix: fix linting error
2025-06-15 01:13:10 -07:00
Krish Dholakia d15664c8e6 Litellm stable docs 06 14 2025 p2 (#11738)
* docs: track which items need docs

* docs(anthropic.md): add tool_choice="none" to docs

* docs: add docs for new anthropic + perplexity features

* docs: cleanup mistral reasoning docs

* docs: add links to docs

* docs(index.md): update docs

* docs: refactor to add a new 'integrations' tab to docs

* refactor(docs/): create separate tab for integrations

make it easier to highlight new integrations

* docs: sort sidebar

* docs: update
2025-06-14 19:38:46 -07:00
Ishaan Jaff 4efdfc00ab docs MCP 2025-06-14 18:13:29 -07:00
Ishaan Jaff 55af010d26 docs mcp 2025-06-14 18:10:37 -07:00
Ishaan Jaff 515e008974 docs mcp 2025-06-14 17:59:28 -07:00
Ishaan Jaff 45a7d1adfe docs mcp 2025-06-14 17:51:45 -07:00
Karthick J ac4c29c352 feat(azure): Make Azure AD scope configurable (#11621) 2025-06-14 17:43:01 -07:00
Ishaan Jaff fc57a53a60 docs - cleanup mcp 2025-06-14 17:31:55 -07:00
Ishaan Jaff a4530b49d8 clean up mcp docs 2025-06-14 17:31:22 -07:00
Ishaan Jaff 001e27aede fix typo 2025-06-14 17:29:50 -07:00
Krish Dholakia 0908618a19 Litellm stable release 06 14 2025 (#11737)
* docs: initial commit with stable release changelog notes

* docs: style updates

* docs(index.md): updated changelog

* docs(index.md): cleanup

* docs(index.md): add general proxy improvements

* docs: index.md

cleanup
2025-06-14 16:56:29 -07:00
Cole McIntosh d1bd68155e Add Langfuse OpenTelemetry Integration (#11607)
* feat(langfuse_otel): add Langfuse OpenTelemetry integration for observability

- Introduced a new integration for Langfuse OpenTelemetry, allowing users to send LiteLLM traces and observability data.
- Updated sidebars to include documentation for the new integration.
- Added example usage and configuration details in the documentation.
- Implemented necessary classes and methods to handle OpenTelemetry attributes and configuration.
- Included tests to validate the integration functionality and environment variable handling.

Still WIP

* Remove example script for Langfuse OpenTelemetry integration with LiteLLM
2025-06-14 14:18:38 -07:00
Ishaan Jaff f79737ad90 docs SCIM enterprise only 2025-06-14 14:03:59 -07:00
Ishaan Jaff b091c1ac4d [Feat] Prometheus - Allow controlling exposed metrics and labels (#11489)
* feat: init prometheus_metrics_config

* feat: add prometheus_metrics_config

* test prometheus

* feat: use factor to init metrics

* fix - pretty print allowed metrics controls

* fix config

* fix - working get_labels_for_metric

* test_no_prometheus_config

* docs update prometheus
2025-06-14 13:56:40 -07:00
Ishaan Jaff 7afbd897b8 docs mcp permissions 2025-06-13 18:32:08 -07:00
Krrish Dholakia 5007ef868f docs: doc updates 2025-06-13 10:21:54 -07:00
Krrish Dholakia b6714cfd4f docs(index.md): add stable pip package 2025-06-13 10:21:54 -07:00
Krrish Dholakia bb75d7e240 docs(caching.md): remove batch redis get recommendation - old code path, no longer necessary 2025-06-12 19:59:45 -07:00
Ishaan Jaff 61838bbbfc [Feat] Use dedicated Rest endpoints for list, calling MCP tools (#11684)
* fix: (fix) use specific rest endpoints for MCP

* ui - use rest mcp endpoints

* fix imports

* docs DISABLE_AIOHTTP_TRUST_ENV
2025-06-12 19:02:20 -07:00
Cole McIntosh 6b9754e2aa Merge pull request #11642 from colesmcintosh/mistral-reasoning
Enhance Mistral model support with reasoning capabilities
2025-06-12 16:42:53 -06:00
Ishaan Jaff 470dc55d48 docs - fix link 2025-06-12 11:24:34 -07:00
Ishaan Jaff e40ffa306c fix img link 2025-06-12 11:22:40 -07:00
Ishaan Jaff 6657012f07 docs release note 2025-06-12 11:11:00 -07:00
Cole McIntosh bee41c1961 Update Mistral documentation and enhance reasoning prompt handling
- Revised the reasoning support indicators in the Mistral model documentation for clarity.
- Improved the `_add_reasoning_system_prompt_if_needed` method to handle both string and list content types for system messages, ensuring the reasoning prompt is correctly prepended.
- Added a new test case to verify the functionality of adding the reasoning system prompt when the existing content is a list.
2025-06-12 11:17:48 -06:00
Cole McIntosh c5f91b9d77 Merge branch 'BerriAI:main' into mistral-reasoning 2025-06-12 11:04:57 -06:00
Krrish Dholakia e4c89135f5 docs(index.md): clarify pip install will be live by eod 2025-06-11 19:04:17 -07:00