Commit Graph

120 Commits

Author SHA1 Message Date
Krish Dholakia e11d34eb69 Permission Management - disable global guardrails by key/team (#16983)
* feat(teams.py): param for disabling guardrails by team

allows use-case where you don't run global guardrails for team - only run team-specific guardrails

* feat(custom_guardrail.py): add support for disabling global guardrails

only run guardrails requested for in the request/key/team

* feat: support adding disable_global_guardrails to metadata if present in key/team metadata

* feat(create_key_button.tsx): new disable global guardrails field

* feat(key_edit_view.tsx): support disabling global guardrails on key edit

* feat(teams.tsx): add disable global guardrails on create team on UI

* feat(team_info.tsx): allow disabling global guardrails on team update
2025-11-22 15:43:50 -08:00
Ishaan Jaffer 725982f39e test_dotprompt_with_prompt_version 2025-11-22 11:00:47 -08:00
Ishaan Jaffer 65b842b6b4 test fix 2025-11-22 10:26:32 -08:00
Ishaan Jaffer 3235807d68 test prompt manager 2025-11-22 10:05:55 -08:00
Ishaan Jaffer 8fbf060ac5 ArizePhoenixConfig 2025-11-22 09:28:37 -08:00
Mubashir Osmani db58f6aeb1 fix: arize phoenix logging (#16301)
* arize phx

* fix arize integration

* traces to specific project name

* fix

* look for http endpoint
2025-11-21 18:46:18 -08:00
Ishaan Jaffer 3296ffd3ca test fixes 2025-11-21 17:38:20 -08:00
Ishaan Jaffer 4e8f1d0143 fix prompt manager 2025-11-21 16:17:44 -08:00
Ishaan Jaff 3ebe489082 [Feat] Prompt Management - Add support for versioning prompts (#16836)
* test_dotprompt_auto_detection_with_model_only

* fix _auto_detect_prompt_management_logger

* test_dotprompt_with_prompt_version

* add v1, v2 tests

* add _compile_prompt_helper

* fix _compile_prompt_helper

* test_dotprompt_with_prompt_version

* test_dotprompt_with_prompt_version, test_get_prompt_with_version
2025-11-19 13:16:03 -08:00
Ishaan Jaff 1f8fe007a1 [Feat] Prompt Management - Allow specifying just prompt_id in a request to a model (#16834)
* test_dotprompt_auto_detection_with_model_only

* fix _auto_detect_prompt_management_logger

* test_dotprompt_auto_detection_with_model_only
2025-11-19 10:20:58 -08:00
Ishaan Jaffer 39c1a970a1 fix 2025-11-14 18:56:30 -08:00
Daniel Sabanov 0ecc38519e [Bug] Updated spend would not be sent to CloudZero (#16201)
* Address a bug where cloudzero spend is not sent to cloudzero if a spend
update happens

* revert change unrelated to PR

* use polars for mocking instead of sqlite
2025-11-10 19:43:15 -08:00
Alan Ponnachan 5f12e4be1e fix(langfuse): Handle null usage values to prevent validation errors (#16396)
* langfuse null validation fix

* formatting
2025-11-10 19:34:56 -08:00
Ishaan Jaffer e4c31add58 test_dd_llms_obs_redaction 2025-11-08 14:23:42 -08:00
Ishaan Jaff c98b125851 [Feat] OTEL - Log Cost Breakdown on OTEL Logger (#16334)
* add gen_ai cost metrics

* TestOpenTelemetryCostBreakdown

* fix QA check

* validate_redacted_message_span_attributes
2025-11-06 13:53:53 -08:00
Deepanshu Lulla 3ef210e5f4 Add allowing Key based prefix to s3 path (#16237)
* add Key based prefix

* add Key based prefix

* update documentation

* correct doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-05 14:43:21 -08:00
Ishaan Jaff 0737cc7c13 [Feat] s3 logger, add support for ssl_verify when using minio logger (#16211)
* fixes s3_v2 verify

* test_s3_verify_false_async_client

* fix

* ruff fixes
2025-11-03 13:56:00 -08:00
YutaSaito 6ed76ff809 feat: change guardrail_information to list type (#16127)
* feat: change guardrail_information to list type to support displaying multiple guardrails

* fix: add missing commit and revert auto-format changes in utils.py

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-11-02 09:47:49 -08:00
Krish Dholakia 74ae7aed44 build: Squashed commit of the following: (#16176)
commit bb0b050fb01633d83c1c2932f8e9c11432911847
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Sat Nov 1 20:00:01 2025 -0700

    test: update tests

commit b2da4bdac23868e69a9452805b231f8830e49912
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:58:01 2025 -0700

    fix(langfuse_otel_attributes.py): log tools and other optional params

commit 75bee1f2748f32b230467de0b085c55bf1d687a9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:42:05 2025 -0700

    feat(langfuse_otel/): working request/response logging on spans

    Closes https://github.com/BerriAI/litellm/issues/13764

commit a3e4fa5b81e82f71c74fb9e7dc859c6cb40495f5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:20:39 2025 -0700

    fix: initial commit fixing langfuse request/response logging with OTEL

commit 09fc9deac844004104822810e42975cd9c68f0e3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 13:33:52 2025 -0700

    fix(litellm_logging.py): for responses api - return a unified usage object for logging

    ensures logging integrations all pull the right usage information
2025-11-02 09:46:40 -08:00
Katsuhiro Muto 99775fa0f8 Support responses API streaming in langfuse otel (#16153)
* streaming support in langfuse otel

* Added testing for Langfuse Otel tracing in the response API

---------

Co-authored-by: eycjur <eycjur@example.com>
2025-11-02 09:36:34 -08:00
Deepanshu Lulla 20b95e9a80 strip base64 in s3 (#16157)
* strip base64

* strip base64

* s3 use key prefix

* s3 use key prefix

* strip base64 doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-02 09:06:53 -08:00
Deepanshu Lulla 3a7c498eff Add GitlabPromptCache and enable subfolder access (#15712)
* Add GitlabPromptCache and enable subfolder access

* Add GitlabPromptCache and enable subfolder access

* Add GitlabPromptCache and enable subfolder access

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-10-28 14:46:50 -07:00
Katsuhiro Muto 2074b4d662 Fix: Support tool usage messages with Langfuse OTEL integration (#15932)
* Log tool use in langfuse otel integration

* Add test for logging function calling

---------

Co-authored-by: eycjur <eycjur@example.com>
2025-10-27 19:47:31 -07:00
Ishaan Jaffer cd0db19750 unstable test 2025-10-25 17:11:13 -07:00
Krish Dholakia 346e036399 fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking (#15916)
* fix(opentelemetry.py): fix issue where headers were not being split correctly

* feat(bedrock/image): Support bedrock titan image generation

Closes https://github.com/BerriAI/litellm/issues/361

* build(model_prices_and_context_window.json): track titan image gen pricing

enables cost tracking per request

* feat(amazon_titan_transformation.py): support titan image generation cost tracking

* docs: document new model

* docs: update docs to indicate cost tracking + refactor rerank into separate doc

* fix: fix mypy linting error

* fix: fix type ignore
2025-10-25 13:45:13 -07:00
Krish Dholakia 2bd41dc034 Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function (#15706)
* fix(presidio.py): handle content as a list of texts

covers openai + anthropic messages api

* fix(presidio.py): safe get messages

* test: add unit testing for presidio guardrails

* fix(unified_guardrail.py): initial commit

* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail

* fix(unified_guardrail.py): support unified guardrail on input

* feat(unified_guardrail.py): add post call success hook implementation

allows us to just have 1 place to handle llm translation to guardrail api spec

* refactor: refactor initial unified guardrail component

* refactor: more refactoring

* feat(responses/): add guardrails to responses api

allows existing guardrails to work for new llm endpoints

* docs(adding_guardrail_support.md): document new guardrail endpoint support

* test: add unit tests

* feat(image_generation/): add guardrail support for image generation endpoint

* feat(openai/text_completion): support guardrails on `/v1/completions` API

* docs: document guardrails support on new endpoints

* docs: clarify when guardrails run

* feat(openai/speech): add guardrail support for input

* docs(rerank/): add guardrail support on input query

* fix: fix ruff check
2025-10-25 13:38:57 -07:00
Sam Chou a173232f8c Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon (#15914)
* Fix mlflow tags - split request_tags into (key, val) if request_tag has colon

* Redundant name: tag_dict -> tags
2025-10-24 12:57:15 -07:00
Sameer Kankute 0f9996a4d0 Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
Javier Garcia b0a3a7c4fb Add details in docs (#15721)
* Add details in docs

* add logic to set span attributes and unit tests

* Restore html files

* Remove html files

* Remove html files
2025-10-21 16:57:51 -07:00
Tom Haynes 98f1d63508 use correct otel logger, and normalise otel paths (#15645) 2025-10-21 09:16:03 -07:00
Ishaan Jaff 6ab9b0af9f [Fix] Anthropic cache_control incorrectly applied to all content items instead of last item only (#15699)
* fix: _safe_insert_cache_control_in_message

* test_anthropic_cache_control_hook_system_message

* docs prompt cache injection

* docs fix
2025-10-18 15:18:08 -07:00
Krish Dholakia 3ff073c811 UI - add arize on ui, LLMs - clarifai refactor to openai compatible route, added azure ai/grok-4 model family
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44f14b2db1429f6fbd32548a7c95d325.

* fix: arize ui integration

* need to remove a file

This reverts commit d6c877b73ac763464f204b77135f3786342373b7.

* fix: add arize from ui

* updated clarifai functions to openai compatible (#15615)

* fix: npm build errors

* Snowflake provider support: added embeddings, PAT, account_id (#15372)

* snowflake support PAT, account_id and embeddings

* format

* test embeddings

* format

* complete test

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* Revert "Snowflake provider support: added embeddings, PAT, account_id (#15372)" (#15632)

This reverts commit c6d58e5b4af8493c020fa519d72ec6ebc90c896b.

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Mubashir Osmani <ilikewafflesomcuh@gmail.com>
Co-authored-by: mogith-pn <143642606+mogith-pn@users.noreply.github.com>
Co-authored-by: Andrey <elkin.andr@gmail.com>
2025-10-16 20:39:15 -07:00
Krish Dholakia 54095a256a Merge pull request #15015 from anthony-liner/fix/langfuse-total-tokens
fix: set usage_details.total in langfuse integration
2025-10-02 17:33:43 -07:00
Krish Dholakia f6b67fd9bd Merge pull request #14799 from tyler-liner/chore/generation-name-opentelemetry
fix (opentelemetry): use generation_name for span naming in logging method
2025-10-01 21:39:38 -07:00
Deepanshu Lulla 68adca04c8 Gitlab based Prompt manager (#14988)
* add prompt

* add prompt

* add prompt

* add prompt

* add prompt management via gitlab

* gitlab client

* gitlab client

* gitlab client

* fix lint issues

* fix lint issues

* remove router changes

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-10-01 18:13:11 -07:00
anthony-liner 9ca73d5504 fix: set usage_details.total in langfuse integration 2025-09-29 16:30:44 +09:00
Sameer Kankute 1681bf7175 (Feat) Add BitBucket Integration for Prompt Management (#14882)
* Add bitbucket integration for prompt management

* remove not needed file

* remove not needed file

* Add the correct readme

* Add the correct readme

* fix test for bitbucket

* fix test for bitbucket
2025-09-25 15:51:00 -07:00
Alexsander Hamir eaa04cd8ce fix: use fastuuid helper (#14903)
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
2025-09-25 15:47:01 -07:00
tyler-liner 47800757fd feat(opentelemetry): use generation_name for span naming in logging method 2025-09-23 15:53:41 +09:00
Fabrício Ceschin f0490ab60d Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-19 13:54:56 -04:00
Krish Dholakia 664c83cfb5 Merge branch 'litellm_contributor_prs_09_18_2025_p2' into litellm_dev_09_17_2025_p2_v2 2025-09-18 19:50:55 -07:00
katsuhiro muto ec61a7152a Support for is_streamed_request widh datadog (#14673) 2025-09-18 15:55:16 -07:00
Krrish Dholakia adc71ad2e7 fix(langsmith.py): add langsmith_sampling_rate as a dynamic param
Closes LIT-879
2025-09-17 17:32:26 -07:00
Fabrício Ceschin 9fdbdaf846 Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-17 09:36:08 -04:00
Krish Dholakia fcf84027e8 Merge pull request #14555 from mubashir1osmani/dd_spend_metric
DataDog shows spend metrics
2025-09-16 22:44:47 -07:00
Mubashir Osmani 0fde408e35 added langfuse logging for responses api (#14597)
* added langfuse logging for responses api

* tests added
2025-09-16 21:43:19 -07:00
mubashir1osmani e56b11cb54 added tests 2025-09-17 00:19:03 -04:00
mubashir1osmani f5f00043ce fixed tests 2025-09-16 00:14:48 -04:00
Fabrício Ceschin c5aa6f540f Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-15 15:51:38 -04:00
Mubashir Osmani 321d5299b2 s3_endpoint_url returned 404 (#14559)
* added spend metrics

* feat: Add Spend metrics in datadog

* fix: lint errors

* fix: s3 endpoint url logging

* fixed lint errors

* remove from branch

This reverts commit e123cae06e.

* Remove from branch

This reverts commit e694cc102a.

* remove "added spend metrics"

This reverts commit 6156590190.
2025-09-15 12:08:18 -07:00