Commit Graph
106 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub c98b125851 [Feat] OTEL - Log Cost Breakdown on OTEL Logger (#16334)
* add gen_ai cost metrics

* TestOpenTelemetryCostBreakdown

* fix QA check

* validate_redacted_message_span_attributes
2025-11-06 13:53:53 -08:00
3ef210e5f4 Add allowing Key based prefix to s3 path (#16237)
* add Key based prefix

* add Key based prefix

* update documentation

* correct doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-05 14:43:21 -08:00
Ishaan JaffandGitHub 0737cc7c13 [Feat] s3 logger, add support for ssl_verify when using minio logger (#16211)
* fixes s3_v2 verify

* test_s3_verify_false_async_client

* fix

* ruff fixes
2025-11-03 13:56:00 -08:00
6ed76ff809 feat: change guardrail_information to list type (#16127)
* feat: change guardrail_information to list type to support displaying multiple guardrails

* fix: add missing commit and revert auto-format changes in utils.py

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-11-02 09:47:49 -08:00
Krish DholakiaandGitHub 74ae7aed44 build: Squashed commit of the following: (#16176)
commit bb0b050fb01633d83c1c2932f8e9c11432911847
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Sat Nov 1 20:00:01 2025 -0700

    test: update tests

commit b2da4bdac23868e69a9452805b231f8830e49912
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:58:01 2025 -0700

    fix(langfuse_otel_attributes.py): log tools and other optional params

commit 75bee1f2748f32b230467de0b085c55bf1d687a9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:42:05 2025 -0700

    feat(langfuse_otel/): working request/response logging on spans

    Closes https://github.com/BerriAI/litellm/issues/13764

commit a3e4fa5b81e82f71c74fb9e7dc859c6cb40495f5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:20:39 2025 -0700

    fix: initial commit fixing langfuse request/response logging with OTEL

commit 09fc9deac844004104822810e42975cd9c68f0e3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 13:33:52 2025 -0700

    fix(litellm_logging.py): for responses api - return a unified usage object for logging

    ensures logging integrations all pull the right usage information
2025-11-02 09:46:40 -08:00
99775fa0f8 Support responses API streaming in langfuse otel (#16153)
* streaming support in langfuse otel

* Added testing for Langfuse Otel tracing in the response API

---------

Co-authored-by: eycjur <eycjur@example.com>
2025-11-02 09:36:34 -08:00
20b95e9a80 strip base64 in s3 (#16157)
* strip base64

* strip base64

* s3 use key prefix

* s3 use key prefix

* strip base64 doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-02 09:06:53 -08:00
3a7c498eff Add GitlabPromptCache and enable subfolder access (#15712)
* Add GitlabPromptCache and enable subfolder access

* Add GitlabPromptCache and enable subfolder access

* Add GitlabPromptCache and enable subfolder access

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-10-28 14:46:50 -07:00
2074b4d662 Fix: Support tool usage messages with Langfuse OTEL integration (#15932)
* Log tool use in langfuse otel integration

* Add test for logging function calling

---------

Co-authored-by: eycjur <eycjur@example.com>
2025-10-27 19:47:31 -07:00
Ishaan Jaffer cd0db19750 unstable test 2025-10-25 17:11:13 -07:00
Krish DholakiaandGitHub 346e036399 fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking (#15916)
* fix(opentelemetry.py): fix issue where headers were not being split correctly

* feat(bedrock/image): Support bedrock titan image generation

Closes https://github.com/BerriAI/litellm/issues/361

* build(model_prices_and_context_window.json): track titan image gen pricing

enables cost tracking per request

* feat(amazon_titan_transformation.py): support titan image generation cost tracking

* docs: document new model

* docs: update docs to indicate cost tracking + refactor rerank into separate doc

* fix: fix mypy linting error

* fix: fix type ignore
2025-10-25 13:45:13 -07:00
Krish DholakiaandGitHub 2bd41dc034 Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function (#15706)
* fix(presidio.py): handle content as a list of texts

covers openai + anthropic messages api

* fix(presidio.py): safe get messages

* test: add unit testing for presidio guardrails

* fix(unified_guardrail.py): initial commit

* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail

* fix(unified_guardrail.py): support unified guardrail on input

* feat(unified_guardrail.py): add post call success hook implementation

allows us to just have 1 place to handle llm translation to guardrail api spec

* refactor: refactor initial unified guardrail component

* refactor: more refactoring

* feat(responses/): add guardrails to responses api

allows existing guardrails to work for new llm endpoints

* docs(adding_guardrail_support.md): document new guardrail endpoint support

* test: add unit tests

* feat(image_generation/): add guardrail support for image generation endpoint

* feat(openai/text_completion): support guardrails on `/v1/completions` API

* docs: document guardrails support on new endpoints

* docs: clarify when guardrails run

* feat(openai/speech): add guardrail support for input

* docs(rerank/): add guardrail support on input query

* fix: fix ruff check
2025-10-25 13:38:57 -07:00
Sam ChouandGitHub a173232f8c Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon (#15914)
* Fix mlflow tags - split request_tags into (key, val) if request_tag has colon

* Redundant name: tag_dict -> tags
2025-10-24 12:57:15 -07:00
Sameer KankuteandGitHub 0f9996a4d0 Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
Javier GarciaandGitHub b0a3a7c4fb Add details in docs (#15721)
* Add details in docs

* add logic to set span attributes and unit tests

* Restore html files

* Remove html files

* Remove html files
2025-10-21 16:57:51 -07:00
Tom HaynesandGitHub 98f1d63508 use correct otel logger, and normalise otel paths (#15645) 2025-10-21 09:16:03 -07:00
Ishaan JaffandGitHub 6ab9b0af9f [Fix] Anthropic cache_control incorrectly applied to all content items instead of last item only (#15699)
* fix: _safe_insert_cache_control_in_message

* test_anthropic_cache_control_hook_system_message

* docs prompt cache injection

* docs fix
2025-10-18 15:18:08 -07:00
3ff073c811 UI - add arize on ui, LLMs - clarifai refactor to openai compatible route, added azure ai/grok-4 model family
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44f14b2db1429f6fbd32548a7c95d325.

* fix: arize ui integration

* need to remove a file

This reverts commit d6c877b73ac763464f204b77135f3786342373b7.

* fix: add arize from ui

* updated clarifai functions to openai compatible (#15615)

* fix: npm build errors

* Snowflake provider support: added embeddings, PAT, account_id (#15372)

* snowflake support PAT, account_id and embeddings

* format

* test embeddings

* format

* complete test

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* Revert "Snowflake provider support: added embeddings, PAT, account_id (#15372)" (#15632)

This reverts commit c6d58e5b4af8493c020fa519d72ec6ebc90c896b.

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Mubashir Osmani <ilikewafflesomcuh@gmail.com>
Co-authored-by: mogith-pn <143642606+mogith-pn@users.noreply.github.com>
Co-authored-by: Andrey <elkin.andr@gmail.com>
2025-10-16 20:39:15 -07:00
Krish DholakiaandGitHub 54095a256a Merge pull request #15015 from anthony-liner/fix/langfuse-total-tokens
fix: set usage_details.total in langfuse integration
2025-10-02 17:33:43 -07:00
Krish DholakiaandGitHub f6b67fd9bd Merge pull request #14799 from tyler-liner/chore/generation-name-opentelemetry
fix (opentelemetry): use generation_name for span naming in logging method
2025-10-01 21:39:38 -07:00
68adca04c8 Gitlab based Prompt manager (#14988)
* add prompt

* add prompt

* add prompt

* add prompt

* add prompt management via gitlab

* gitlab client

* gitlab client

* gitlab client

* fix lint issues

* fix lint issues

* remove router changes

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-10-01 18:13:11 -07:00
anthony-liner 9ca73d5504 fix: set usage_details.total in langfuse integration 2025-09-29 16:30:44 +09:00
Sameer KankuteandGitHub 1681bf7175 (Feat) Add BitBucket Integration for Prompt Management (#14882)
* Add bitbucket integration for prompt management

* remove not needed file

* remove not needed file

* Add the correct readme

* Add the correct readme

* fix test for bitbucket

* fix test for bitbucket
2025-09-25 15:51:00 -07:00
Alexsander HamirandGitHub eaa04cd8ce fix: use fastuuid helper (#14903)
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
2025-09-25 15:47:01 -07:00
tyler-liner 47800757fd feat(opentelemetry): use generation_name for span naming in logging method 2025-09-23 15:53:41 +09:00
Fabrício CeschinandGitHub f0490ab60d Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-19 13:54:56 -04:00
Krish DholakiaandGitHub 664c83cfb5 Merge branch 'litellm_contributor_prs_09_18_2025_p2' into litellm_dev_09_17_2025_p2_v2 2025-09-18 19:50:55 -07:00
katsuhiro mutoandGitHub ec61a7152a Support for is_streamed_request widh datadog (#14673) 2025-09-18 15:55:16 -07:00
Krrish Dholakia adc71ad2e7 fix(langsmith.py): add langsmith_sampling_rate as a dynamic param
Closes LIT-879
2025-09-17 17:32:26 -07:00
Fabrício CeschinandGitHub 9fdbdaf846 Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-17 09:36:08 -04:00
Krish DholakiaandGitHub fcf84027e8 Merge pull request #14555 from mubashir1osmani/dd_spend_metric
DataDog shows spend metrics
2025-09-16 22:44:47 -07:00
Mubashir OsmaniandGitHub 0fde408e35 added langfuse logging for responses api (#14597)
* added langfuse logging for responses api

* tests added
2025-09-16 21:43:19 -07:00
mubashir1osmani e56b11cb54 added tests 2025-09-17 00:19:03 -04:00
mubashir1osmani f5f00043ce fixed tests 2025-09-16 00:14:48 -04:00
Fabrício CeschinandGitHub c5aa6f540f Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-15 15:51:38 -04:00
Mubashir OsmaniandGitHub 321d5299b2 s3_endpoint_url returned 404 (#14559)
* added spend metrics

* feat: Add Spend metrics in datadog

* fix: lint errors

* fix: s3 endpoint url logging

* fixed lint errors

* remove from branch

This reverts commit e123cae06e.

* Remove from branch

This reverts commit e694cc102a.

* remove "added spend metrics"

This reverts commit 6156590190.
2025-09-15 12:08:18 -07:00
Fabrício CeschinandGitHub 1a715b2eb3 Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-15 09:20:27 -04:00
mubashir1osmani e694cc102a feat: Add Spend metrics in datadog 2025-09-14 14:42:26 -04:00
Krish DholakiaandGitHub 510332b886 Merge pull request #14491 from Rasmusafj/main
Resolve cache key collision issue where all soft budget alerts use identical cache keys
2025-09-14 00:51:27 -07:00
Krish DholakiaandGitHub dc4bbba0a5 Merge pull request #14520 from boopesh07/email_prometheus
Added user_email labels to the prometheus monitoring.
2025-09-13 23:19:46 -07:00
mubashir1osmani 6156590190 added spend metrics 2025-09-13 01:38:08 -04:00
mubashir1osmani fd11159148 fix: DD tool calls passed in metadata 2025-09-13 00:11:18 -04:00
boopesh07 36299dbc73 Added user_email labels to the prometheus monitoring. 2025-09-12 15:38:23 -07:00
rasmus.arpe@gmail.comandruek b8c362e443 add tests for the fix 2025-09-12 12:07:08 +02:00
Fabricio Ceschin faaae888f1 Merge conflicts 2025-09-08 11:34:45 -04:00
Ishaan Jaff b1552cae46 fix mocked CZ tests 2025-09-06 14:33:02 -07:00
Ishaan Jaff fc7accad74 test_dry_run_export_usage_data_cbf_transformation_failure 2025-09-06 13:59:28 -07:00
Ishaan JaffandGitHub 5310bba35b [Feat] Litellm x CloudZero Integration - Cost Tracking (#14296)
* fix: just pull LiteLLM_DailyUserSpend

* get the team_id from user daily spend table

* cloudzero_dry_run_export

* fix CZ endpoints

* trace entity_id

* fix: get_usage_data

* fix get_usage_data

* fix _create_cbf_record

* fix get_usage_data

* ensure start and end time is used for exporting data

* fix init_cloudzero_background_job

* fix CloudZeroExportRequest

* fix initialize_cloudzero_export_job

* fix initialize_cloudzero_export_job

* allow init with env + config.yaml for cloudzero

* fix: init CZ through config.yaml

* fix DRY run on CZ

* TestCloudZeroDryRunEndpoint

* fix: CLOUDZERO_EXPORT_INTERVAL_MINUTES

* fix init_cloudzero_background_job

* fix exporting data

* fix transform

* stash cloudzero docs

* docs: CloudZero

* ruff fix

* fix rendering key alias

* fix polars
2025-09-05 21:29:41 -07:00
Ishaan Jaff 0a60390521 Revert "[Feat] LiteLLM CloudZero Integration updates - using LiteLLM_SpendLogs Table (#12922)"
This reverts commit e3b752d3dc.
2025-09-05 10:04:07 -07:00
Krish DholakiaandGitHub 62d4623d1c Merge pull request #14107 from uc4w6c/feat/add_guardrails_anthropic
feat: Add guardrail to the Anthropic API endpoint
2025-09-04 20:50:43 -07:00