Commit Graph
3875 Commits
Author SHA1 Message Date
d8b44f4dbf Enable OpenTelemetry context propagation by external tracers (#15940)
* Fix: Enable OpenTelemetry context propagation with external tracers

* Add test for handling external tracers

* change priority to get tracer

---------

Co-authored-by: eycjur <eycjur@example.com>
2025-10-26 10:09:54 -07:00
Ishaan Jaffer 4fc692d1e3 TestGooglePSESearch 2025-10-25 17:13:45 -07:00
Ishaan Jaffer cd0db19750 unstable test 2025-10-25 17:11:13 -07:00
Ishaan Jaffer cbadcd4a77 TestPerplexityIntegration 2025-10-25 17:06:01 -07:00
Ishaan Jaffer 04ff660276 fixes exception handling 2025-10-25 16:55:16 -07:00
Ishaan Jaffer bbfddd00e4 test fix 2025-10-25 16:46:29 -07:00
Ishaan Jaffer a1d3790198 TestAzureResponsesAPITest 2025-10-25 16:22:52 -07:00
Krrish Dholakia 0f7e1acfc6 test: update tests 2025-10-25 16:14:19 -07:00
Krrish Dholakia 3c0df6a2da test: update unit testing 2025-10-25 16:09:53 -07:00
Ishaan Jaffer f0ae2bef4f TestAzureResponsesAPITest 2025-10-25 16:09:04 -07:00
Ishaan Jaffer 4341495508 search test fix credits 2025-10-25 16:05:44 -07:00
Ishaan Jaffer 3bd42b7ccf test_image_generation_azure_dall_e_3 2025-10-25 15:47:40 -07:00
Krrish Dholakia da3988b768 fix: fix test 2025-10-25 15:43:16 -07:00
Ishaan Jaffer a9c7fbbb60 test_router_init 2025-10-25 15:14:18 -07:00
Ishaan Jaffer 679374fe79 test_img_gen_on_router 2025-10-25 15:11:52 -07:00
Ishaan Jaffer ab0fc0a30d test_aimg_gen_on_router 2025-10-25 15:11:21 -07:00
Ishaan Jaffer 20d8345a7c test: fixes because azure deactivated our account 2025-10-25 15:10:45 -07:00
Ishaan Jaffer e878f2b1ef test_router_get_available_deployments 2025-10-25 15:09:08 -07:00
Ishaan Jaffer e67e4b893d test_completion_azure_ai_gpt_4o_with_flexible_api_base 2025-10-25 14:59:11 -07:00
Ishaan Jaffer 5cca4c8b4f test_image_generation_openai 2025-10-25 14:57:40 -07:00
Ishaan Jaffer 2c52791b83 test_model_function_invoke 2025-10-25 14:56:12 -07:00
Ishaan Jaffer 667f2613de TestAzureOpenAIVectorStore 2025-10-25 14:06:28 -07:00
Krish DholakiaandGitHub 346e036399 fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking (#15916)
* fix(opentelemetry.py): fix issue where headers were not being split correctly

* feat(bedrock/image): Support bedrock titan image generation

Closes https://github.com/BerriAI/litellm/issues/361

* build(model_prices_and_context_window.json): track titan image gen pricing

enables cost tracking per request

* feat(amazon_titan_transformation.py): support titan image generation cost tracking

* docs: document new model

* docs: update docs to indicate cost tracking + refactor rerank into separate doc

* fix: fix mypy linting error

* fix: fix type ignore
2025-10-25 13:45:13 -07:00
Krish DholakiaandGitHub 72bbdfd3f3 (security) Responses API - prevent User A from retrieving User B's response, if response.id is leaked (#15757)
* feat(responses_id_security.py): encrypt response.id - prevent user A from retrieving user B's response

additional security for retrievals on shared accounts

Closes LIT-1307

* feat(responses_id_security.py): allow admin to disable responses id security check

* test: add initial unit testing

* feat(responses_id_security.py): add streaming support

* docs: document new param

* docs: document new param

* feat(responses_id_security.py): add team id checks - ensure it works for service accounts

prevent service accounts keys from different teams from accessing each other's responses

more secure

* test: add unit testing

* fix: fix linting error
2025-10-25 13:41:59 -07:00
Krish DholakiaandGitHub 6bb1d77f45 Org level tpm/rpm limits + Team tpm/rpm validation when assigned to org (#15549)
* fix(support-model-specific-tpm/rpm-limits): Allows setting rate limits by tpm/rpm for models by team

* fix(key_management_endpoints.py): enforce guaranteed throughput with key-level model tpm/rpm limits, when team-level tpm/rpm limits are set

* test: add unit testing

* feat(schema.prisma): add metadata to litellm budget table

* feat(proxy/utils.py): add org limits to user api key auth

allows org level tpm/rpm limiting to work

* feat: add org level tpm/rpm limits + inherit org id in key from team

enables org level tpm/rpm limits

* feat: validated working org tpm/rpm limits

* feat: support updating org level, model specific tpm/rpm limits

* fix: working key validation for org level tpm/rpm limits

* fix: working validation for orgs when giving tpm/rpm to teams

* fix(key_management_endpoints.py): fix tpm/rpm limits on orgs

* fix(key_management_endpoints.py): support limits

* refactor: remove duplicate var

* fix: refactor to avoid ruff errors

* fix: fix typign

* fix: fix linting error

* fix: fix testing

* fix(key_management_endpoints.py): document params
2025-10-25 13:40:29 -07:00
Krish DholakiaandGitHub 2bd41dc034 Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function (#15706)
* fix(presidio.py): handle content as a list of texts

covers openai + anthropic messages api

* fix(presidio.py): safe get messages

* test: add unit testing for presidio guardrails

* fix(unified_guardrail.py): initial commit

* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail

* fix(unified_guardrail.py): support unified guardrail on input

* feat(unified_guardrail.py): add post call success hook implementation

allows us to just have 1 place to handle llm translation to guardrail api spec

* refactor: refactor initial unified guardrail component

* refactor: more refactoring

* feat(responses/): add guardrails to responses api

allows existing guardrails to work for new llm endpoints

* docs(adding_guardrail_support.md): document new guardrail endpoint support

* test: add unit tests

* feat(image_generation/): add guardrail support for image generation endpoint

* feat(openai/text_completion): support guardrails on `/v1/completions` API

* docs: document guardrails support on new endpoints

* docs: clarify when guardrails run

* feat(openai/speech): add guardrail support for input

* docs(rerank/): add guardrail support on input query

* fix: fix ruff check
2025-10-25 13:38:57 -07:00
Ishaan Jaffer 818c44bde5 test_databricks_anthropic_function_call_with_no_schema 2025-10-25 13:01:08 -07:00
Ishaan Jaffer e96c61a3cb test_completion_azure_deployment_id 2025-10-25 13:00:41 -07:00
Ishaan Jaffer 2acedb4826 test_bedrock_apply_guardrail_with_masking 2025-10-25 12:59:19 -07:00
Ishaan Jaffer c9bc6d593c TestAzureOpenAIDalle3 2025-10-25 12:46:38 -07:00
Ishaan Jaffer 44d0cfc648 TestAzureOpenAIO3Mini 2025-10-25 12:42:01 -07:00
Ishaan Jaffer 964e683c85 test_databricks_anthropic_function_call_with_no_schema 2025-10-25 12:29:15 -07:00
Ishaan Jaffer 0bf3d1f226 test_aaaaazure_tenant_id_auth 2025-10-25 12:26:06 -07:00
Ishaan Jaffer e6b61213ca test_completion_azure_deployment_id 2025-10-25 12:26:06 -07:00
Ishaan Jaffer 762053a8e9 test_model_function_invoke 2025-10-25 12:26:06 -07:00
Krrish Dholakia b90e916c68 build: squash merge litellm_dev_10_10_2025_p1 2025-10-25 12:21:12 -07:00
Ishaan Jaffer 6350c20d9f test_azure_streaming_and_function_calling 2025-10-25 12:19:26 -07:00
Ishaan Jaffer cff70ece5a test_azure_astreaming_and_function_calling 2025-10-25 12:18:53 -07:00
Ishaan Jaffer 2b16731273 TestAzureOpenAIO3Mini 2025-10-25 12:13:50 -07:00
Krish DholakiaandGitHub 1543891763 Responses API - support tags in metadata
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors

* fix(vector_store_pre_call_hook.py): Fix https://github.com/BerriAI/litellm/issues/15724

* fix(responses/main.py): have 'tags' work across metadata + litellm_metadata

* fix: add unit testing
2025-10-25 12:07:54 -07:00
Krish DholakiaandGitHub f8d6a6edb9 fix(managed_files.py): don't raise error if managed object is not found + (Feat) Azure AI - Search Vector Stores + (Fix) Batches - “User default_user_id does not have access to the object” when object not in db + (fix) Vector Stores - show config.yaml vector stores on UI (#15873)
* fix(managed_files.py): don't raise error if managed object is not found

* feat(vector_stores): add azure ai search vector store support

Enables direct querying a vector store on azure

* fix(azure/vector_stores): working azure ai search api vector stores

allows azure direct querying on vector stores

* test: update env vars

* docs(docs/): document new azure ai vector store search

* docs(azure_ai_vector_stores.md): add table

* docs: clarify support for 'create' vector stores

* fix(vector_stores/endpoints.py): Fixes https://github.com/BerriAI/litellm/issues/14606

* fix: fix linting errors
2025-10-25 12:06:24 -07:00
Ishaan Jaffer 214c10f6ef test_completion_cost_databricks_embedding 2025-10-25 11:47:03 -07:00
Ishaan Jaffer d47555760e test_azure_transcribe_model_mapping 2025-10-25 11:42:34 -07:00
Ishaan Jaffer 8c8e53c7ac whisper test fix 2025-10-25 11:33:02 -07:00
Ishaan Jaffer e227e8c8a0 mv test_whisper 2025-10-25 11:31:55 -07:00
Ishaan Jaffer 74106589d0 test_completion_azure_ai_gpt_4o_with_flexible_api_base 2025-10-25 11:30:51 -07:00
Ishaan Jaffer c06098c351 test_databricks_embeddings 2025-10-25 11:28:21 -07:00
Ishaan Jaffer a3febef431 test_azure_ai_request_format 2025-10-25 11:23:02 -07:00
Ishaan Jaffer 3dffb6bd40 test fixes 2025-10-25 11:20:09 -07:00
Ishaan Jaffer ec6c166548 _add_azure_related_dynamic_params 2025-10-25 11:11:36 -07:00