Commit Graph
15 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub 39955129f5 fix mapped tests (#12320)
* fix - use flush llm client cache

* faster mapped tests

* test_async_multiple_response_ids_routing

* fix tests

* test_ateam_member_update_admin_requires_premium

* regular mapped tests

* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"

This reverts commit 2c60c316ec.

* reset num workers
2025-07-04 10:04:43 -07:00
Krish DholakiaandGitHub ee6e76e1f9 Bedrock Passthrough cost tracking (/invoke + /converse routes - streaming + non-streaming) (#12123)
* refactor(passthrough_endpoints-success-handler): refactor llm passthrough logging logic

isolate the llm translation work to enable cost tracking on sdk

* feat: initial implementation of passthrough SDK cost calculation

enables bedrock passthrough cost tracking to work

* feat(cost_calculator.py): working cost calculation for bedrock passthrough

* feat(litellm_logging.py): consider allm_passthrough in cost tracking

allows async calls (e.g. via proxy) to work

* feat(bedrock/passthrough): working event stream decoding for bedrock passthrough calls + logging instrumentation for passthrough sdk calls (log on stream completion)

Enables bedrock streaming cost calculation

* feat(litellm_logging.py): support streaming passthrough cost tracking

* feat(passthrough/main.py): working async streaming cost calculation

Closes https://github.com/BerriAI/litellm/issues/11359

* feat(proxy_server.py): fix passthrough routing when llm router enabled

* feat: further fixes

* feat(bedrock/): working bedrock passthrough cost tracking (non-streaming)

* feat(litellm_logging.py): working usage tracking for bedrock passthrough calls

ensures tokens are logged

* feat(bedrock/passthrough): add converse passthrough cost tracking support

* feat(base_llm/passthrough): remove redundant function

* refactor(litellm_logging.py): refactor function to be below 50 LOC

* test: update test

* test: remove redundant test
2025-06-27 20:01:12 -07:00
Ishaan JaffandGitHub 5b8e300150 [Feat] gemini-cli integration - Add Logging + Cost tracking for stream + non-stream Vertex / Google AI Studio routes (#12058)
* add google generate content to call types

* Revert "add google generate content to call types"

This reverts commit 6f57dde293e50674c8fca0feac6e01c27d9e1c96.

* add CallTypesLiteral for gemini

* allow passing model to vertexpass through logging handler

* update logging handler

* fix checking if stream

* add async streaming logging for vtx

* refactor _transform_google_generate_content_to_openai_model_response

* fix logging_obj

* fixes _handle_non_streaming_google_genai_generate_content_response_logging

* logging callback tests

* ruff check fixes

* test _is_streaming_request

* test_ensure_initialize_azure_sdk_client_always_used

* fix BaseGoogleGenAIGenerateContentStreamingIterator

* fix - linting errors

* req - add google-genai
2025-06-25 22:26:20 -07:00
Ishaan JaffandGitHub d6cc384780 [Feat] OpenAI/Azure OpenAI - Add support for creating vector stores on LiteLLM (#12021)
* add create/acreate vector store

* add azure config

* add _base_validate_azure_environment

* fix base test

* add get_base_create_vector_store_args

* use base llm for headers responses api

* add _get_base_azure_url

* fix AzureOpenAIVectorStoreConfig

* TestAzureOpenAIVectorStore

* fix azure openai vector store

* fix test comment

* fix unused imports

* test_validate_environment_azure_api_key_within_secret_str

* test_azure_transformation.py
2025-06-24 20:46:48 -07:00
Ishaan Jaff 3d542846b2 test fix 2025-06-23 20:03:26 -07:00
Ishaan Jaff 4f98862ef9 test_azure_common_utils.py 2025-06-23 18:34:48 -07:00
hsuyumingandGitHub 180ed14918 fix: fix test_get_azure_ad_token_with_oidc_token testcase issue, because (#11988)
CI/CD pipeline setup AZURE_CLIENT_SECRET within enviroment variable, so
we need to setup as None in this case
2025-06-23 18:11:06 -07:00
hsuyumingandGitHub e3ba888c63 fix: make response api support Azure Authentication method (#11941)
* fix: make response api support Azure Authentication method
1. Support diverse Azure authentication methods
2. Use distinct headers for API key and Azure AD token base on this
   documentation (https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/responses?tabs=rest-api#generate-a-text-response)

* fix: fix path issue

* fix lint error

* rename test_transformation.py to test_azure_transformation.py

* change litellm_params as Optional type
2025-06-23 08:43:20 -07:00
Krish DholakiaandGitHub cf83b541e5 Volcengine - thinking param support + Azure - handle more gpt custom naming patterns (#11914)
* fix(volcengine.py): add thinking param support

Closes https://github.com/BerriAI/litellm/issues/11879

* fix(gpt_transformation.py): handle azure custom names - e.g. `gpt-4-1`

Closes https://github.com/BerriAI/litellm/issues/11834
2025-06-20 09:40:33 -07:00
Krish DholakiaandGitHub 7f8b2579a2 Minor Fixes (#11868)
* fix(litellm_pre_call_utils.py): add user agent tags to spend logs in standard logging payload logic

avoid clash when tag based routing is enabled

* test: remove redundant test

* test: rename oidc test to run earlier

quicker debuging

* fix(azure.py): return more detailed error message

* fix(azure/common_utils.py): use default scope, if scope is none

fixes oidc test

* fix: always default to cognitiveservices.azure.com

* test: update test
2025-06-18 14:12:59 -07:00
Karthick JandGitHub ac4c29c352 feat(azure): Make Azure AD scope configurable (#11621) 2025-06-14 17:43:01 -07:00
Ishaan JaffandGitHub 4dc9626fd5 [Feat] New LLM API Endpoint - Add List input items for Responses API (#11602)
* (feat) add list_input_items

* add alist_input_items to router

* add GET input_items for responses API

* test_basic_openai_list_input_items_endpoint

* TestTransformListInputItemsRequest

* test_ensure_initialize_azure_sdk_client_always_used
2025-06-10 15:47:16 -07:00
Krish DholakiaandGitHub 1995c7aad5 fix(utils.py): support non default params for audio transcription (#11212)
* fix(utils.py): support non default params for audio transcription

allows passing provider specific params straight through on transcription calls

* fix(gpt_transformation.py): fix o_series model routing

call _transform_request on async event

* refactor: refactor tests

* test(test_azure_chat_o_series_transformation.py): add unit test for azure o series error

* test: update test

* test: update json

* fix: fix mutiple keyword error
2025-05-28 22:24:02 -07:00
Ishaan Jaff 1a17755c60 test: fix test_ensure_initialize_azure_sdk_client_always_used 2025-05-27 19:02:11 -07:00
Krish DholakiaandGitHub ef42461c1e Litellm fix GitHub action testing (#11163)
* test: add __init__.py files

* refactor: rename test folder to avoid naming conflict

* test: update workflows

* test: update tests

* test: update imports

* test: update tests

* test: remove unused import

* ci(test-litellm.yml): add pytest retry to github workflow

* test: fix test
2025-05-26 14:41:42 -07:00