* fix(helicone.py): add helicone api base support
Fixes https://github.com/BerriAI/litellm/issues/10825
* test: add unit test for cache hit response on embedding calls
* fix(caching_handler.py): fix handling cache hit on embedding when input is string
Fixes LIT-197
* docs(helicone_integration.md): document new helicone api base param
* fix(vertex_and_google_ai_studio_gemini.py): handle both camel case and underscores in the tool for vertex ai code execution
support vertex ai code execution
* docs(vertex.md): add code execution example to vertex ai
* fix(vertex_ai/common_utils.py): when anyof in field, just select anyof - don't include other k,v pairs - vertex throws error
Fixes https://github.com/BerriAI/litellm/issues/11164
* fix(common_utils.py): add title field inside anyof - to retain some description
Addresses https://github.com/BerriAI/litellm/issues/11164#issuecomment-2914728385
* fix: AiohttpResponseStream transport
* fix: use AiohttpResponseStream transport by default
* fix: AiohttpResponseStream transport
* fixes: mapping aiohttp exceptions
* fixes: aiohttp rollout
* fixes: add support ssl_verify for aiohttp
* fixes: add support ssl_verify for aiohttp
* fixes: remove duplicates
* fix(main.py): use processed non-default-params as standard input params for langfuse
Fixes https://github.com/BerriAI/litellm/issues/11072
Fixes https://github.com/BerriAI/litellm/issues/11096
* fix(main.py): rename variable to be more accurate
* test(test_langfuse_e2e_test.py): add router unit test for langfuse e2e testing
Prevent https://github.com/BerriAI/litellm/issues/11072 from happening again
* build: update lock
* fix(utils.py): refactor optional params function
make it easier to get the standardized non default params
* fix(utils.py): improve process non default params function
* fix(main.py): include provider specific params in processed non default params used in logging
ensures user can see any provider specific params on langfuse
ensures user can see any provider specific params on langfus e
* fix: add flag for disabling use_aiohttp_transport
* feat: add _create_async_transport
* feat: fixes for transport
* add httpx-aiohttp
* feat: fixes for transport
* refactor: fixes for transport
* build: fix deps
* fixes: test fixes
* fix: ensure aiohttp does not auto set content type
* test: test fixes
* feat: add LiteLLMAiohttpTransport
* fix: fixes for responses API handling
* test: fixes for responses API handling
* test: fixes for responses API handling
* feat: fixes for transport
* fix: base embedding handler
* test: test_async_http_handler_force_ipv4
* test: fix failing deepeval test
* fix: add YARL for bedrock urls
* fix: issues with transport
* fix: comment out linting issues
* test fix
* test: XAI is unstable
* test: fixes for using respx
* test: XAI fixes
* test: XAI fixes
* test: infinity testing fixes
* docs(config_settings.md): document param
* test: test_openai_image_edit_litellm_sdk
* test: remove deprecated test
* bump respx==0.22.0
* test: test_xai_message_name_filtering
* test: fix anthropic test after bumping httpx
* use n 4 for mapped tests (#11109)
* fix: use 1 session per event loop
* test: test_client_session_helper
* fix: linting error
* fix: resolving GET requests on httpx 0.28.1
* test fixes proxy unit tests
* fix: add ssl verify settings
* fix: proxy unit tests
* fix: refactor
* tests: basic unit tests for aiohttp transports
* tests: fixes xai
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* Add support for supports_computer_use in model info
* Corrected list of supports_computer_use models
* Further fix computer use compatible claude models, fix existing test that predated supports_computer_use in the model list
* Move computer use test case into existing test_utils file
* Moved tests in to test_utils.py
* fix(openai/gpt_transformation.py): handle missing filename for openai file data call
* fix(openai/gpt_transformation.py): clean handling for sync + async pdf url transformation flows
Fixes https://github.com/BerriAI/litellm/issues/10820
* build(model_prices_and_context_window.json): add 'supports_pdf_input' for all openai models which have 'vision' support
Follows openai guidelines
* feat(bedrock/chat): support cache pointing tool calls on Bedrock
Closes https://github.com/BerriAI/litellm/pull/10613
* fix: fix linting error
* Enable update/delete org members on UI (#8560)
* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys
Ensures any org deletion is handled correctly
* test(test_organizations.py): add simple test to ensure org deletion works
* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update
* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE
* feat(organization_endpoints.py): support `/organization/member_update` endpoint
Allow admin to update member's role within org
* feat(organization_endpoints.py): support deleting member from org
* test(test_organizations.py): add e2e test to ensure org member flow works
* fix(organization_endpoints.py): fix code qa check
* fix(schema.prisma): don't introduce ondelete:cascade - breaking change
* docs(organization_endpoints.py): document missing params
* support anonymize and deanonymize
* use new response schema
* don't use detected because action already means there are detections
* log to debug
* CR fixes
* lint
* add tests
* use single quotes in deanonymiztion
* remove engage action case
* set max entities to 100 to prevent memory leak
* add test case for de-anonymization of llm response
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* ui fix bedrock guard
* polish: logo should appear after selecting provider
* fix ui config bedrock
* fix: refactor - use specific configs per provider
* fix: refactor - use specific configs per provider
* feat: ui, show provider specific params for guardrails
* fix: updated type of LiteLLM params for guardrails
* fix: updated type of LiteLLM params for guardrails
* ui, use endpoint for adding presidio, bedrock guardrails
* fix: linting error
* add llama guard and secret detector on UI
* add aim on ui
* allow adding lakera AI on litellm ui
* fix: fixes for params to init guardrails
* test: test_guardrail_info_response
* test: test_initialize_presidio_guardrail
* fix: init guardrails
* fix: init guardrails
* add showSearch
* working bedrock guard
* Add new model provider Novita AI (#7582)
* feat: add new model provider Novita AI
* feat: use deepseek r1 model for examples in Novita AI docs
* fix: fix tests
* fix: fix tests for novita
* fix: fix novita transformation
* ci: fix ci yaml
* fix: fix novita transformation and test (#10056)
---------
Co-authored-by: Jason <ggbbddjm@gmail.com>
* Support pdf url's to openai (#10640)
* fix(gpt_transformation.py): support pdf url input to openai
pass as base64 as openai doesn't support image url's
* fix(openai.py): support async message transformation
allows async get request to convert url to base64
* fix(gpt_transformation.py): fix linting errrors and use common components across sync + async flows
* fix: fix linting errors
* fix(openai.py): pop correct var
* Fix sagemaker chat calls - content length error (#10607)
* fix(sagemaker_chat/): support passing dynamic aws params
previously being ignored
* refactor(sagemaker/chat): more refactoring
* fix(sagemaker_chat/): make sure streaming is correctly handled post-refactor
* refactor: more refactoring to support using signed json str
* fix(sagemaker/chat): working sync streaming post refactor
* fix(sagemaker/chat): support async streaming post refactor
* fix(llm_http_handler.py): await async function
* fix: remove print statements
* test: update test
* test: update test
* fix(llm_http_handler.py): retain passing in data as json str
* test: update test
* fix(base_model_iterator.py): fix linting error
* test: test auth
* fix: fix linting error
* test: update test
* test: update translation test
* fix(gpt_transformation.py): handle awaitable/non-awaitable object
* fix: handle async flow for message transformation on openai compatible api's
* test: cleanup testing
* test: update test
* test(test_router.py): use model with higher quota
* test: simplify test
* test: update test
* fix(auth_checks.py): enforce auth checks on target model names
ensures user has access to models they are trying to call
* test(test_auth_utils.py): add unit tests for auth check
* fix(exception_mapping_utils.py): handle mistral 429 exception
* fix: fix linting error
* fix(auth_checks.py): add max fallback depth
* fix(caching_handler.py): fix embedding str caching result
Fixes issue where str caching results were not being correctly assembled on str input
* feat(azure/image_generation): Support dropping response_format for azure gpt-image-1
Fixes LIT-118
* test(test_utils.py): add unit testing
* test: rename file to avoid testing conflict
* feat(team_info.tsx): allow user to reassign team to another org
* style(team_info.tsx): fix org id styling
* feat(team_endpoints.py): add validation check before migrating team to another org
ensure model access, budgets and membership is respected
* fix(team_endpoints.py): update model migration validation to check if org has 'all-proxy-models' access
* fix(organization_view.tsx): show teams belonging to org
* feat(team_endpoints.py): handle wildcard model check on org migration
* fix(team_endpoints.py): nest router check
* test: update testing - use model with higher quota
* build: update poetry lock
* added tests
messages_with_counts: Made tolerance explicit for each test. But they match the new implementation(which beats the old)
* new token counter impl
* compare old and new implementation in test
* delete old token counter
* moved tests to /tests/litellm/litellm_core_utils
* use existing types
* docstrings
* warn about using default params on unknown model.
* created type for the token_counter_function
* check key == "content"
* throw error on invalid detail-type, ignore type-warning.
* fix imports
* Update docs for OpenAI compatible providers, add Llamafile docs, include Llamafile in the sidebar
* Add Llamafile as an LlmProviders enum
* Add llamafile as a OpenAI compatible provider (in the list of compatible providers)
* Add Llamafile chat config and tests
* Wire up Llamafile
Co-authored-by: Peter Wilson <peter@mozilla.ai>
* fix(exception_mapping_utils.py): correctly pass through 504 status code
openai also raises a 504 status code
* build(model_prices_and_context_window.json): add gpt-4o-mini-tts to model cost map
Fixes https://github.com/BerriAI/litellm/issues/9591
* fix(cost_calculator.py): fix input cost calculation for gpt-4o-mini-tts
Fixes https://github.com/BerriAI/litellm/issues/9591
* test: testing updates
* fix: initial commit of v2 parallel request limiter hook
enables multi-instance rate limiting to work
* fix: subsequent commit with additional refactors
* fix(parallel_request_limiter_v2.py): cleanup initial call hook
simplify it
* fix(parallel_request_limiter_v2.py): working v2 parallel request limiter
* fix: more updates - still not passing testing
* fix(test_parallel_request_limiter_v2.py): update test + add conftest
* fix: fix ruff checks
* fix(parallel_request_limiter_v2.py): use pull via pattern method to load in keys instance wouldn't have seen yet
Fixes issue where redis syncing was not pulling key until instance had seen it
* test: update testing to cover tpm and rpm
* fix(parallel_request_limiter_v2.py): fix ruff errors
* fix(proxy/hooks/__init__.py): feature flag export
* fix(proxy/hooks/__init_.py): fix linting error
* ci(config.yml): add tests/enterprise to ci/cd
* fix: fix ruff check
* test: update testing
* build(model_prices_and_context_window.json): add fireworks ai new 0-4b pricing tier
* build(model_prices_and_context_window.json): add more fireworks ai models
* test: update testing
* fix(caching_handler.py): handle str + list cache
Fixes issue on cache hits for embedding when initial cached input was str
* test(test_caching.py): add e2e test on caching with individual item and then list
* fix(caching_handler.py): set usage tokens for cache hits
enables token counting to work
* fix(caching_handler.py): combine usage between cached result and embedding response
Handles case of new input to embedding response
* fix: cleanup
* test: move to gpt-4o-new-test
* test: update test
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly
Fixes https://github.com/BerriAI/litellm/issues/7209
* fix(utils.py): handle none type in message
* fix: fix model name in test
* fix(utils.py): fix validate check for openai messages
* fix: fix model returned
* fix(main.py): fix text completion routing
* test: update testing
* test: skip test - cohere having RBAC issues
Krish Dholakia