0x5751
d6cd50dfdb
feat: Add support for custom Anthropic-compatible API endpoints
...
This commit adds support for custom Anthropic-compatible API endpoints
that don't follow the standard /v1/messages or /v1/complete path convention.
## Changes
- Added LITELLM_ANTHROPIC_DISABLE_URL_SUFFIX environment variable
- When set to true, prevents automatic appending of /v1/messages (for anthropic)
- When set to true, prevents automatic appending of /v1/complete (for anthropic_text)
- Added debug logging to indicate when suffix is being skipped
- Maintained full backward compatibility - existing deployments are unaffected
2025-08-26 01:22:51 +08:00
Steven Anton
d95578464e
Add support for AWS assume_role with a session token
2025-08-23 22:37:21 -07:00
Ishaan Jaff
cf64033044
test fix
2025-08-23 14:04:41 -07:00
Ishaan Jaff
f1c710a608
test_e2e_generate_cold_storage_object_key_not_configured
2025-08-23 13:33:33 -07:00
Ishaan Jaff and GitHub
b9132968b2
[Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS ( #13905 )
...
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895 )
* fix: litellm.configured_cold_storage_logger
* fix Session Management - Non-OpenAI Models docs
* ruff fix
* test fix
* create LoggingWorker
* add GLOBAL_LOGGING_WORKER for async task handling
* fix logging tests
* add conftest
* fix conftest
* test fix location of encode bedrock runtime modelid arn
* fix conftest.py
* tuning LoggingWorker
* conftest.py
* fix conftest batches/
* test_async_chat_azure
* event_loop
* test_bedrock_streaming_passthrough_test2
* fix GLOBAL_LOGGING_WORKER
* logging worker
* add flush for global logging worker
* Revert "fix GLOBAL_LOGGING_WORKER"
This reverts commit d254f508f48935652f054777652938ad71976cce.
* fix conftest clear_queue
* fix conftest clear_queue
* setup_and_teardown for llm translation
* docs AWS_REGION
* test_async_chat_azure
* change test DIR
* run ci/cd again
* use 1 job for litellm_router_unit_testing
* fix space
* fix litellm_router_unit_testing
* test_aaarouter_dynamic_cooldown_message_retry_time
* litellm_router_unit_testing
* conftest.py clearing qu
* fixes litellm_router_unit_testing
* fixes clear_queue
* fix router_unit_tests
* remove conftest
* add back conftest for router
* fix event loop test
* test fix
* fixes for LoggingWorker
* ruff fix
2025-08-23 13:13:23 -07:00
Krish Dholakia and GitHub
9df6f4ef08
Merge branch 'main' into litellm_dev_08_20_2025_p1
2025-08-23 12:09:53 -07:00
Krrish Dholakia
b23e35f75e
test: update tests
2025-08-23 12:09:19 -07:00
Krish Dholakia and GitHub
f0b0ee51e0
Merge branch 'main' into litellm_dev_08_15_2025_p1
2025-08-23 10:51:56 -07:00
Krish Dholakia and GitHub
867b48a24f
Merge pull request #13866 from jinskjoy/Mistral-File-ID
...
[Enhancement] Add support for Mistral model file handling and update documentation
2025-08-22 23:54:17 -07:00
Ishaan Jaff
825d6b08f0
test fix location of encode bedrock runtime modelid arn
2025-08-22 17:34:09 -07:00
Ishaan Jaff and GitHub
56c16fc6f6
[Performance] Reduce Significant CPU overhead from litellm_logging.py ( #13895 )
...
* fix: litellm.configured_cold_storage_logger
* fix Session Management - Non-OpenAI Models docs
* ruff fix
* test fix
2025-08-22 16:17:57 -07:00
Jugal D. Bhatt and GitHub
3c1dcb64cc
fix by adding init lock ( #13666 )
2025-08-22 11:42:43 -07:00
Krish Dholakia and GitHub
dcad39c00f
Merge pull request #13753 from RichardoC/fix-bedrock-irsa-role-assumption
...
fix: role chaining and session name with webauthentication for aws bedrock
2025-08-22 07:20:27 -07:00
Krish Dholakia and GitHub
59e43595b9
Merge pull request #13852 from michal-otmianowski/async-s3-implementation
...
Async s3 implementation
2025-08-21 23:19:51 -07:00
Krish Dholakia and GitHub
8b45e53878
Merge pull request #13880 from datarobot-forks/carsongee/improve_datarobot_api_handling
...
🐛 Bug Fix: Updated URL handling for DataRobot provider URL
2025-08-21 23:18:40 -07:00
Krish Dholakia and GitHub
6214d22126
Merge pull request #13863 from luizrennocosta/fix/pll-req-lim-v3-api-key
...
fix: fixing descriptor/response size mismatch on parallel_request_limiter_v3
2025-08-21 23:14:04 -07:00
Krish Dholakia and GitHub
a61275f6e2
Merge pull request #13796 from 0x-fang/fix_aip_pass_through_08_20
...
Fix(Bedrock): fix application inference profile for pass-through endpoints for bedrock
2025-08-21 19:41:23 -07:00
Carson Gee
b46de0ab5c
Updated URL handling for DataRobot provider base
2025-08-21 19:42:46 -06:00
Krish Dholakia and GitHub
7c2c1c11d9
Merge pull request #13783 from philipkiely-baseten/main
...
Update Baseten LiteLLM integration
2025-08-21 18:34:36 -07:00
Ishaan Jaff and GitHub
77a5de37b1
[Fix] Ensure Service Account Keys require team_id field on API Endpoints ( #13873 )
...
* fixes for generating service accounts through the API
* fixes for test_update_service_account_works_with_team_id
2025-08-21 14:55:06 -07:00
jinskjoy
d8a33e137d
[Enhancement] Add support for Mistral model file handling and update documentation
2025-08-21 20:35:07 +02:00
Philip Kiely
5fa4a18073
add testing and remove redundant function
2025-08-21 11:25:26 -07:00
Luiz Rennó Costa
f8711200d0
fix: fixing descriptor/response size difference when indexing descriptor
2025-08-21 13:43:23 -03:00
Michal Otmianowski
a6f052882b
fix test
2025-08-21 12:10:29 +02:00
Michal Otmianowski
feae09da5d
minor adjustments
2025-08-21 11:33:11 +02:00
Michal Otmianowski
5dd7e00127
formatting adjustments
2025-08-21 11:27:12 +02:00
Michal Otmianowski
08c942c306
refactor tests
2025-08-21 10:38:43 +02:00
Michal Otmianowski
744dca1dc3
init asyc implementation
2025-08-21 10:22:54 +02:00
Krish Dholakia and GitHub
0e6e8af1c3
Merge pull request #13795 from TomeHirata/feat/predict/mlflow
...
Include predicted output in MLflow tracing
2025-08-20 22:40:33 -07:00
Krish Dholakia and GitHub
1f84451925
Merge pull request #13803 from UlookEE/fix_bedrock_epc_acompletion
...
fix : Synchronize cache behavior between acompletion and completion
2025-08-20 22:38:30 -07:00
Krish Dholakia and GitHub
ff267ab68e
Merge pull request #13820 from kankute-sameer/litellm_rerank_deepinfra_endpoint
...
Add rerank endpoint support for deepinfra
2025-08-20 22:33:25 -07:00
Fang Gong
91040b6b56
fix the api key support for bedrock guardrail in proxy
2025-08-20 18:46:00 -07:00
Krrish Dholakia
34ad646101
fix(auth_checks.py): handle both flows + add unit tests
2025-08-20 16:42:08 -07:00
Krrish Dholakia
e5c3e006c2
fix(proxy_server.py): fix merge config + db router settings logic
2025-08-20 16:33:31 -07:00
Sameer Kankute
5b5a2d12a8
remove test
2025-08-21 00:32:16 +05:30
Sameer Kankute
1d1d637ac0
fix automated checks
2025-08-21 00:09:17 +05:30
Sameer Kankute
37b84d1582
Add rerank endpoint support for deepinfra
2025-08-20 23:36:05 +05:30
hgyun.lee
036b358636
Synchronize cache behavior between acompletion and completion
2025-08-20 18:27:36 +09:00
Fang Gong
caad9b3ca7
fix application inference profile for pass-through endpoints for bedrock
2025-08-20 00:40:43 -07:00
Tomu Hirata
5879f6e930
comment
2025-08-20 15:46:53 +09:00
Tomu Hirata
d64b579131
Include predicted output in tracing
...
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com >
2025-08-20 15:39:32 +09:00
Krish Dholakia and GitHub
be30bc68ae
Merge pull request #13759 from kankute-sameer/litellm_feat_correct_cost_calculations
...
Add long context support for claude-4-sonnet
2025-08-19 22:30:25 -07:00
Krrish Dholakia
7d09375d52
fix: fix gpt-5-chat mappings
2025-08-19 22:21:00 -07:00
Ishaan Jaff and GitHub
56ac778316
[Bug Fix] Bedrock KB - Using LiteLLM Managed Credentials for Query ( #13787 )
...
* fix: add get_credentials_for_vector_store
* test_search_uses_registry_credentials
* test_bedrock_search_with_credentials_managed_registry
2025-08-19 15:39:36 -07:00
Sameer Kankute
48622a4ee7
add cache above 200k keys in INTENDED_SCHEMA
2025-08-20 01:47:34 +05:30
Sameer Kankute and GitHub
fa81b6c639
Update test_cost_calculator.py
2025-08-19 23:14:22 +05:30
Ishaan Jaff and GitHub
195ea6515e
[Feat] Datadog LLM Observability - Add support for tracing guardrail input/output ( #13767 )
...
* add guardrail information on DD LLM Obs
* test_guardrail_information_in_metadata
2025-08-19 10:26:25 -07:00
Sameer Kankute and GitHub
b5f0a7b49b
Merge branch 'main' into litellm_feat_correct_cost_calculations
2025-08-19 18:55:36 +05:30
Sameer Kankute
d39b2e8888
Add test for long context cost calculation
2025-08-19 16:56:13 +05:30
openhands
93651c9da7
Revert "Revert "fix: role chaining and session name with webauthentication for aws be…" ( #13230 )"
...
This reverts commit 342fd2d8b6
2025-08-19 08:19:18 +00:00
0x5751