Commit Graph
9 Commits
Author SHA1 Message Date
David ChenandGitHub d1df4e838b Litellm fix update bedrock models (#24947)
* update bedrock models in tests

* updated more tests and model_prices_and_context_window

* fix model id and pricing

* replace more sonnet models

* update tests

* git push

* update pricing

* flaky total cost

* monkey patch

* relax the cost change

* fix and revert some changes

* revert the pricing

* chore: move cost/pricing changes to bedrock-cost-fixes branch

* chore: split Bedrock file-api beta stripping to separate branch

Removes strip_unsupported_file_api_betas_for_bedrock_invoke from this branch;
see litellm_bedrock_invoke_strip_file_api_betas for that fix.

Made-with: Cursor
2026-04-01 19:22:54 -07:00
Ishaan JaffandGitHub a42132f329 fix(passthrough): propagate Azure 429/5xx errors in async streaming instead of silent HTTP 200 (#22913)
* fix(passthrough): raise_for_status in _async_streaming to propagate Azure 429s

* address greptile review feedback (greploop iteration 1)

Guard data/json args when content is provided to avoid httpx ValueError

* address greptile review feedback (greploop iteration 2)

Use bare raise to preserve original traceback in _async_streaming exception handler

* address greptile review feedback (greploop iteration 3)

Close httpx streaming response on error to prevent connection pool exhaustion

* address greptile review feedback (greploop iteration 4)

Guard aclose() call to prevent masking original exception; add explicit test for content param forwarding

* address greptile review feedback (greploop iteration 5)

Pass content to sign_request so AWS body-hash signing is correct when content is the sole body source

* revert sign_request content change - request_data expects dict, not bytes

Bedrock's sign_request calls json.dumps(request_data) — passing content bytes
would TypeError. sign_request should only receive data/json (dict), not raw bytes.
2026-03-05 10:12:43 -08:00
Ishaan Jaffer 159db27d5c fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
Krrish Dholakia d7aab1d5de test: add unit tests 2025-10-06 13:53:36 -07:00
Ishaan Jaffer f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
Ishaan JaffandGitHub c7b0c57b1e [Bug Fix] Azure Passthrough request with streaming (#13831)
* fix: _update_stream_param_based_on_request_body

* test_update_stream_param_based_on_request_body

* test_pass_through_request_stream_param_override
2025-08-25 14:11:17 -07:00
Fang Gong caad9b3ca7 fix application inference profile for pass-through endpoints for bedrock 2025-08-20 00:40:43 -07:00
Krish DholakiaandGitHub dd92b1a0ac Refactor: bedrock passthrough fixes - migrate to Passthrough SDK (#12089)
* feat: initial commit adding bedrock support via the new sdk passthrough logic

ensures correct sequencing of tasks (pre call checks etc. can run before signing request)

* fix(route_llm_requests.py): passthrough to allm_passthrough_route if no model found

* feat(bedrock/passthrough): working bedrock passthrough via sdk support

* fix(passthrough/main.py): re-add data and json

* feat(passthrough/main): support async passthrough calls to bedrock

* feat(passthrough/main.py): async streaming + completion support

* feat(llm_passthrough_endpoints.py): migrate bedrock passthrough calls to to new bedrock passthrough sdk

Enables calls to work correctly

* fix: fix linting errors

* test: update test
2025-06-26 22:51:35 -07:00
Krrish Dholakia 51f716c762 build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205 2025-05-31 09:00:04 -07:00