Commit Graph
36855 Commits
Author SHA1 Message Date
Ishaan Jaffer dd87f3be5b docs(advisor): move supported providers to top, focus how it works on litellm native loop 2026-04-11 18:27:18 -07:00
Ishaan Jaffer a8bc7bfcd4 docs(advisor): add how it works section with mermaid diagram + non-native provider table 2026-04-11 18:23:33 -07:00
Ishaan Jaffer 9be7b4c07c test(advisor): add integration tests for full dispatch path, max_uses, provider bypass 2026-04-11 18:16:56 -07:00
Ishaan Jaffer fa5258466d test(advisor): add unit tests for max_uses=0, missing model, default fallback 2026-04-11 18:16:56 -07:00
Ishaan Jaffer 22f45c6666 fix(advisor): restore AdvisorMaxIterationsError, raise on cap, fix max_uses=0 falsy 2026-04-11 18:16:56 -07:00
Ishaan Jaffer d29f40d39f fix(advisor): inject max_uses_exceeded error result instead of raising exception
When the advisor loop hits max_uses, inject a tool_result error so the executor
sees the cap and continues without further advice — matches Anthropic server-side
behaviour (error_code: max_uses_exceeded).
2026-04-11 18:03:12 -07:00
Ishaan Jaffer 844e34b68b test(advisor): remove live e2e test file (tests run locally via script) 2026-04-11 17:52:06 -07:00
Ishaan Jaffer 742e2fe1aa test(advisor): add live e2e tests for advisor orchestration against real proxy 2026-04-11 17:46:17 -07:00
Ishaan Jaffer ce3d039bcd test(advisor): add unit tests for orchestration loop (mocked backends, 8 tests) 2026-04-11 17:43:26 -07:00
Ishaan Jaffer b92e4c5595 docs(advisor): add interceptors README explaining when to use vs pre-request hooks 2026-04-11 17:43:23 -07:00
Ishaan Jaffer c12ebdb221 feat(advisor): add AdvisorOrchestrationHandler for non-Anthropic providers 2026-04-11 17:43:20 -07:00
Ishaan Jaffer ea765f7509 feat(advisor): add MessagesInterceptor ABC and registry 2026-04-11 17:43:16 -07:00
Ishaan Jaffer ebc57a157d feat(advisor): wire MessagesInterceptor registry into anthropic_messages handler 2026-04-11 17:43:13 -07:00
Ishaan Jaffer a89b0672c7 feat(advisor): add replace_with_text param to strip_advisor_blocks_from_messages 2026-04-11 17:43:10 -07:00
Ishaan Jaffer 23e20facb6 feat(advisor): add ADVISOR_MAX_USES, ADVISOR_NATIVE_PROVIDERS, ADVISOR_TOOL_DESCRIPTION constants 2026-04-11 17:43:07 -07:00
yuneng-jiangandGitHub 2c786ca2e6 Merge pull request #25578 from BerriAI/yj_apr11_bump
bump: version 1.83.6 → 1.83.7
v1.83.7.rc.1
2026-04-11 17:09:31 -07:00
Yuneng Jiang e162c6d502 bump: version 1.83.6 → 1.83.7 2026-04-11 17:00:33 -07:00
yuneng-jiangandGitHub d7823e3a09 Merge pull request #25577 from BerriAI/yj_apr11_build_3
[Infra] Rebuild UI
2026-04-11 16:04:12 -07:00
Yuneng Jiang a89ad3aaff chore: update Next.js build artifacts (2026-04-11 22:57 UTC, node v22.16.0) 2026-04-11 15:57:01 -07:00
yuneng-jiangandGitHub 99cb3af1a6 Merge pull request #25562 from BerriAI/litellm_internal_staging_04_11_2026
Litellm internal staging 04 11 2026
2026-04-11 15:55:29 -07:00
Yuneng Jiang c40e459447 fix linting 2026-04-11 15:44:15 -07:00
Yuneng Jiang 909247785e Merge remote-tracking branch 'origin' into litellm_internal_staging_04_11_2026 2026-04-11 15:41:03 -07:00
yuneng-jiangandGitHub 9a43e32d6e Merge pull request #25464 from BerriAI/litellm_passthrough_contenttype
fix(proxy): pass-through multipart uploads and Bedrock JSON body
2026-04-11 15:40:16 -07:00
shivam 3742b0cee1 refactor: define pass-through custom body state key in types module
Avoid module-level cyclic import between llm_passthrough_endpoints and
pass_through_endpoints; CodeQL and partial init order no longer risk
undefined LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY.

Made-with: Cursor
2026-04-11 15:26:44 -07:00
shivam 574eb207c8 fix: import LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY for bedrock passthrough
Fixes NameError when bedrock_proxy_route sets custom body on request.state.
Remove unused lazy-loader helper.

Made-with: Cursor
2026-04-11 15:08:10 -07:00
Shivam RawatGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
627d70affd Potential fix for pull request finding 'CodeQL / Module-level cyclic import'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-11 14:46:02 -07:00
yuneng-jiangandGitHub 21e4071f25 Merge pull request #25573 from BerriAI/yj_apr11_build_2
[Infra] Rebuild UI
2026-04-11 14:25:44 -07:00
shivam 20b16b377c chore(proxy): match CRLF line endings in pass_through_endpoints
Restores Windows-style line endings to match main/origin main for this
file, removing the full-file noise diff from an accidental LF-only
normalization.

Made-with: Cursor
2026-04-11 14:24:26 -07:00
Yuneng Jiang b42a8750df chore: update Next.js build artifacts (2026-04-11 21:00 UTC, node v22.16.0) 2026-04-11 14:00:21 -07:00
yuneng-jiangandGitHub 56e82451df Merge pull request #25458 from BerriAI/litellm_team_members_logs
Team member permission /spend/logs for team-wide spend logs (UI + RBAC)
2026-04-11 13:55:00 -07:00
yuneng-jiangandGitHub dbcdb1565c Merge pull request #25241 from BerriAI/es_allow_applyguardrails_for_iam
added applyguardrail to inline iam
2026-04-11 13:51:45 -07:00
yuneng-jiangandGitHub 0169f140bf Merge pull request #25571 from BerriAI/yj_apr11_build
[Infra] Build UI for release
2026-04-11 13:31:23 -07:00
ishaan-berriandGitHub da75012a50 Merge pull request #25569 from BerriAI/litellm_harish_april11
Litellm harish april11
2026-04-11 13:26:18 -07:00
Yuneng Jiang 18caee3ef4 chore: update Next.js build artifacts (2026-04-11 20:19 UTC, node v22.16.0) 2026-04-11 13:19:41 -07:00
harish-berriandGitHub ec0cd5c17d Update streaming.py. Provide Type Annotation for empty dict 2026-04-11 13:04:25 -07:00
yuneng-jiangandGitHub 150c37c47d Merge pull request #25568 from BerriAI/litellm_yj_apr_10_2026
[Infra] Merge dev with main
2026-04-11 13:03:52 -07:00
Yuneng Jiang 218daca867 [Fix] Address Greptile review: POST /organization/info auth bypass, inline imports, team access denial tests
- Add _verify_org_access to deprecated POST /organization/info endpoint
- Move get_user_object to module-level import in organization_endpoints.py
- Add tests for _verify_team_access 403 denial path
2026-04-11 12:40:55 -07:00
ishaan-berriandGitHub c70a3c7093 Merge pull request #25450 from harish876/oom-file-fix-openai
Add file content streaming support for OpenAI and related utilities
2026-04-11 12:25:36 -07:00
ishaan-berriandGitHub 5cb9b087ad Merge pull request #25545 from BerriAI/litellm_ishaan_april10
Litellm ishaan april10
2026-04-11 12:14:08 -07:00
harish876 69eb34597c Refactor file content streaming handling to improve routing and support
- Introduced a new method in `FileContentStreamingHandler` to resolve streaming request parameters, enhancing the routing logic based on credentials.
- Updated the `should_stream_file_content` method to check against supported providers.
- Cleaned up type hints and imports across multiple files for better organization and clarity.
- Added comprehensive tests to validate the new routing behavior and ensure original data integrity during streaming requests.
2026-04-11 18:56:15 +00:00
yuneng-jiangandGitHub 57b77fecbd Merge pull request #25554 from BerriAI/litellm_cross_tenant_resource_checks
[Fix] Align Org and Team Endpoint Permission Checks
2026-04-11 11:52:04 -07:00
Ishaan Jaffer 011e087939 fix(anthropic): guard against non-dict messages in strip_advisor_blocks_from_messages 2026-04-11 11:33:29 -07:00
Ishaan Jaffer 082f0afb83 refactor(anthropic): extract output_config validation to helper to fix ruff PLR0915 2026-04-11 11:09:59 -07:00
Ishaan Jaffer f40995418b fix(types): annotate ANTHROPIC_ADVISOR_TOOL_TYPE as Literal to satisfy mypy 2026-04-11 11:09:55 -07:00
ryan-crabbe-berriandGitHub eabb6a31f1 Merge pull request #25542 from BerriAI/litellm_fix-timestamp
fix(spend): session-TZ-independent date filtering for spend/error log queries
2026-04-11 10:57:25 -07:00
01b9b50b43 Add Screenshots / Proof of Fix section to PR template (#25564)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
2026-04-11 10:20:34 -07:00
Ryan Crabbe 423677f19e fix(spend): convert string dates to tz-aware UTC datetimes in _get_spend_report_for_time_range
This helper takes start_date/end_date as plain strings and passed them
straight to query_raw. Prisma forwards untyped text to Postgres, which
parses `'2026-04-10'::timestamptz` using the session timezone because
there's no +00:00 offset to respect. Under a non-UTC session this shifts
the resolved instant by the session offset, and the AT TIME ZONE 'UTC'
wrap introduced in the previous commit then strips it to a plain
timestamp that's still offset by the same amount — producing the exact
4h drift that wrap was meant to prevent.

Normalize the strings to datetime(..., tzinfo=timezone.utc) at the top of
the function so Prisma serializes them with the explicit +00:00 suffix,
which makes the ::timestamptz cast session-TZ-independent. Also replaces
the $1::date comparison in the team_alias query with the same
::timestamptz AT TIME ZONE 'UTC' pattern used by every other site in
this PR, so both queries share one consistent shape.

Verified live on a real Postgres under session TZ America/New_York:
the function now returns exactly the April 10 UTC demo rows
($4.20 / 10 rows) whereas the previous shape returned 13 rows
(2 correct April 10 rows dropped, 5 rows from April 11 early-morning
wrongly shifted in).
2026-04-11 09:47:09 -07:00
michelligabrieleandGitHub 363f9fe5da fix(proxy): preserve dict guardrail HTTPException.detail + bedrock context (#25558) 2026-04-11 09:40:39 -07:00
jimmychen-p72andGitHub 2fe615b373 fix(s3): add retry with exponential backoff for transient S3 503/500 errors (#25530)
* fix(s3): add retry with exponential backoff for transient S3 503/500 errors

S3 occasionally returns 503 "Slow Down" during PUT operations when
request rates spike above partition limits. The current code makes a
single upload attempt via httpx — unlike boto3, httpx has no built-in
retry for transient S3 errors. Failed uploads permanently lose the
request's audit/logging data.

Add exponential backoff retry (3 attempts, 1s/2s delays) for S3
500/503 responses in both async_upload_data_to_s3 and
upload_data_to_s3. Logs a warning on each retry with the S3 object
key for observability.

In production we observed ~18 permanent S3 upload failures per day
(124 over 7 days) — all transient 503s that would have succeeded on
a single retry.

* test(s3): add unit tests for S3 upload retry logic

Tests cover:
- Async retry on 503 (succeeds on second attempt)
- Async retry on 500
- Exhausted retries on persistent 503 (calls handle_callback_failure)
- No retry on 4xx errors (403)
- Sync retry on 503

* style(s3): move time import to module level

Address review feedback: move `import time` from inside
upload_data_to_s3 to the top-level imports per project style guide.
2026-04-11 09:39:12 -07:00
JoshandGitHub 7d2f069361 Reduce default latency histogram bucket cardinality (#25527)
* feat(prometheus): reduce default latency bucket cardinality and make configurable

* test(prometheus): add coverage for PrometheusServicesLogger latency buckets

* Revert "test(prometheus): add coverage for PrometheusServicesLogger latency buckets"

This reverts commit 1bfd004ad1797e212dfd9d1de502810f81a056a1.

* test(prometheus): add coverage for PrometheusServicesLogger latency buckets
2026-04-11 09:34:45 -07:00