Commit Graph
404 Commits
Author SHA1 Message Date
yuneng-jiangandGitHub cf9b5e4fa7 [Infra] Bump versions (#28094)
* bump: version 0.1.40 → 0.1.41

* bump: version 1.85.0 → 1.86.0

* add uv lock
2026-05-16 18:31:43 -07:00
cbdc70d544 fix(managed_batches): convert raw output_file_id to managed ID in CheckBatchCost poller (#27984)
* fix(managed_batches): convert raw output_file_id to managed ID in CheckBatchCost poller

CheckBatchCost bypasses async_post_call_success_hook, causing raw provider
output_file_ids to be persisted in LiteLLM_ManagedObjectTable. This fix converts
output_file_id and error_file_id to managed base64 IDs before the DB write.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(check_batch_cost): persist managed file before mutating response and propagate team_id

- Move setattr after store_unified_file_id so the response only receives the
  managed ID once the DB record is successfully written. Avoids serializing
  an orphaned managed ID into file_object when the store call fails.
- Populate team_id on the minimal UserAPIKeyAuth from job.team_id so the
  managed file record is created with the correct team ownership, allowing
  other team members to access the batch output file via /files/{id}/content.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(managed_batches): extend test to cover error_file_id conversion

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix managed file test

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
2026-05-15 04:41:38 -07:00
yuneng-jiangandGitHub e84282b7b3 [Infra] Bump deps (#27157)
* bump: version 0.4.70 → 0.4.71

* bump: version 0.1.39 → 0.1.40

* uv lock
2026-05-05 15:58:05 -07:00
user 7faba9656f Merge remote-tracking branch 'upstream/litellm_internal_staging' into fix/managed-resource-service-account-isolation 2026-05-05 01:38:11 +00:00
user aee064ad37 Merge remote-tracking branch 'upstream/litellm_internal_staging' into fix/managed-resource-service-account-isolation 2026-05-05 01:29:05 +00:00
Yuneng Jiang e35cd5af76 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_may4 2026-05-04 18:22:47 -07:00
yuneng-jiangandGitHub 68c120a68f Merge pull request #26957 from stuxf/chore/guardrail-coverage
chore(guardrails): cover multimodal + Responses-API content shapes
2026-05-04 18:01:27 -07:00
user 83971a8712 fix(proxy): normalize managed resource team owner field 2026-05-04 17:05:50 -07:00
user abbefccad4 fix(guardrails): align banned_keywords + azure_content_safety call_type gates with runtime route_type
The hooks gated on ``call_type == "completion"`` but the proxy ingress
passes ``route_type`` straight through as ``call_type`` —
``"acompletion"`` for /v1/chat/completions and ``"aresponses"`` for
/v1/responses. Tests passed because they used the literal sync
``"completion"`` value, masking the gap.

Switch both hooks to ``is_text_content_call_type`` (matches the
canonical runtime values: completion / acompletion / aresponses) and
update existing tests to assert against runtime values, plus parametrize
a regression test that pins the gate.
2026-05-04 21:27:24 +00:00
user bfdd786962 chore(deps): refresh dependency locks 2026-05-04 11:36:18 -07:00
shin-berriandGitHub 38ddcdabdb Merge pull request #27032 from BerriAI/litellm_yj_may1_2
[Infra] Merge dev branch
2026-05-01 19:39:42 -07:00
user f909fa79b5 fix(guardrails): close post-call coverage gaps 2026-05-01 19:26:40 -07:00
yuneng-jiangandGitHub 5614469f22 Merge pull request #26825 from stuxf/fix/oauth2-proxy-header-forgery
chore(auth): require trusted proxy for header identity auth
2026-05-01 18:47:58 -07:00
yuneng-jiangandGitHub e78d87ee00 Merge pull request #27011 from stuxf/fix/project-update-cross-team-hijack
fix(proxy): close project hijacking and key org IDOR (VERIA-55)
2026-05-01 18:01:06 -07:00
userandClaude Opus 4.7 1b2756811e fix(proxy): close project hijacking and key org IDOR
Two related authorization gaps in management endpoints:

1. `/project/update` evaluated permission against the team_id supplied in
   the request body. By passing `data.team_id` pointing at a team they
   admin, a caller could hijack any project — `_check_user_permission_for_project`
   was given the attacker's team_object and happily checked admin
   membership against that. Drop the team_object kwarg so the helper
   re-fetches the existing project's team. Also require admin rights on
   the destination team when reassigning a project across teams, so a
   team admin cannot shed projects into another team's namespace.

2. `/key/update` accepted any `organization_id` and only checked that
   the org existed before applying limits. A caller could thereby point
   their key at an arbitrary org. Add `_validate_caller_can_assign_key_org`
   which enforces the same membership rule already applied on the
   `/key/list` filter path (`validate_key_list_check`); proxy admins and
   no-change updates skip the check.

Tests cover both helpers in isolation: existing-team-admin allow,
unrelated-team admin deny, proxy-admin shortcut, org-member allow,
non-member deny, missing user_id deny, no-memberships deny.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 21:32:38 +00:00
userandClaude Opus 4.7 84fede37b4 fix(proxy): isolate managed resources for service-account API keys
Service-account API keys are issued without a `user_id`, and managed
file/batch/vector-store ownership checks compared
`resource.created_by == user_api_key_dict.user_id`. Because Python
evaluates `None == None` as True, any service-account key passed
ownership checks for any resource also created without a user id, and
listing endpoints skipped the `created_by` filter entirely when the
caller had no user id — returning every tenant's records.

Replace the bare equality with an identity-aware helper:

- Admins (PROXY_ADMIN, PROXY_ADMIN_VIEW_ONLY) keep their unscoped view.
- Callers with a `user_id` are scoped to records they created.
- Callers without a `user_id` but with a `team_id` are scoped to records
  created within their team via a new `created_by_team_id` column.
- Callers with no admin role and no identifying ids are denied — the
  listing path returns an empty page without issuing a query.

Schema migration adds `created_by_team_id` to LiteLLM_ManagedFileTable,
LiteLLM_ManagedObjectTable, and LiteLLM_ManagedVectorStoreTable, plus
indexes for the new filter. Writes in BaseManagedResource and the
enterprise managed_files hook now stamp the column from
`user_api_key_dict.team_id`. Reads in `can_user_access_unified_resource_id`,
`can_user_call_unified_file_id`, `can_user_call_unified_object_id`,
`list_user_resources`, `list_user_batches`, and `get_user_created_file_ids`
all delegate to the new helper.

Tests cover the helper in isolation, the base-class listing/access paths,
and the enterprise file-access hook (including a regression test for the
original `None == None` bypass).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 20:22:37 +00:00
ishaan-berriandGitHub 32704ff7b2 fix(projects): project dropdown empty for internal_user (3 bugs) (#26664)
* fix(projects): fire useProjects hook for all authenticated users, not just admins

* fix(routes): add /project/list and /project/info to internal_user_routes allowlist

* fix(projects): use members_with_roles + LiteLLM_UserTable.teams for membership checks

* feat(ui): add "Your Usage" view for admin users on usage page

Admins were forced to use the global usage view with no way to scope it
to their own activity without manually searching for themselves in the
user filter dropdown.

Adds a new "Your Usage" option (admin-only) to the usage view selector.
When selected, it locks the data to the admin's own user_id and hides
the "Filter by user" dropdown.

* feat(ui): wire my-usage view to admin's own user_id in UsagePageView

When usageView is "my-usage", effectiveUserId resolves to the logged-in
admin's own userID. The "Filter by user" dropdown is hidden in this
view (only shown for "global").

* add: screenshots for usage page Your Usage admin fix

* fix(ui): gate useProjects on admin roles to fix failing unit test

* feat(proxy): add /project/list and /project/info to internal user routes

* fix(enterprise): use members_with_roles and litellm_usertable.teams for project access checks

* remove .github screenshots and workflow file from PR
2026-05-01 11:42:22 -07:00
userandClaude Opus 4.7 7514bb4740 fix(guardrails): close mixed-list gap, drop dead code, rename helper
Greptile P2 follow-ups on _content_utils.py:

- Drop unreachable ``_resolve_messages``. The new
  ``_iter_inspection_messages`` walks ``messages`` AND ``input``
  independently; leaving the old fallback-only variant around invited a
  future maintainer to wire it back up and silently narrow coverage.
- Rename ``iter_user_text`` → ``iter_message_text``. The helper walks
  every role (user, assistant, system); the old name implied user-turn
  content only. Callers and tests updated.
- Close mixed-list coverage gap. When ``data["input"]`` was a list
  mixing content-part dicts and bare strings, ``iter_message_text`` and
  ``build_inspection_messages`` only saw the dict parts while
  ``walk_user_text`` already inspected both. ``_iter_text_parts_in_content``
  now treats bare strings inside a content list as text fragments, so
  read and write helpers agree on coverage.

Adds two regression tests for the mixed-list shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 04:10:04 +00:00
userandClaude Opus 4.7 b1b00e4bdc chore(guardrails): cover multimodal + Responses-API content shapes
Several guardrail hooks short-circuit when ``message.content`` is a list
or when the request uses the Responses-API ``input`` field instead of
``messages``. Centralise the content-walking logic in a shared helper and
update the affected hooks so list-format and Responses-API payloads no
longer skip inspection.

Also: Aim's ``async_post_call_success_hook`` now inspects every choice
(via ``asyncio.gather``) instead of only ``choices[0]`` — the prior
behaviour let ``n>1`` callers hide content in subsequent completions.

Hooks updated to use the new helper:
- aim, lakera_ai_v2, lasso (post a synthesised messages list to a remote
  guardrail service)
- azure_content_safety, ibm_detector, banned_keywords, openai_moderation,
  google_text_moderation (iterate text fragments locally)
- secret_detection (walk-and-rewrite to redact in place)

Drive-by fix: the legacy ``data["prompt"]`` list-handling path in
secret_detection rebound the loop variable instead of mutating the list,
leaving secrets unredacted on text-completion calls; corrected to index
back into the list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 03:50:15 +00:00
user 2f4641752b chore(auth): require trusted proxy for header identity auth 2026-04-29 21:20:21 -07:00
Yuneng Jiang 92c1e9b63c bump: version 0.1.38 → 0.1.39 2026-04-25 19:31:49 -07:00
Yuneng Jiang 4884b0b611 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr23
# Conflicts:
#	litellm/proxy/management_endpoints/key_management_endpoints.py
2026-04-25 09:47:47 -07:00
yuneng-jiangandGitHub 7723a54478 Merge pull request #25677 from BerriAI/litellm_migration_projects
[Refactor] Proxy: move projects management to enterprise package
2026-04-24 17:40:33 -07:00
Yuneng Jiang ae9f12fb8a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr23 2026-04-24 16:13:52 -07:00
Yuneng Jiang 5d8ef97fa1 chore(packaging): declare proprietary license in litellm-enterprise metadata
The enterprise package ships under the BerriAI Enterprise License defined
in enterprise/LICENSE.md, which is not an SPDX-listed license. Declare it
via PEP 639's LicenseRef-Proprietary expression so metadata-reading tools
(PyPI classifiers, Nexus IQ, pip-licenses) resolve it instead of reporting
License-None. The existing license-files entry already ships the full terms.
2026-04-24 14:56:28 -07:00
Yuneng Jiang 000ce70127 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_migration_projects
# Conflicts:
#	litellm/proxy/ui_crud_endpoints/proxy_setting_endpoints.py
#	uv.lock
2026-04-24 12:52:10 -07:00
user 5ba6bc0784 chore(deps): bump uv to 0.11.7 + drop dead npm sed
- UV_IMAGE across all Dockerfiles: 0.10.9 -> 0.11.7.
- Loosen `required-version` in enterprise/ and litellm-proxy-extras/
  from strict `==0.10.9` to `>=0.10.9` so the new Docker image can
  build those workspace members. Matches the main pyproject range.
- Drop the `sed` block that rewrote tar/minimatch version ranges in
  npm's bundled package.json files. The override loop above already
  swaps the vendored directories on disk; npm doesn't re-resolve at
  runtime, so the sed was cosmetic.
2026-04-24 00:36:59 +00:00
ishaan-berriandGitHub 2f22a1293e bump litellm-proxy-extras to 0.4.67 (#26043)
* bump litellm-proxy-extras version to 0.4.67

* bump litellm-proxy-extras pin to 0.4.67 in litellm pyproject

* regenerate uv.lock for litellm-proxy-extras 0.4.67

* bump litellm-enterprise version to 0.1.38

* bump litellm-enterprise pin to 0.1.38 in litellm pyproject

* regenerate uv.lock for litellm-enterprise 0.1.38
2026-04-18 19:03:56 -07:00
Ryan Crabbe 0a4b02fe76 fix: tighten recipient_emails guard to reject empty list
send_max_budget_alert_email previously guarded with `is not None`, which
accepts `[]` and then crashes on `recipient_emails[0]` inside
_get_email_params. The current caller (_handle_multi_threshold_max_budget_alert)
already filters empty lists upstream, but the public method signature makes
no such guarantee — a future caller passing [] would hit IndexError.

Switch to truthiness so both None and [] fall through to the single-recipient
path.
2026-04-18 14:07:18 -07:00
Ryan Crabbe ec564f5e00 fix: resolve mypy errors in multi-threshold budget alerts
- Add early return guard in _handle_multi_threshold_max_budget_alert
  for None max_budget_alert_emails and max_budget
- Add explicit type annotation on alert_email_config in auth_checks
2026-04-17 21:51:41 -07:00
Ryan Crabbe c1503d3088 fix: escape user-controlled greeting in max budget alert email template
html.escape() the greeting (user_email/key_alias/token fallback) before
inserting into HTML email body to prevent HTML injection via key_alias.
2026-04-17 18:52:13 -07:00
Ryan Crabbe 44eb2ea56e fix: address Greptile review — empty recipients guard, type annotation, task pre-filter
- Guard empty recipients in _handle_multi_threshold_max_budget_alert:
  log warning and skip instead of falling through to old path error loop
- Widen max_budget_alert_emails type to Dict[str, Union[str, List[str]]]
  to match _parse_email_list runtime behavior (accepts comma-separated strings)
- Pre-filter asyncio.create_task with min threshold check to avoid
  unnecessary task allocation on every request when spend is below
  all configured thresholds
2026-04-17 17:54:16 -07:00
Ryan Crabbe 779a9fab8e feat: add configurable multi-threshold budget alerts for virtual keys
Users can set metadata.max_budget_alert_emails as a JSON map of threshold
percentages to email recipients on virtual keys. When configured, the email
handler loops over each threshold, checks per-threshold dedup cache, and
sends to the configured recipients (auto-including the key owner's email).

When no map is set, the existing single 80% threshold behavior is preserved
unchanged. Teams support is out of scope for this v0.
2026-04-17 17:12:35 -07:00
Yuneng Jiang 937d81331f [Refactor] Proxy: tighten UI settings extras registry
- drop unused Any annotation on register_extra_ui_setting's field
  param; type it as FieldInfo and have enterprise callers construct
  FieldInfo directly (pydantic.Field's stub reports the default's
  type, which doesn't match FieldInfo)
- cache the effective UISettings class and invalidate it inside
  register_extra_ui_setting so GET /get/ui_settings does not rebuild
  a pydantic model on every request
- annotate _EXTRA_UI_SETTINGS_FIELDS with a concrete
  Dict[str, Tuple[Any, FieldInfo]] instead of bare Dict[str, tuple];
  the annotation remains Any because pydantic field annotations
  include generics (Optional[X], List[X]) that are not instances of
  type
2026-04-13 21:58:02 -07:00
Yuneng Jiang d3a1f63af2 [Refactor] Proxy: move projects management to enterprise package
Remove the /project/* management endpoints and the enable_projects_ui
admin-settings flag from the OSS litellm package. Project endpoints now
live under litellm_enterprise and are wired through the existing
enterprise router; OSS builds return 404 for every /project/* route.

The enable_projects_ui UI flag is registered back onto UISettings via a
small extension registry when the enterprise package is imported, so the
admin toggle and downstream key/sidebar gating continue to work in
enterprise builds. On OSS, explicit PATCH attempts with the flag return
403 with a clear enterprise-only message instead of being silently
dropped.

Pydantic request/response types (NewProjectRequest, UpdateProjectRequest,
DeleteProjectRequest, NewProjectResponse) stay in litellm/proxy/_types.py
because management_endpoints/common_utils.py and pydantic-shape tests
import them. LiteLLM_ProjectTable and all FK columns in schema.prisma
are unchanged.
2026-04-13 21:41:12 -07:00
a6c30b30bf build: migrate packaging, CI, and Docker from Poetry to uv (#25007)
* build: migrate packaging metadata to uv

* ci: move automation and local tooling to uv

* docker: migrate image builds and runtime setup to uv

* docs: update install and deployment guidance for uv

* chore: align auxiliary scripts and tests with uv

* test: harden test_litellm isolation

* fix: keep release and health check images self-contained

* build: pin uv tooling and health check deps

* test: isolate bedrock image request formatting from suite state

* test: cover sandbox executor requirements flow

* ci: fix circleci no-op command steps

* ci: fix circleci publish workflow parsing

* fix: stabilize remaining uv migration CI checks

* ci: increase matrix test timeout headroom

* fix: restore published docker and license coverage

* fix: restore proxy runtime build parity

* fix: restore proxy extras parity and venv migrations

* ci: persist uv path across circleci steps

* fix: keep psycopg binary in default test env

* docker: preserve prisma cache across stages

* test: run local proxy checks through uv python

* build: restore runtime deps moved into ci

* build: refresh uv lock after upstream merge

* fix: restore module import in test_check_migration after merge

The conflict resolution imported only the function but the test body
references check_migration as a module throughout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching

- Move google-generativeai, Pillow, tenacity back to ci group (they are
  lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
  in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
  from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
  environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
  deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate uv.lock after removing nodejs-wheel-binaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): use cache/restore instead of cache to prevent cache poisoning

The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert

The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv cache in publish workflow

Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): remove duplicate verbose_logger mock in test_check_migration

The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): free disk space before Docker build in test-server-root-path

The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 11:46:23 -07:00
+2 f42ffed2bd Litellm oss staging 04 02 2026 p1 (#25055)
* fix(vertex_ai): support pluggable (executable) credential_source for WIF auth (#24700)

The WIF credential dispatch in load_auth() only handled identity_pool and
aws credential types. When credential_source.executable was present (used
for Azure Managed Identity via Workload Identity Federation), it fell
through to identity_pool.Credentials which rejected it with MalformedError.

Add dispatch to google.auth.pluggable.Credentials for executable-type
credential sources, following the same pattern as the existing identity_pool
and aws helpers.

Fixes authentication for Azure Container Apps → GCP Vertex AI via WIF
with executable credential sources.

* feat(logging): add component and logger fields to JSON logs for 3rd p… (#24447)

* feat(logging): add component and logger fields to JSON logs for 3rd party filtering

* Let user-supplied extra fields win over auto-generated component/logger, tighten test assertions

* Feat - Add organization into the metrics metadata for org_id & org_alias (#24440)

* Add org_id and org_alias label names to Prometheus metric definitions

* Add user_api_key_org_alias to StandardLoggingUserAPIKeyMetadata

* Populate user_api_key_org_alias in pre-call metadata

* Pass org_id and org_alias into per-request Prometheus metric labels

* Add test for org labels on per-request Prometheus metrics

* chore: resolve test mockdata

* Address review: populate org_alias from DB view, add feature flag, use .get() for org metadata

* Add org labels to failure path and verify flag behavior in test

* Fix test: build flag-off enum_values without org fields

* Gate org labels behind feature flag in get_labels() instead of static metric lists

* Scope org label injection to metrics that carry team context, remove orphaned budget label defs, add test teardown

* Use explicit metric allowlist for org label injection instead of team heuristic

* Fix duplicate org label guard, move _org_label_metrics to class constant

* Reset custom_prometheus_metadata_labels after duplicate label assertion

* fix: emit org labels by default, remove flag, fix missing org_alias in all metadata paths

* fix: emit org labels by default, no opt-in flag required

* fix: write org_alias to metadata unconditionally in proxy_server.py

* fix: 429s from batch creation being converted to 500 (#24703)

* add us gov models (#24660)

* add us gov models

* added max tokens

* Litellm dev 04 02 2026 p1 (#25052)

* fix: replace hardcoded url

* fix: Anthropic web search cost not tracked for Chat Completions

The ModelResponse branch in response_object_includes_web_search_call()
only checked url_citation annotations and prompt_tokens_details, missing
Anthropic's server_tool_use.web_search_requests field. This caused
_handle_web_search_cost() to never fire for Anthropic Claude models.

Also routes vertex_ai/claude-* models to the Anthropic cost calculator
instead of the Gemini one, since Claude on Vertex uses the same
server_tool_use billing structure as the direct Anthropic API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix(anthropic): pass logging_obj to client.post for litellm_overhead_time_ms (#24071)

When LITELLM_DETAILED_TIMING=true, litellm_overhead_time_ms was null for
Anthropic because the handler did not pass logging_obj to client.post(),
so track_llm_api_timing could not set llm_api_duration_ms. Pass
logging_obj=logging_obj at all four post() call sites (make_call,
make_sync_call, acompletion, completion). Add test to ensure make_call
passes logging_obj to client.post.

Made-with: Cursor

* sap - add additional parameters for grounding

- additional parameter for grounding added for the sap provider

* sap - fix models

* (sap) add filtering, masking, translation SAP GEN AI Hub modules

* (sap) add tests and docs for new SAP modules

* (sap) add support of multiple modules config

* (sap) code refactoring

* (sap) rename file

* test(): add safeguard tests

* (sap) update tests

* (sap) update docs, solve merge conflict in transformation.py

* (sap) linter fix

* (sap) Align embedding request transformation with current API

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) mock commit

* (sap) run black formater

* (sap) add literals to models, add negative tests, fix test for tool transformation

* (sap) fix formating

* (sap) fix models

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) commit for rerun bot review

* (sap) minor improve

* (sap) fix after bot review

* (sap) lint fix

* docs(sap): update documentation

* fix(sap): change creds priority

* fix(sap): change creds priority

* fix(sap): fix sap creds unit test

* fix(sap): linter fix

* fix(sap): linter fix

* linter fix

* (sap) update logic of fetching creds, add additional tests

* (sap) clean up code

* (sap) fix after review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) add a possibility to put the service key by both variants

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) update test

* (sap) update service key resolve function

* (sap) run black formater

* (sap) fix validate credentials, add negative tests for credential fetching

* (sap) fix validate credentials, add negative tests for credential fetching

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) lint fix

* (sap) lint fix

* feat: support service_tier in gemini

* chore: add a service_tier field mapping from openai to gemini

* fix: use x-gemini-service-tier header in response

* docs: add service_tier to gemini docs

* chore: add defaut/standard mapping, and some tests

* chore: tidying up some case insensitivity

* chore: remove unnecessary guard

* fix: remove redundant test file

* fix: handle 'auto' case-insensitively

* fix: return service_tier on final steamed chunk

* chore: black

* feat: enable supports_service_tier to gemini models

* Fix get_standard_logging_metadata tests

* Fix test_get_model_info_bedrock_models

* Fix test_get_model_info_bedrock_models

* Fix remaining tests

* Fix mypy issues

* Fix tests

* Fix merge conflicts

* Fix code qa

* Fix code qa

* Fix code qa

* Fix greptile review

---------

Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Josh <36064836+J-Byron@users.noreply.github.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Alperen Kömürcü <alperen.koemuercue@sap.com>
Co-authored-by: Vasilisa Parshikova <vasilisa.parshikova@sap.com>
Co-authored-by: Lin Xu <lin.xu03@sap.com>
Co-authored-by: Mark McDonald <macd@google.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
2026-04-08 21:37:10 -07:00
ishaan-berriandGitHub 7a9a9f0c79 fix: batch-limit stale managed object cleanup to prevent 300K row UPD… (#25258)
* fix: batch-limit stale managed object cleanup to prevent 300K row UPDATE (#25257)

* Add STALE_OBJECT_CLEANUP_BATCH_SIZE constant

Configurable batch limit (default 1000) for stale managed object cleanup,
preventing unbounded UPDATE queries from hitting 300K+ rows at once.

* Batch-limit stale managed object cleanup with single bounded SQL query

Two fixes to _cleanup_stale_managed_objects:

1. Replace unbounded update_many with a single execute_raw using a
   subquery LIMIT, capping each poll cycle to STALE_OBJECT_CLEANUP_BATCH_SIZE
   rows. Zero rows loaded into Python memory — everything stays in Postgres.
   Uses the same PostgreSQL raw-SQL pattern as spend_log_cleanup.py
   (the proxy requires PostgreSQL per schema.prisma).

2. Extract _expire_stale_rows as a separate method for testability.

Keeps the file_purpose='response' filter to avoid incorrectly expiring
long-running batch or fine-tune jobs that legitimately exceed the
staleness cutoff.

* docs: add STALE_OBJECT_CLEANUP_BATCH_SIZE to env vars reference

* test: remove deprecated embed-english-v2.0 cohere embedding tests
2026-04-06 19:11:55 -07:00
ishaan-berriandGitHub 1e66050423 bump litellm-enterprise to 0.1.36 (#25164)
* bump litellm-enterprise version to 0.1.36

* bump litellm-enterprise==0.1.36 in pyproject.toml

* bump litellm-enterprise==0.1.36 in requirements.txt
2026-04-04 17:14:31 -07:00
Yuneng JiangandClaude Opus 4.6 0f785f988b [Fix] Add missing user_api_key_project_alias to failed-response PagerDuty event
The hanging-response constructor was fixed but the sibling failed-response
constructor at line 104 was still missing this field.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:07:04 -07:00
Yuneng JiangandClaude Opus 4.6 f08d281641 [Fix] Resolve mypy type errors across 3 files
Add missing `user_api_key_project_alias` key to SpendLogsMetadata and
PagerDutyInternalEvent constructors, and cast `reasoning_items` to list
for safe iteration in responses transformation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 10:56:54 -07:00
Sameer Kankute bbd8ca3b3d feat(prometheus): add metrics for managed batch lifecycle
- Add Prometheus metrics for managed batch and file operations
- Track batch creation, file size, duration, and deletion events
- Add CheckBatchCost polling metrics (jobs polled/processed, errors)
- Record metrics in managed_files hook and check_batch_cost utility
- Metrics include labels for model, provider, user, and status

Made-with: Cursor
2026-03-27 20:30:09 +05:30
Krrish Dholakia df2a36dd27 docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
Sameer Kankute 4f1e484a9b Merge branch 'main' into litellm_dev_sameer_16_march_week
Resolve conflicts in common_request_processing.py (keep main streaming,
post_call_success_hook try/finally, deferred logging; retain skip_pre_call_logic)
and utils.py (defer + internal-call skip + sync success callbacks for all calls).

Tighten _has_post_call_guardrails for event_hook=None; align deferred
guardrail test. Sync model_prices_and_context_window_backup.json.

Pyright: narrow ignores for passthrough StreamingResponse and post_call hook.
Made-with: Cursor
2026-03-22 00:29:38 +05:30
Sameer Kankute 676a79e9f7 bump: litellm-enterprise 0.1.34 → 0.1.35 2026-03-21 20:42:34 +05:30
Sameer Kankute 32ded9b2f8 fix double-billing issue 2026-03-18 12:47:42 +05:30
Sameer Kankute 7660f39fdb fix(file_search): promote DB helper, suppress sub-call billing, add queries-plural test
- Promote _fetch_managed_vector_stores_by_uuids from @staticmethod to a module-level
  async helper get_managed_vector_store_rows_by_uuids, following the same standalone
  helper pattern as get_team_object / get_key_object so the hot-path DB read is a
  named importable function rather than an inline prisma_client.db.* call
- Pass no-log=True to both inner _call_aresponses sub-calls so they do not fire
  independent billing/monitoring callbacks; cost is accumulated in the synthesized
  response's _hidden_params for the outer responses() call
- Add test_H11b covering the primary queries (plural array) function-tool schema,
  complementing H11 which exercises only the backward-compat singular query path

Made-with: Cursor
2026-03-18 11:38:49 +05:30
Sameer Kankute 76176f2a64 fix(file_search): restore should_use_emulated helper, fix dedup, extract DB helper, clean docstring
- Re-add should_use_emulated_file_search() to emulated_handler.py so H5/H6/H7/H13 tests don't fail with ImportError
- Remove per-file-id deduplication from _build_search_results_for_include so all chunks are returned (matching OpenAI native file_search behaviour); update test_H14 to assert 2 results
- Extract raw prisma DB query in check_vector_store_ids_access into a static _fetch_managed_vector_stores_by_uuids helper so the hot request path uses a named, testable function instead of an inline prisma_client.db.* call
- Remove developer-local path from test module docstring

Made-with: Cursor
2026-03-18 11:26:27 +05:30
Sameer KankuteandClaude Sonnet 4.6 c735251570 feat(responses): file_search support — Phase 1 native passthrough + Phase 2 emulated fallback
Phase 1 (native passthrough):
- _decode_vector_store_ids_in_tools(): decode LiteLLM-managed unified
  vector_store_ids to provider-native IDs in file_search tools
- Split update_responses_tools_with_model_file_ids() into decode pass
  (always runs) + code_interpreter mapping pass (guarded)
- BaseResponsesAPIConfig.supports_native_file_search() → False by default;
  OpenAIResponsesAPIConfig overrides to True
- ManagedFiles.async_pre_call_hook(): batch team-level access check for
  unified vector_store_ids in file_search tools (no N+1)
- Docs: file_search section in response_api.md

Phase 2 (emulated fallback for non-native providers):
- litellm/responses/file_search/emulated_handler.py: converts file_search
  tool → function tool, intercepts tool call, runs asearch(), makes
  follow-up call, synthesizes OpenAI-format output (file_search_call +
  message + file_citation annotations)
- responses/main.py: routes to emulated handler when provider doesn't
  support file_search natively

Tests: 41 unit tests across 8 families (A-H) in test_file_search_responses.py

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 11:41:44 +05:30
Sameer KankuteandGitHub ab377f396e Merge pull request #23718 from BerriAI/litellm_fix_vertex_ai_batch
Fix: Vertex ai Batch Output File Download Fails with 500
2026-03-16 19:05:49 +05:30