mirror of https://github.com/tiennm99/litellm.git synced 2026-07-17 12:17:35 +00:00

Files

T

ishaan-berriandGitHub 7a9a9f0c79 fix: batch-limit stale managed object cleanup to prevent 300K row UPD… (#25258 )

* fix: batch-limit stale managed object cleanup to prevent 300K row UPDATE (#25257)

* Add STALE_OBJECT_CLEANUP_BATCH_SIZE constant

Configurable batch limit (default 1000) for stale managed object cleanup,
preventing unbounded UPDATE queries from hitting 300K+ rows at once.

* Batch-limit stale managed object cleanup with single bounded SQL query

Two fixes to _cleanup_stale_managed_objects:

1. Replace unbounded update_many with a single execute_raw using a
   subquery LIMIT, capping each poll cycle to STALE_OBJECT_CLEANUP_BATCH_SIZE
   rows. Zero rows loaded into Python memory — everything stays in Postgres.
   Uses the same PostgreSQL raw-SQL pattern as spend_log_cleanup.py
   (the proxy requires PostgreSQL per schema.prisma).

2. Extract _expire_stale_rows as a separate method for testability.

Keeps the file_purpose='response' filter to avoid incorrectly expiring
long-running batch or fine-tune jobs that legitimately exceed the
staleness cutoff.

* docs: add STALE_OBJECT_CLEANUP_BATCH_SIZE to env vars reference

* test: remove deprecated embed-english-v2.0 cohere embedding tests

2026-04-06 19:11:55 -07:00

auth

removed /models and v1/models from llm api routes (#20988 )

2026-02-13 18:32:41 +05:30

common_utils

fix: batch-limit stale managed object cleanup to prevent 300K row UPD… (#25258 )

2026-04-06 19:11:55 -07:00

hooks

feat(prometheus): add metrics for managed batch lifecycle

2026-03-27 20:30:09 +05:30

management_endpoints

Org level tpm/rpm limits + Team tpm/rpm validation when assigned to org (#15549 )