Ryan Crabbe 86ec2fbf84 perf(proxy): batch 11 create_task() calls into 1 in update_database() ...

Replace 11 separate asyncio.create_task() calls per request with a
single batched task that runs all spend-update helpers sequentially.
This reduces task scheduling overhead at high RPS (11,000 -> 1,000
tasks/sec at 1K RPS) and cuts 5 copy.deepcopy(payload) calls to 1
shared copy.

Also fixes a mutation bug where the daily agent spend handler received
the raw payload without deepcopy, unlike all other daily helpers.

2026-02-24 10:04:43 -08:00

..

a2a_protocol

fix: prompt registry

2026-02-18 00:34:54 +05:30

anthropic_interface

fix: prompt registry

2026-02-18 00:34:54 +05:30

assistants

…

batch_completion

fix: prompt registry

2026-02-18 00:34:54 +05:30

batches

fix: prompt registry

2026-02-18 00:34:54 +05:30

caching

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

completion_extras

fix code quality tests and mypy

2026-02-18 17:36:36 +05:30

containers

fix: prompt registry

2026-02-18 00:34:54 +05:30

endpoints/speech/speech_to_completion_bridge

…

evals

fix: prompt registry

2026-02-18 00:34:54 +05:30

experimental_mcp_client

fix: prompt registry

2026-02-18 00:34:54 +05:30

files

fix: prompt registry

2026-02-18 00:34:54 +05:30

fine_tuning

…

google_genai

fix: prompt registry

2026-02-18 00:34:54 +05:30

images

fix: prompt registry

2026-02-18 00:34:54 +05:30

integrations

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

interactions

fix: prompt registry

2026-02-18 00:34:54 +05:30

litellm_core_utils

fix(bedrock): pass timeout param to bedrock rerank http client (#22021)

2026-02-24 09:32:11 -08:00

llms

fix(bedrock): pass timeout param to bedrock rerank http client (#22021)

2026-02-24 09:32:11 -08:00

ocr

…

passthrough

fix(types): fix mypy errors in pass-through endpoint query param types

2026-02-19 12:24:14 -03:00

proxy

perf(proxy): batch 11 create_task() calls into 1 in update_database()

2026-02-24 10:04:43 -08:00

proxy_auth

fix: prompt registry

2026-02-18 00:34:54 +05:30

rag

fix: prompt registry

2026-02-18 00:34:54 +05:30

realtime_api

feat(realtime): guardrail hook for voice transcription (#21976)

2026-02-23 21:04:40 -08:00

rerank_api

fix(bedrock): pass timeout param to bedrock rerank http client (#22021)

2026-02-24 09:32:11 -08:00

responses

fix(responses): eliminate per-chunk thread spawning in async streaming path (#21709)

2026-02-20 16:26:23 -08:00

router_strategy

fix(lint): fix ruff/flake8 violations - unused imports, PLR0915, print statements (#21846)

2026-02-21 15:07:47 -08:00

router_utils

Revert "Merge pull request #21957 from jquinter/fix/flaky-rpm-limit-test"

2026-02-24 09:52:50 -03:00

search

…

secret_managers

fix: prompt registry

2026-02-18 00:34:54 +05:30

skills

fix: prompt registry

2026-02-18 00:34:54 +05:30

types

Merge pull request #21982 from BerriAI/litellm_fix_pat_token_mcp

2026-02-24 19:36:08 +05:30

vector_store_files

…

vector_stores

…

videos

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

__init__.py

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

_lazy_imports_registry.py

Merge branch 'main' into litellm_oss_staging_02_17_2026

2026-02-18 17:26:33 +05:30

_lazy_imports.py

fix: prompt registry

2026-02-18 00:34:54 +05:30

_logging.py

fix: prompt registry

2026-02-18 00:34:54 +05:30

_redis.py

fix: close leaked Redis connection pools on cache eviction and disconnect

2026-02-20 17:09:32 -08:00

_service_logger.py

fix: prompt registry

2026-02-18 00:34:54 +05:30

_uuid.py

…

_version.py

…

anthropic_beta_headers_config.json

fix: enable context-1m-2025-08-07 beta header for vertex_ai provider (#21867)

2026-02-21 20:12:23 -08:00

anthropic_beta_headers_manager.py

fix: prompt registry

2026-02-18 00:34:54 +05:30

blog_posts.json

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

budget_manager.py

…

constants.py

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

cost_calculator.py

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

cost.json

…

exceptions.py

fix: prompt registry

2026-02-18 00:34:54 +05:30

main.py

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

model_prices_and_context_window_backup.json

fix(bedrock): pass timeout param to bedrock rerank http client (#22021)

2026-02-24 09:32:11 -08:00

mypy.ini

…

policy_templates_backup.json

fix(ollama): thread api_base to get_model_info + graceful fallback (#21970)

2026-02-23 21:00:37 -08:00

py.typed

…

router.py

Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817)

2026-02-21 15:32:47 -08:00

scheduler.py

fix: prompt registry

2026-02-18 00:34:54 +05:30

timeout.py

…

utils.py

Merge pull request #21992 from BerriAI/litellm_fix_oauth_mcp

2026-02-24 19:37:09 +05:30