mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-13 09:06:18 +00:00
Ishaan Jaff
b7b20664c1
* feat: add LITELLM_WORKER_STARTUP_HOOKS for per-worker initialization (gflags support) Add support for running user-defined startup hooks in each worker process during proxy_startup_event. This enables re-initialization of in-process state (like gflags.FLAGS) that doesn't survive uvicorn worker spawning. Usage: export LITELLM_WORKER_STARTUP_HOOKS=mymodule:init_fn,other:setup_fn Hooks run early in proxy_startup_event (before config/DB loading). Supports both sync and async callables. Errors propagate to prevent broken workers from serving traffic. No-op when env var is unset. Includes 5 tests covering sync/async hooks, multiple hooks, error propagation, and no-hooks-set scenarios. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * docs: add Worker Startup Hooks page with gflags usage example - New docs page: docs/proxy/worker_startup_hooks.md - Explains the problem (per-process state lost in multi-worker deployments) - Full gflags example with wrapper module and startup script - Covers multiple hooks, async hooks, error behavior - Architecture diagram showing master→worker flow - Added LITELLM_WORKER_STARTUP_HOOKS to config_settings.md env var table - Added to sidebar under Setup & Deployment Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * Update litellm/proxy/proxy_server.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Apply suggestion from @greptile-apps[bot] Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Testing for litellm/
This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.
The point of this is to:
- Increase test coverage of
litellm/ - Make it easy for contributors to add tests for the
litellm/package and easily run tests without needing LLM API keys.
File name conventions
litellm/proxy/test_caching_routes.pymaps tolitellm/proxy/caching_routes.pytest_<filename>.pymaps tolitellm/<filename>.py