mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-12 21:04:10 +00:00
Ishaan Jaff
b7b20664c1
* feat: add LITELLM_WORKER_STARTUP_HOOKS for per-worker initialization (gflags support) Add support for running user-defined startup hooks in each worker process during proxy_startup_event. This enables re-initialization of in-process state (like gflags.FLAGS) that doesn't survive uvicorn worker spawning. Usage: export LITELLM_WORKER_STARTUP_HOOKS=mymodule:init_fn,other:setup_fn Hooks run early in proxy_startup_event (before config/DB loading). Supports both sync and async callables. Errors propagate to prevent broken workers from serving traffic. No-op when env var is unset. Includes 5 tests covering sync/async hooks, multiple hooks, error propagation, and no-hooks-set scenarios. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * docs: add Worker Startup Hooks page with gflags usage example - New docs page: docs/proxy/worker_startup_hooks.md - Explains the problem (per-process state lost in multi-worker deployments) - Full gflags example with wrapper module and startup script - Covers multiple hooks, async hooks, error behavior - Architecture diagram showing master→worker flow - Added LITELLM_WORKER_STARTUP_HOOKS to config_settings.md env var table - Added to sidebar under Setup & Deployment Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * Update litellm/proxy/proxy_server.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Apply suggestion from @greptile-apps[bot] Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Website
This website is built using Docusaurus 2, a modern static website generator.
Installation
$ yarn
Local Development
$ yarn start
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
Build
$ yarn build
This command generates static content into the build directory and can be served using any static contents hosting service.
Deployment
Using SSH:
$ USE_SSH=true yarn deploy
Not using SSH:
$ GIT_USER=<Your GitHub username> yarn deploy
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.