* change deployment configs to include a litellm.cache for litellm-backend pod mirroring litellm-gateway pod
* omit backend annotations block when config and podAnnotations are both empty
* reuse gateway config/configmap for backend instead of separate backend config
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Tin Chi Lo <tin@Tins-MBP.localdomain>
Co-authored-by: Tin Chi Lo <tin@Tins-MacBook-Pro.local>
* feat(helm): split per-component ServiceAccounts for gateway, backend, and UI
Replace the single shared serviceAccount with three separate serviceAccounts
(gateway, backend, ui) so operators can attach different IRSA / Workload
Identity annotations per component without granting data-plane credentials
to the UI pod.
Key changes:
- values.yaml: rename serviceAccount → serviceAccounts with gateway/backend/ui
sub-keys; UI defaults to automount: false
- _helpers.tpl: replace litellm.serviceAccountName with three component-scoped
helpers (litellm.gateway/backend/ui.serviceAccountName)
- serviceaccount.yaml: create up to three separate ServiceAccount objects with
component labels and per-SA automountServiceAccountToken
- gateway/backend deployments: use their respective SA helpers
- ui deployment: use litellm.ui.serviceAccountName + explicit
automountServiceAccountToken: false on the pod spec so the projected token
is absent even when the SA itself allows it
- migrations-job: share the backend SA (both need DB write access)
Resolves LIT-3171
https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF
* fix(helm): enforce automountServiceAccountToken on all pod specs; fix leading --- in serviceaccount.yaml
- gateway/backend deployments: add explicit automountServiceAccountToken on
the pod spec so serviceAccounts.*.automount is honoured regardless of
whether the SA is chart-created or operator-supplied (previously the flag
only took effect on the SA object when create: true, creating an asymmetry
with the UI which already enforced it at pod-spec level)
- serviceaccount.yaml: use a $prev sentinel to emit --- only between
documents, preventing a leading --- when gateway SA is skipped but
backend or ui SA is created (avoids lint/GitOps warnings from strict
YAML parsers and tools like ArgoCD)
https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF
---------
Co-authored-by: Claude <noreply@anthropic.com>
Split the monolithic LiteLLM proxy into independently scalable Kubernetes components to allow separate horizontal scaling of the LLM data plane and management API surfaces
- Add DatabaseURLSettings pydantic-settings model that assembles DATABASE_URL (and optional DATABASE_URL_READ_REPLICA) from discrete DATABASE_* env vars before Prisma initializes, supporting both IAM token auth (minting short-lived RDS tokens) and password auth; replaces the CLI-only path that componentized entrypoints bypass
- Add gateway component (port 4000) that trims the proxy route table to the LLM data-plane surface (chat, embeddings, completions, audio, realtime, provider passthroughs, health/metrics) via an allowlist applied inside the lifespan context so plugin-registered routes are captured
- Add backend component (port 4001) that exposes the management/admin surface (keys, users, teams, orgs, spend analytics, model management, SSO, audit logs) with a complementary allowlist
- Add ui component — Next.js static export served by nginx (port 3000) with RSC payload routing, asset prefix aliasing, and SPA fallback for dashboard routes
- Add migrations component with dedicated Dockerfile that runs prisma migrate deploy via a Helm pre-install/pre-upgrade Job, eliminating per-pod schema contention on the Prisma advisory lock
- Add Helm chart (helm/litellm) with separate Deployments, Services, HPAs, and ConfigMap for each component; shared _helpers.tpl emits DATABASE_*, IAM_TOKEN_DB_AUTH, REDIS_*, and DISABLE_SCHEMA_UPDATE env vars from chart values; ingress template routes traffic to the correct component by path prefix
- Add comprehensive tests for DatabaseURLSettings covering IAM auth, password auth, read replica fallbacks, operator-pinned URL preservation, and percent-encoding; add coverage test asserting gateway + backend allowlist union equals the full proxy route set
- Add pydantic-settings>=2.14.1 as a proxy extra dependency and update liccheck allowlist
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
