diff --git a/deploy/charts/litellm/README.md b/deploy/charts/litellm/README.md index 817781ed04..e005280b8b 100644 --- a/deploy/charts/litellm/README.md +++ b/deploy/charts/litellm/README.md @@ -2,7 +2,7 @@ ## Prerequisites -- Kubernetes 1.23+ +- Kubernetes 1.21+ - Helm 3.8.0+ If `db.deployStandalone` is used: @@ -33,6 +33,7 @@ If `db.useStackgresOperator` is used (not yet implemented): | `proxy_config.*` | See [values.yaml](./values.yaml) for default settings. See [example_config_yaml](../../../litellm/proxy/example_config_yaml/) for configuration examples. | N/A | #### Example `environmentSecrets` Secret + ``` apiVersion: v1 kind: Secret diff --git a/deploy/charts/litellm/values.yaml b/deploy/charts/litellm/values.yaml index cc53fc59c9..891c44f2a9 100644 --- a/deploy/charts/litellm/values.yaml +++ b/deploy/charts/litellm/values.yaml @@ -6,7 +6,6 @@ replicaCount: 1 image: # Use "ghcr.io/berriai/litellm-database" for optimized image with database - # Alternatively, use "ghcr.io/berriai/litellm" for the default image repository: ghcr.io/berriai/litellm-database pullPolicy: IfNotPresent # Overrides the image tag whose default is the chart appVersion. @@ -85,10 +84,13 @@ proxy_config: litellm_params: model: gpt-3.5-turbo api_key: eXaMpLeOnLy + - model_name: fake-openai-endpoint + litellm_params: + model: openai/fake + api_key: fake-key + api_base: https://exampleopenaiendpoint-production.up.railway.app/ general_settings: master_key: os.environ/PROXY_MASTER_KEY -# litellm_settings: -# cache: true resources: {} # We usually recommend not to specify default resources and to leave this as a conscious diff --git a/docs/my-website/docs/proxy/deploy.md b/docs/my-website/docs/proxy/deploy.md index 175806d274..5a5b5d99fb 100644 --- a/docs/my-website/docs/proxy/deploy.md +++ b/docs/my-website/docs/proxy/deploy.md @@ -150,17 +150,20 @@ To avoid issues with predictability, difficulties in rollback, and inconsistent ## Deploy with Database +### Docker, Kubernetes, Helm Chart + + + + + We maintain a [seperate Dockerfile](https://github.com/BerriAI/litellm/pkgs/container/litellm-database) for reducing build time when running LiteLLM proxy with a connected Postgres Database - - - -``` +```shell docker pull docker pull ghcr.io/berriai/litellm-database:main-latest ``` -``` +```shell docker run --name litellm-proxy \ -e DATABASE_URL=postgresql://:@:/ \ -p 4000:4000 \ @@ -233,6 +236,8 @@ Your OpenAI proxy server is now running on `http://0.0.0.0:4000`. +Use this to deploy litellm using a helm chart. Link to [the LiteLLM Helm Chart](https://github.com/BerriAI/litellm/tree/main/deploy/charts/litellm) + #### Step 1. Clone the repository ```bash @@ -241,9 +246,11 @@ git clone https://github.com/BerriAI/litellm.git #### Step 2. Deploy with Helm +Run the following command in the root of your `litellm` repo. This will set the litellm proxy master key as `sk-1234` + ```bash helm install \ - --set masterkey=SuPeRsEcReT \ + --set masterkey=sk-1234 \ mydeploy \ deploy/charts/litellm ```