From 27cd012c2ded48c1eaad16f567a12ea1bec73cdb Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Sat, 16 Mar 2024 12:14:35 -0700 Subject: [PATCH] (docs) litellm hel --- docs/my-website/docs/proxy/deploy.md | 95 ++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/docs/my-website/docs/proxy/deploy.md b/docs/my-website/docs/proxy/deploy.md index 28773e2eb4..37c7414dfd 100644 --- a/docs/my-website/docs/proxy/deploy.md +++ b/docs/my-website/docs/proxy/deploy.md @@ -135,6 +135,50 @@ To avoid issues with predictability, difficulties in rollback, and inconsistent + + + + +:::info + +[BETA] Helm Chart is BETA. If you run into an issues/have feedback please let us know [https://github.com/BerriAI/litellm/issues](https://github.com/BerriAI/litellm/issues) + +::: + +Use this when you want to use litellm helm chart as a dependency for other charts. The `litellm-helm` OCI is hosted here [https://github.com/BerriAI/litellm/pkgs/container/litellm-helm](https://github.com/BerriAI/litellm/pkgs/container/litellm-helm) + +#### Step 1. Pull the litellm helm chart + +```bash +helm pull oci://ghcr.io/berriai/litellm-helm + +# Pulled: ghcr.io/berriai/litellm-helm:0.1.2 +# Digest: sha256:7d3ded1c99c1597f9ad4dc49d84327cf1db6e0faa0eeea0c614be5526ae94e2a +``` + +#### Step 2. Unzip litellm helm +Unzip the specific version that was pulled in Step 1 + +```bash +tar -zxvf litellm-helm-0.1.2.tgz +``` + +#### Step 3. Install litellm helm + +```bash +helm install lite-helm ./litellm-helm +``` + +#### Step 4. Expose the service to localhost + +```bash +kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT +``` + +Your OpenAI proxy server is now running on `http://127.0.0.1:4000`. + + + **That's it ! That's the quick start to deploy litellm** @@ -236,6 +280,14 @@ Your OpenAI proxy server is now running on `http://0.0.0.0:4000`. + + +:::info + +[BETA] Helm Chart is BETA. If you run into an issues/have feedback please let us know [https://github.com/BerriAI/litellm/issues](https://github.com/BerriAI/litellm/issues) + +::: + Use this to deploy litellm using a helm chart. Link to [the LiteLLM Helm Chart](https://github.com/BerriAI/litellm/tree/main/deploy/charts/litellm-helm) #### Step 1. Clone the repository @@ -269,6 +321,49 @@ Your OpenAI proxy server is now running on `http://127.0.0.1:4000`. If you need to set your litellm proxy config.yaml, you can find this in [values.yaml](https://github.com/BerriAI/litellm/blob/main/deploy/charts/litellm-helm/values.yaml) + + + + + +:::info + +[BETA] Helm Chart is BETA. If you run into an issues/have feedback please let us know [https://github.com/BerriAI/litellm/issues](https://github.com/BerriAI/litellm/issues) + +::: + +Use this when you want to use litellm helm chart as a dependency for other charts. The `litellm-helm` OCI is hosted here [https://github.com/BerriAI/litellm/pkgs/container/litellm-helm](https://github.com/BerriAI/litellm/pkgs/container/litellm-helm) + +#### Step 1. Pull the litellm helm chart + +```bash +helm pull oci://ghcr.io/berriai/litellm-helm + +# Pulled: ghcr.io/berriai/litellm-helm:0.1.2 +# Digest: sha256:7d3ded1c99c1597f9ad4dc49d84327cf1db6e0faa0eeea0c614be5526ae94e2a +``` + +#### Step 2. Unzip litellm helm +Unzip the specific version that was pulled in Step 1 + +```bash +tar -zxvf litellm-helm-0.1.2.tgz +``` + +#### Step 3. Install litellm helm + +```bash +helm install lite-helm ./litellm-helm +``` + +#### Step 4. Expose the service to localhost + +```bash +kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT +``` + +Your OpenAI proxy server is now running on `http://127.0.0.1:4000`. +