From 8cced8eabc8953fc0ef7eb03eb71cb0f91f949c7 Mon Sep 17 00:00:00 2001 From: ishaan-jaff Date: Wed, 15 Nov 2023 13:33:28 -0800 Subject: [PATCH] (docs) proxy cli settings --- docs/my-website/docs/simple_proxy.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/my-website/docs/simple_proxy.md b/docs/my-website/docs/simple_proxy.md index 81a73f14bd..64e4deead0 100644 --- a/docs/my-website/docs/simple_proxy.md +++ b/docs/my-website/docs/simple_proxy.md @@ -347,7 +347,7 @@ Since LiteLLM provides an OpenAI compatible proxy `-t` and `-m` don't need to ch MLflow provides an API `mlflow.evaluate()` to help evaluate your LLMs https://mlflow.org/docs/latest/llms/llm-evaluate/index.html -## Pre Requisites +#### Pre Requisites ```shell pip install litellm ``` @@ -355,7 +355,7 @@ pip install litellm pip install mlflow ``` -### Step 1: Start LiteLLM Proxy on the CLI +#### Step 1: Start LiteLLM Proxy on the CLI LiteLLM allows you to create an OpenAI compatible server for all supported LLMs. [More information on litellm proxy here](https://docs.litellm.ai/docs/simple_proxy) ```shell @@ -364,7 +364,7 @@ $ litellm --model huggingface/bigcode/starcoder #INFO: Proxy running on http://0.0.0.0:8000 ``` -### Step 2: Run ML Flow +#### Step 2: Run ML Flow Before running the eval we will set `openai.api_base` to the litellm proxy from Step 1 ```python