From 8213e239709e1a9d31da454b8a94c5d9cd8deefa Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Thu, 28 Sep 2023 22:39:41 -0700 Subject: [PATCH] update docs --- docs/my-website/docs/proxy_server.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/my-website/docs/proxy_server.md b/docs/my-website/docs/proxy_server.md index 061efae2eb..7b565a6078 100644 --- a/docs/my-website/docs/proxy_server.md +++ b/docs/my-website/docs/proxy_server.md @@ -149,13 +149,13 @@ $ litellm --model command-nightly [**Jump to Code**](https://github.com/BerriAI/litellm/blob/fef4146396d5d87006259e00095a62e3900d6bb4/litellm/proxy.py#L36) -## setting api base -For **local model** or model on **custom endpoint** - -Pass in the api_base as well +## setting api base, temperature, max tokens ```shell -litellm --model huggingface/meta-llama/llama2 --api_base https://my-endpoint.huggingface.cloud +litellm --model huggingface/bigcode/starcoder \ + --api_base https://my-endpoint.huggingface.cloud \ + --max_tokens 250 \ + --temperature 0.5 ``` **Ollama example**