diff --git a/docs/my-website/docs/simple_proxy.md b/docs/my-website/docs/simple_proxy.md index 4aad578d08..81a73f14bd 100644 --- a/docs/my-website/docs/simple_proxy.md +++ b/docs/my-website/docs/simple_proxy.md @@ -19,22 +19,56 @@ $ litellm --model huggingface/bigcode/starcoder ``` ### Test -In a new shell, run, this will make an `openai.ChatCompletion` request +In a new shell, run, this will make an `openai.chat.completions` request. Ensure you're using openai v1.0.0+ ```shell litellm --test ``` This will now automatically route any requests for gpt-3.5-turbo to bigcode starcoder, hosted on huggingface inference endpoints. -### Replace openai base +### Using LiteLLM Proxy - Curl Request, OpenAI Package + + + + +```shell +curl --location 'http://0.0.0.0:8000/chat/completions' \ +--header 'Content-Type: application/json' \ +--data ' { + "model": "gpt-3.5-turbo", + "messages": [ + { + "role": "user", + "content": "what llm are you" + } + ], + } +' +``` + + ```python -import openai +import openai +client = openai.OpenAI( + api_key="anything", + base_url="http://0.0.0.0:8000" +) -openai.api_base = "http://0.0.0.0:8000" +# request sent to model set on litellm proxy, `litellm --model` +response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ + { + "role": "user", + "content": "this is a test request, write a short poem" + } +]) + +print(response) -print(openai.chat.completions.create(model="test", messages=[{"role":"user", "content":"Hey!"}])) ``` + + + ### Supported LLMs