diff --git a/docs/my-website/docs/learn/gateway_quickstart.md b/docs/my-website/docs/learn/gateway_quickstart.md index f29f6e7d48..2823e97e40 100644 --- a/docs/my-website/docs/learn/gateway_quickstart.md +++ b/docs/my-website/docs/learn/gateway_quickstart.md @@ -59,7 +59,7 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \ ## 6. Check The Response -If the request succeeds, the proxy returns `200 OK` with the same OpenAI-style response shape LiteLLM uses in the SDK. +If the request succeeds, the proxy returns `200 OK` with an OpenAI-style response. The assistant text will be in: @@ -67,33 +67,48 @@ The assistant text will be in: choices[0].message.content ``` -It looks like this: +If your gateway is routing to OpenAI, a real response can look like this: ```json { "id": "chatcmpl-abc123", - "object": "chat.completion", "created": 1677858242, - "model": "gpt-4o-mini", + "model": "gpt-4o-mini-2024-07-18", + "object": "chat.completion", + "system_fingerprint": "fp_406d6473f8", "choices": [ { + "finish_reason": "stop", "index": 0, "message": { "role": "assistant", - "content": "Hello! How can I help?" - }, - "finish_reason": "stop" + "content": "Hello! How can I assist you today?", + "tool_calls": null, + "function_call": null, + "annotations": [] + } } ], "usage": { - "prompt_tokens": 12, "completion_tokens": 9, - "total_tokens": 21 - } + "prompt_tokens": 13, + "total_tokens": 22, + "completion_tokens_details": { + "accepted_prediction_tokens": 0, + "audio_tokens": 0, + "reasoning_tokens": 0, + "rejected_prediction_tokens": 0 + }, + "prompt_tokens_details": { + "audio_tokens": 0, + "cached_tokens": 0 + } + }, + "service_tier": "default" } ``` -`id`, `created`, token counts, and message text will vary by request. +`id`, `created`, the resolved model version, token counts, and message text will vary by request. Other providers may return a smaller or slightly different set of fields, but `choices[0].message.content` is the main field to read. ## 7. Add Keys And The UI diff --git a/docs/my-website/docs/learn/sdk_quickstart.md b/docs/my-website/docs/learn/sdk_quickstart.md index 3726f1f22b..b4342d54cd 100644 --- a/docs/my-website/docs/learn/sdk_quickstart.md +++ b/docs/my-website/docs/learn/sdk_quickstart.md @@ -56,7 +56,48 @@ prints the assistant text, for example: Hello! I'm doing well, thanks for asking. ``` -The full response is an OpenAI-style `ModelResponse` object. It looks like this: +If you print the full object with: + +```python +print(response) +``` + +you will see a Python `ModelResponse(...)` object. For an OpenAI-backed model, it can look like this: + +```python +ModelResponse( + id='chatcmpl-abc123', + created=1773782130, + model='gpt-4o-2024-08-06', + object='chat.completion', + system_fingerprint='fp_4ff89bf575', + choices=[ + Choices( + finish_reason='stop', + index=0, + message=Message( + content="Hello! I'm just a program, but I'm here to help you. How can I assist you today?", + role='assistant', + tool_calls=None, + function_call=None, + provider_specific_fields={'refusal': None}, + annotations=[] + ), + provider_specific_fields={} + ) + ], + usage=Usage( + completion_tokens=21, + prompt_tokens=13, + total_tokens=34, + completion_tokens_details=CompletionTokensDetailsWrapper(...), + prompt_tokens_details=PromptTokensDetailsWrapper(...) + ), + service_tier='default' +) +``` + +The same response follows an OpenAI-style shape. Conceptually, it looks like this: ```json { @@ -82,7 +123,9 @@ The full response is an OpenAI-style `ModelResponse` object. It looks like this: } ``` -`id`, `created`, token counts, and message text will vary by request. For the full output reference, see [completion output](/docs/completion/output). +`id`, `created`, token counts, and message text will vary by request. + +If you call an OpenAI-backed model, you may also see extra fields such as `system_fingerprint`, `service_tier`, `tool_calls`, `function_call`, `annotations`, `provider_specific_fields`, and detailed token usage. For the full output reference, see [completion output](/docs/completion/output). Need more provider examples? See the main [Getting Started](/docs/#quick-start) page.