mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-13 02:22:50 +00:00
docs: enhance gateway and SDK quickstart documentation
This commit is contained in:
@@ -59,7 +59,7 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \
|
||||
|
||||
## 6. Check The Response
|
||||
|
||||
If the request succeeds, the proxy returns `200 OK` with the same OpenAI-style response shape LiteLLM uses in the SDK.
|
||||
If the request succeeds, the proxy returns `200 OK` with an OpenAI-style response.
|
||||
|
||||
The assistant text will be in:
|
||||
|
||||
@@ -67,33 +67,48 @@ The assistant text will be in:
|
||||
choices[0].message.content
|
||||
```
|
||||
|
||||
It looks like this:
|
||||
If your gateway is routing to OpenAI, a real response can look like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-abc123",
|
||||
"object": "chat.completion",
|
||||
"created": 1677858242,
|
||||
"model": "gpt-4o-mini",
|
||||
"model": "gpt-4o-mini-2024-07-18",
|
||||
"object": "chat.completion",
|
||||
"system_fingerprint": "fp_406d6473f8",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "Hello! How can I help?"
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
"content": "Hello! How can I assist you today?",
|
||||
"tool_calls": null,
|
||||
"function_call": null,
|
||||
"annotations": []
|
||||
}
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 12,
|
||||
"completion_tokens": 9,
|
||||
"total_tokens": 21
|
||||
}
|
||||
"prompt_tokens": 13,
|
||||
"total_tokens": 22,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"reasoning_tokens": 0,
|
||||
"rejected_prediction_tokens": 0
|
||||
},
|
||||
"prompt_tokens_details": {
|
||||
"audio_tokens": 0,
|
||||
"cached_tokens": 0
|
||||
}
|
||||
},
|
||||
"service_tier": "default"
|
||||
}
|
||||
```
|
||||
|
||||
`id`, `created`, token counts, and message text will vary by request.
|
||||
`id`, `created`, the resolved model version, token counts, and message text will vary by request. Other providers may return a smaller or slightly different set of fields, but `choices[0].message.content` is the main field to read.
|
||||
|
||||
## 7. Add Keys And The UI
|
||||
|
||||
|
||||
@@ -56,7 +56,48 @@ prints the assistant text, for example:
|
||||
Hello! I'm doing well, thanks for asking.
|
||||
```
|
||||
|
||||
The full response is an OpenAI-style `ModelResponse` object. It looks like this:
|
||||
If you print the full object with:
|
||||
|
||||
```python
|
||||
print(response)
|
||||
```
|
||||
|
||||
you will see a Python `ModelResponse(...)` object. For an OpenAI-backed model, it can look like this:
|
||||
|
||||
```python
|
||||
ModelResponse(
|
||||
id='chatcmpl-abc123',
|
||||
created=1773782130,
|
||||
model='gpt-4o-2024-08-06',
|
||||
object='chat.completion',
|
||||
system_fingerprint='fp_4ff89bf575',
|
||||
choices=[
|
||||
Choices(
|
||||
finish_reason='stop',
|
||||
index=0,
|
||||
message=Message(
|
||||
content="Hello! I'm just a program, but I'm here to help you. How can I assist you today?",
|
||||
role='assistant',
|
||||
tool_calls=None,
|
||||
function_call=None,
|
||||
provider_specific_fields={'refusal': None},
|
||||
annotations=[]
|
||||
),
|
||||
provider_specific_fields={}
|
||||
)
|
||||
],
|
||||
usage=Usage(
|
||||
completion_tokens=21,
|
||||
prompt_tokens=13,
|
||||
total_tokens=34,
|
||||
completion_tokens_details=CompletionTokensDetailsWrapper(...),
|
||||
prompt_tokens_details=PromptTokensDetailsWrapper(...)
|
||||
),
|
||||
service_tier='default'
|
||||
)
|
||||
```
|
||||
|
||||
The same response follows an OpenAI-style shape. Conceptually, it looks like this:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -82,7 +123,9 @@ The full response is an OpenAI-style `ModelResponse` object. It looks like this:
|
||||
}
|
||||
```
|
||||
|
||||
`id`, `created`, token counts, and message text will vary by request. For the full output reference, see [completion output](/docs/completion/output).
|
||||
`id`, `created`, token counts, and message text will vary by request.
|
||||
|
||||
If you call an OpenAI-backed model, you may also see extra fields such as `system_fingerprint`, `service_tier`, `tool_calls`, `function_call`, `annotations`, `provider_specific_fields`, and detailed token usage. For the full output reference, see [completion output](/docs/completion/output).
|
||||
|
||||
Need more provider examples? See the main [Getting Started](/docs/#quick-start) page.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user