From 7f5c92ddebf1dea5a1fcdff403c898c0a88d2d77 Mon Sep 17 00:00:00 2001 From: Teddy Amkie <38896345+TeddyAmkie@users.noreply.github.com> Date: Fri, 29 Aug 2025 09:59:07 -0700 Subject: [PATCH] Update docs for stream_timeout and timeout (#14073) --- docs/my-website/docs/proxy/timeout.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/my-website/docs/proxy/timeout.md b/docs/my-website/docs/proxy/timeout.md index 85428ae53e..52cb160cf7 100644 --- a/docs/my-website/docs/proxy/timeout.md +++ b/docs/my-website/docs/proxy/timeout.md @@ -38,9 +38,15 @@ $ litellm --config /path/to/config.yaml -### Custom Timeouts, Stream Timeouts - Per Model -For each model you can set `timeout` & `stream_timeout` under `litellm_params` +### Custom Timeouts & Stream Timeouts (Per Model) +For each model, you can set `timeout` and `stream_timeout` under `litellm_params`: + +- **`timeout`** → maximum time for the *complete response*. + Use this to cap long-running completions. + +- **`stream_timeout`** → maximum time to wait for the *first chunk* (i.e., first token) in a streaming response. + Use this to abort “hanging” providers (e.g., Bedrock slow start) and retry another model.