mirror of https://github.com/tiennm99/litellm.git synced 2026-06-17 22:48:35 +00:00

Files

T

Emerson Gomes 8f242c42a1 fix(batch_completion): submit all model futures before waiting (#20705 )

* fix(batch_completion): submit all model futures before waiting

* test: add batch_completion all responses concurrency regression

* fix(batch_completion): continue collecting responses on per-model failures

* fix(batch_completion): handle empty and string models in all responses

* test(batch_completion): avoid blocking wait in concurrency regression

2026-02-11 15:44:37 +05:30

main.py

fix(batch_completion): submit all model futures before waiting (#20705 )

2026-02-11 15:44:37 +05:30

Readme.md

(fix) batch_completion fails with bedrock due to extraneous [max_workers] key (#6176 )

2024-10-12 14:10:24 +05:30

Readme.md

Implementation of `litellm.batch_completion`, `litellm.batch_completion_models`, `litellm.batch_completion_models_all_responses`

Doc: https://docs.litellm.ai/docs/completion/batching

LiteLLM Python SDK allows you to:

litellm.batch_completion Batch litellm.completion function for a given model.
litellm.batch_completion_models Send a request to multiple language models concurrently and return the response as soon as one of the models responds.
litellm.batch_completion_models_all_responses Send a request to multiple language models concurrently and return a list of responses from all models that respond.

Readme.md

Implementation of litellm.batch_completion, litellm.batch_completion_models, litellm.batch_completion_models_all_responses

Implementation of `litellm.batch_completion`, `litellm.batch_completion_models`, `litellm.batch_completion_models_all_responses`