mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-08 03:14:09 +00:00
Ishaan Jaff
4d1b4beb3d
* use folder for caching * fix importing caching * fix clickhouse pyright * fix linting * fix correctly pass kwargs and args * fix test case for embedding * fix linting * fix embedding caching logic * fix refactor handle utils.py * fix test_embedding_caching_azure_individual_items_reordered
Implementation of litellm.batch_completion, litellm.batch_completion_models, litellm.batch_completion_models_all_responses
Doc: https://docs.litellm.ai/docs/completion/batching
LiteLLM Python SDK allows you to:
litellm.batch_completionBatch litellm.completion function for a given model.litellm.batch_completion_modelsSend a request to multiple language models concurrently and return the response as soon as one of the models responds.litellm.batch_completion_models_all_responsesSend a request to multiple language models concurrently and return a list of responses from all models that respond.