litellm

tiennm99/litellm

Fork 0

mirror of https://github.com/tiennm99/litellm.git synced 2026-07-17 08:18:09 +00:00

Files

T

History

Krish DholakiaandGitHub fdfef04d93 Gemini Multimodal Live API support (#10841 )

* fix: initial commit

* refactor(gemini/realtime/transformation.py): initial instrumentation

* fix: fix default api base if not set

* feat(gemini/): passes initial user request to backend

* feat(realtime_streaming.py): support transforming message input before sending it

enables gemini realtime streaming to be sent in correct format

* feat: initial working commit of setup message being sent + working

* fix(gemini/): initial commit supporting realtime response transformation

* feat(gemini/realtime): transform session.created event correctly

* fix(realtime_streaming.py): more gemini/realtime response mapping - handle new message

* test(gemini/realtime/test_): add more unit tests

* feat(gemini/realtime): handles consecutive deltas

* feat(gemini/realtime): support openai 'response.text.done' event

* feat(gemini/realtime): add openai 'response.text.done' and 'response.content_part.done' event support

* fix(gemini/realtime): add openai 'response.done' event support

unified realtime api support

* fix: fix linting errors

* fix: fix linting error

* fix: fix linting error

* fix: handle infinite loop

* fix: fix linting error

* fix: fix recursive detector

* fix: fix file

* fix: fix linting error

* fix: fix linting error

2025-05-15 22:39:09 -07:00

audio_utils

(Refactor) - Re use litellm.completion/litellm.embedding etc for health checks (#7455 )

2024-12-28 18:38:54 -08:00

llm_cost_calc

Anthropic unified web search + tool cost tracking support (#10846 )

2025-05-14 22:41:12 -07:00

llm_response_utils

Migrate cohere to /v2/embed - adds support for output_dimensions param (#10809 )

2025-05-13 23:06:56 -07:00

prompt_templates

[Bedrock] Fixes to only allow accepted fields for tool json schema (#10062 )

2025-05-13 19:49:30 -07:00

specialty_caches

Fix team-based logging to langfuse + allow custom tokenizer on /token_counter endpoint (#7493 )

2024-12-31 23:18:41 -08:00

tokenizers

Code Quality Improvement - remove tokenizers/ from /llms (#7163 )

2024-12-10 23:50:15 -08:00

asyncify.py

(core sdk fix) - fix fallbacks stuck in infinite loop (#7751 )

2025-01-13 19:34:34 -08:00

core_helpers.py

fix(proxy/_types.py): add missing comma for /v2/rerank (#10861 )

2025-05-15 21:37:46 -07:00

credential_accessor.py

fix(router.py): support reusable credentials via passthrough router (#9758 )

2025-04-04 18:40:14 -07:00

dd_tracing.py

(Bug fix) - don't log messages in model_parameters in StandardLoggingPayload (#8932 )

2025-03-01 13:39:45 -08:00

default_encoding.py

build(pyproject.toml): add new dev dependencies - for type checking (#9631 )

2025-03-29 11:02:13 -07:00

dot_notation_indexing.py

feat(handle_jwt.py): initial commit adding custom RBAC support on jwt… (#8037 )

2025-01-28 16:27:06 -08:00

duration_parser.py

Anthropic unified web search + tool cost tracking support (#10846 )

2025-05-14 22:41:12 -07:00

exception_mapping_utils.py

Add target model name validation (#10722 )

2025-05-10 14:27:06 -07:00

fallback_utils.py

[Bug Fix] UI QA - Fix wildcard model test connection not working (#10347 )

2025-04-26 17:42:06 -07:00

get_litellm_params.py

[Feat] Option to force/always use the litellm proxy (#10559 ) (#10633 ) (#10773 )

2025-05-12 20:22:54 -07:00

get_llm_provider_logic.py

Add new model provider Novita AI (#7582 ) (#9527 )

2025-05-12 21:49:30 -07:00

get_model_cost_map.py

Litellm UI qa 04 12 2025 p1 (#9955 )

2025-04-12 19:30:48 -07:00

get_supported_openai_params.py

Add new model provider Novita AI (#7582 ) (#9527 )

2025-05-12 21:49:30 -07:00

health_check_utils.py

(Refactor) - Re use litellm.completion/litellm.embedding etc for health checks (#7455 )

2024-12-28 18:38:54 -08:00

initialize_dynamic_callback_params.py

Fix team-based logging to langfuse + allow custom tokenizer on /token_counter endpoint (#7493 )

2024-12-31 23:18:41 -08:00

json_validation_rule.py

Fixed Ollama Structured Response not working #10616 (#10617 )

2025-05-12 07:02:52 -07:00

litellm_logging.py

[Refactor] Make Pagerduty a free feature (#10857 )

2025-05-15 10:12:06 -07:00

llm_request_utils.py

Revert "test_completion_mistral_api_mistral_large_function_call"

2025-01-17 07:20:46 -08:00

logging_callback_manager.py

[Feat] V2 Emails - Fixes for sending emails when creating keys + Resend API support (#10602 )

2025-05-06 22:50:48 -07:00

logging_utils.py

fix(streaming_handler.py): emit deep copy of completed chunk

2025-03-17 17:26:21 -07:00

mock_functions.py

Ensure base_model cost tracking works across all endpoints (#7989 )

2025-01-24 21:05:26 -08:00

model_param_helper.py

[Bug Fix] caching does not account for thinking or reasoning_effort config (#10140 )

2025-04-21 22:39:40 -07:00

README.md

(QOL improvement) Provider budget routing - allow using 1s, 1d, 1mo, 2mo etc (#6885 )

2024-11-23 16:59:46 -08:00

realtime_streaming.py

Gemini Multimodal Live API support (#10841 )

2025-05-15 22:39:09 -07:00

redact_messages.py

build(pyproject.toml): add new dev dependencies - for type checking (#9631 )

2025-03-29 11:02:13 -07:00

response_header_helpers.py

fix(utils.py): guarantee openai-compatible headers always exist in response

2024-09-28 21:08:15 -07:00

rules.py

Litellm dev 11 07 2024 (#6649 )

2024-11-08 19:34:22 +05:30

safe_json_dumps.py

Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state

2025-03-28 13:11:19 -07:00

safe_json_loads.py

add support to parse metadata (#10832 )

2025-05-14 12:03:49 -07:00

sensitive_data_masker.py

test: fix litellm_mapped_tests

2025-05-14 18:29:01 -07:00

streaming_chunk_builder_utils.py

Fix: Prevent cache token overwrite by last chunk in streaming usage (#10284 )

2025-04-26 12:06:12 -07:00

streaming_handler.py

fix(streaming_handler.py): patch vertex ai meta llama streaming error (#10878 )

2025-05-15 21:44:57 -07:00

thread_pool_executor.py

(Fixes) OpenAI Streaming Token Counting + Fixes usage track when litellm.turn_off_message_logging=True (#8156 )

2025-01-31 15:06:37 -08:00

token_counter.py

fix(token_counter.py): handle empty lists

2025-05-02 08:07:08 -07:00

README.md

Folder Contents

This folder contains general-purpose utilities that are used in multiple places in the codebase.

Core files:

streaming_handler.py: The core streaming logic + streaming related helper utils
core_helpers.py: code used in types/ - e.g. map_finish_reason.
exception_mapping_utils.py: utils for mapping exceptions to openai-compatible error types.
default_encoding.py: code for loading the default encoding (tiktoken)
get_llm_provider_logic.py: code for inferring the LLM provider from a given model name.
duration_parser.py: code for parsing durations - e.g. "1d", "1mo", "10s"