tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-07 22:21:46 +00:00
Code Issues Packages Projects Releases Wiki Activity
24,346 Commits 2,072 Branches 1,425 Tags
6cd5afa8b11ea3654a2c26d3d64cd2a2e3eff3f9
Commit Graph
2825 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub 04dc1a5351 [Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
 2025-08-27 16:16:19 -07:00
Xingyao WangGitHubopenhands
 		9acf80b8ad Fix issue #13995: Handle None metadata in batch requests (#13996) 
* Fix issue #13995: Handle None metadata in batch requests

- Added null check in add_key_level_controls method to prevent NoneType error
- Updated type hint to Optional[dict] for better type safety
- Added comprehensive test suite to verify the fix works correctly
- All existing tests pass, confirming no regression

Fixes #13995

Co-authored-by: openhands <openhands@all-hands.dev>

* Move test file to tests/test_litellm/proxy/ directory

- Moved test_batch_metadata_none_fix.py from tests/ to tests/test_litellm/proxy/
- Updated import structure to match existing test patterns
- This ensures the test runs in GitHub Actions as requested by @krrishdholakia

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
 2025-08-27 14:51:09 -07:00
Ishaan JaffandGitHub 165242e31f [Feat] langfuse_otel logger - allow using LANGFUSE_OTEL_HOST for configuring host (#14013) 
* feat - add _get_langfuse_otel_host

* test_get_langfuse_otel_config_with_otel_host_priority

* docs: LANGFUSE_OTEL_HOST
 2025-08-27 14:43:47 -07:00
Sam XieandGitHub ee13c65701 [Bug]: /responses endpoint proxy ignores extra_headers in GitHub Copilot (#13775) 
* fix: pass extra_headers parameter through responses API transformation chain

Ensure extra_headers parameter is properly forwarded from the responses() function
through the transformation handler and config to maintain header propagation in
litellm_completion_request dict.

* Add tests
 2025-08-27 12:35:09 -07:00
Ishaan JaffandGitHub 4fff05f1cc [Feature]: Support Gemini requests with only system prompt (#14010) 
* _default_user_message_when_system_message_passed

* test_system_prompt_only_adds_blank_user_message

* test_system_message_with_no_user_message
 2025-08-27 12:12:47 -07:00
Ishaan JaffandGitHub f35ce02475 [Bug Fix] LLM Translation - Allow using dynamic api_key for image generation requests (#14007) 
* fix - allow using dyanmic api key for img gen

* test_aiml_image_generation_with_dynamic_api_key
 2025-08-27 10:51:24 -07:00
Ishaan Jaff ed52b67fcf test_image_edit_array_handling 2025-08-27 09:51:22 -07:00
Ishaan Jaff d5440f9614 test_image_edit_array_handling 2025-08-27 08:27:57 -07:00
Ishaan Jaff 76f78f7b2a fix: LITELLM_LOG_FILE test 2025-08-27 08:25:24 -07:00
Krish DholakiaandGitHub 900bd10905 Merge branch 'main' into feature/enhance_logging_for_containers 2025-08-26 23:21:15 -07:00
Krish DholakiaandGitHub 71af7bcf9c Merge pull request #13374 from blahgeek/fix-count-special-token 
Fix token_counter with special token input
 2025-08-26 23:19:26 -07:00
Krish DholakiaandGitHub 45f465d84f Merge pull request #13572 from Noma-Security/noma-security-guardrails-support 
Add Noma Security guardrail support
 2025-08-26 23:15:17 -07:00
Ishaan JaffandGitHub ab7efaa832 test_pre_process_non_default_params (#13990) 2025-08-26 19:15:17 -07:00
Cole FrenchandGitHub 6696955506 [Bug]: Fix tests to reference moved attributes in braintrust_logging module (#13978) 
* Mock instance attribute moved from globals

* Support looking up mock message on mock choice
 2025-08-26 18:58:16 -07:00
Krish DholakiaandGitHub ae678a6642 Merge pull request #13573 from Ivy-Interactive/feature/braintrust-span-name-metadata 
Feature/braintrust span name metadata
 2025-08-25 23:19:56 -07:00
Krish DholakiaandGitHub 3ed1220f75 Merge pull request #13916 from mubashir1osmani/openai-multiple-images 
feat: multiple images in openai images/edits endpoint
 2025-08-25 23:11:13 -07:00
Krish DholakiaandGitHub 22934907ba Merge pull request #13933 from michal-otmianowski/ignore-expired-s3-cache-entries 
Verify if cache entry has expired prior to serving it to client
 2025-08-25 23:10:48 -07:00
Krish DholakiaandGitHub b1bc8e3aa2 Merge pull request #13955 from ckoehler/push-oymwkvxmkqxv 
feat: add image headers for Copilot
 2025-08-25 23:04:24 -07:00
Ishaan JaffandGitHub 6ce1d82970 [Bug] Fix: Vertex Mistral not working for streaming (#13952) 
* fix OpenAI like chat handler

* fix MockResponse

* test_partner_models_httpx_streaming

* test_partner_models_httpx_streaming
 2025-08-25 17:39:40 -07:00
Christoph Koehler 04cca1e7f3 feat: add image headers for Copilot 
Fixes #13696. See issue for details.
 2025-08-25 18:15:26 -06:00
Ishaan JaffandGitHub c7b0c57b1e [Bug Fix] Azure Passthrough request with streaming (#13831) 
* fix: _update_stream_param_based_on_request_body

* test_update_stream_param_based_on_request_body

* test_pass_through_request_stream_param_override
 2025-08-25 14:11:17 -07:00
Ishaan JaffandGitHub 433d1a4947 [Bug fix] - Fix /messages fallback from Anthropic API -> Bedrock API (#13946) 
* use helper get_provider_specific_headers

* fix get_provider_specific_headers

* test_anthropic_messages_fallbacks

* bedrock/us.anthropic.claude-sonnet-4

* fix: get_provider_specific_headers

* TestProviderSpecificHeaderUtils

* test_anthropic_messages_fallbacks
 2025-08-25 13:44:54 -07:00
Michal Otmianowski c444263e7d verify expires field prior to serving cache entry 2025-08-25 10:42:12 +02:00
Steven Anton d95578464e Add support for AWS assume_role with a session token 2025-08-23 22:37:21 -07:00
Ishaan Jaff e0e356e328 test mapped test fixes 2025-08-23 17:04:23 -07:00
mubashir1osmani 2fa8f971e0 feat: multiple images in openai images/edits endpoint 2025-08-23 19:13:32 -04:00
Ishaan Jaff 0fccd619ea test_vertex_ai_deepseek 2025-08-23 14:13:03 -07:00
Ishaan Jaff cf64033044 test fix 2025-08-23 14:04:41 -07:00
Ishaan Jaff f1c710a608 test_e2e_generate_cold_storage_object_key_not_configured 2025-08-23 13:33:33 -07:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
 2025-08-23 13:13:23 -07:00
Ishaan JaffandGitHub 9750374081 [Feat] New LLM API - AI/ML API for Image Gen (#13893) 
* add LlmProviders.AIML

* add AIMLChatConfig

* add aiml

* fix AimlImageGenerationConfig

* add AimlImageGenerationConfig

* add cost_calculator

* fixes for AI ML API

* add known AIML Flux image models

* add AIML Cost Calc

* add AI/ML API

* add aiml_models
 2025-08-23 13:12:44 -07:00
Krish DholakiaandGitHub 9df6f4ef08 Merge branch 'main' into litellm_dev_08_20_2025_p1 2025-08-23 12:09:53 -07:00
Krrish Dholakia b23e35f75e test: update tests 2025-08-23 12:09:19 -07:00
Krish DholakiaandGitHub ef66bb87b3 Merge branch 'main' into litellm_dev_08_16_2025_p2 2025-08-23 11:30:51 -07:00
Krrish Dholakia 385bb15219 test: skip test 2025-08-23 11:27:46 -07:00
Krish DholakiaandGitHub 06a2915e9f Merge pull request #13671 from BerriAI/litellm_dev_08_15_2025_p1 
Responses API - support `allowed_openai_params` + Mistral - handle empty assistant content + support new mistral 'thinking' response block
 2025-08-23 11:15:23 -07:00
Krrish Dholakia 4d7433b5ad test: reduce mistral direct tests b/c of rate limit errors 2025-08-23 11:15:03 -07:00
Krrish Dholakia cbb161f10b test: handle internal server errors 2025-08-23 11:00:18 -07:00
Krish DholakiaandGitHub f0b0ee51e0 Merge branch 'main' into litellm_dev_08_15_2025_p1 2025-08-23 10:51:56 -07:00
Krrish Dholakia ba9f6894b7 test: update tests 2025-08-23 10:42:41 -07:00
Krish DholakiaandGitHub 3e764ec268 Merge pull request #13808 from mainred/validate_api_version 
feat(utils.py): accept 'api_version' as param for validate_environment
 2025-08-22 23:59:38 -07:00
Krish DholakiaandGitHub 867b48a24f Merge pull request #13866 from jinskjoy/Mistral-File-ID 
[Enhancement] Add support for Mistral model file handling and update documentation
 2025-08-22 23:54:17 -07:00
Ishaan Jaff 825d6b08f0 test fix location of encode bedrock runtime modelid arn 2025-08-22 17:34:09 -07:00
Ishaan JaffandGitHub 56c16fc6f6 [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895) 
* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix
 2025-08-22 16:17:57 -07:00
Jugal D. BhattandGitHub 3c1dcb64cc fix by adding init lock (#13666) 2025-08-22 11:42:43 -07:00
Krish DholakiaandGitHub dcad39c00f Merge pull request #13753 from RichardoC/fix-bedrock-irsa-role-assumption 
fix: role chaining and session name with webauthentication for aws bedrock
 2025-08-22 07:20:27 -07:00
Krish DholakiaandGitHub 59e43595b9 Merge pull request #13852 from michal-otmianowski/async-s3-implementation 
Async s3 implementation
 2025-08-21 23:19:51 -07:00
Krish DholakiaandGitHub 8b45e53878 Merge pull request #13880 from datarobot-forks/carsongee/improve_datarobot_api_handling 
🐛 Bug Fix: Updated URL handling for DataRobot provider URL
 2025-08-21 23:18:40 -07:00
Krish DholakiaandGitHub 9c4a86e98e Merge pull request #13868 from kankute-sameer/litellm_feat_voyage_context_3_embedding_model 
[Feat] Add support for voyage-context-3 embedding model
 2025-08-21 23:18:20 -07:00
Krish DholakiaandGitHub 6214d22126 Merge pull request #13863 from luizrennocosta/fix/pll-req-lim-v3-api-key 
fix: fixing descriptor/response size mismatch on parallel_request_limiter_v3
 2025-08-21 23:14:04 -07:00