tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-07 08:24:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
24,415 Commits 2,072 Branches 1,425 Tags
f73b9da2cd2690f844b8fdc454e4e38296da3edc
Commit Graph
2855 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub fffccc0675 Merge pull request #14042 from WilsonSunBritten/10988-set-truncation-threshold 
Allow configuration to set threshold before request entry in spend log gets truncated
 2025-08-29 06:24:28 -07:00
Krish DholakiaandGitHub dd33e3f928 Merge pull request #14050 from lmwang9527/fix/logging-worker-graceful-shutdown 
Fix LoggingWorker graceful shutdown to prevent CancelledError warnings
 2025-08-29 06:23:53 -07:00
WilsonSunBritten b3edff02e8 fix mock tests 2025-08-28 17:40:16 -06:00
WilsonSunBritten 0721345703 Switch to string constant based truncation 2025-08-28 16:28:36 -06:00
Ishaan JaffandGitHub 37a599932d [Bug Fix] Virtual keys with llm_api type cause Internal Server Error when using /anthropic/* and other llm passthrough routes (#14046) 
* fix route checks passthrough_routes_wildcard

* refactor check_route_access

* test_virtual_key_llm_api_route_includes_passthrough_prefix
 2025-08-28 15:14:09 -07:00
WilsonSunBritten 026480358b Revert "Allow configuration to on what threshold to try truncating request content in db" 
This reverts commit 771052d67d.
 2025-08-28 16:12:42 -06:00
lmwang 6760a6aaa1 Fix LoggingWorker graceful shutdown to prevent CancelledError warnings 
- Change exception logging to debug level during shutdown cancellation
- Add clear_queue() call to process remaining coroutines during shutdown
- Prevents 'RuntimeWarning: coroutine was never awaited' messages
- Add comprehensive tests for graceful shutdown behavior

Fixes #13970
 2025-08-28 14:15:17 -07:00
Wilson Britten 771052d67d Allow configuration to on what threshold to try truncating request content in db 2025-08-28 09:48:50 -06:00
Krish DholakiaandGitHub e638c5490d Merge pull request #13945 from NoWall57/feature/force-anthropic-custom-api-base 
feat: Add support for custom Anthropic-compatible API endpoints
 2025-08-27 22:28:55 -07:00
Krish DholakiaandGitHub 74c9663189 Merge pull request #12797 from ericgtkb/fix-vertex-ai-anyof 
Fix additional anyOf corner cases for Vertex AI Gemini tool calls - issue #11164
 2025-08-27 22:26:13 -07:00
Krish DholakiaandGitHub fc738c5822 Merge branch 'main' into holmberg/default-reasoning 2025-08-27 22:25:22 -07:00
Krish DholakiaandGitHub 67d432a6d7 Merge pull request #13243 from nagyv-team/main 
Ensure that `function_call_prompt` extends system messages following its current schema
 2025-08-27 22:16:53 -07:00
Krish DholakiaandGitHub 10816a44d4 Merge pull request #13160 from pangeacyber/aidr-update 
Update Pangea Guardrail to support new AIDR endpoint
 2025-08-27 22:15:32 -07:00
Krish DholakiaandGitHub a913ecf60b Merge branch 'main' into feat/github-copilot-thinking-reasoning-support 2025-08-27 22:09:15 -07:00
Krish DholakiaandGitHub f01c7d13e7 Merge pull request #13991 from ikaadil/fix/gemini-imagen-model-name-validation 
[Bug Fix] Resolve invalid model name error for Gemini Imagen models
 2025-08-27 21:56:19 -07:00
Ishaan JaffandGitHub 04dc1a5351 [Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
 2025-08-27 16:16:19 -07:00
Xingyao WangGitHubopenhands
 		9acf80b8ad Fix issue #13995: Handle None metadata in batch requests (#13996) 
* Fix issue #13995: Handle None metadata in batch requests

- Added null check in add_key_level_controls method to prevent NoneType error
- Updated type hint to Optional[dict] for better type safety
- Added comprehensive test suite to verify the fix works correctly
- All existing tests pass, confirming no regression

Fixes #13995

Co-authored-by: openhands <openhands@all-hands.dev>

* Move test file to tests/test_litellm/proxy/ directory

- Moved test_batch_metadata_none_fix.py from tests/ to tests/test_litellm/proxy/
- Updated import structure to match existing test patterns
- This ensures the test runs in GitHub Actions as requested by @krrishdholakia

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
 2025-08-27 14:51:09 -07:00
Ishaan JaffandGitHub 165242e31f [Feat] langfuse_otel logger - allow using LANGFUSE_OTEL_HOST for configuring host (#14013) 
* feat - add _get_langfuse_otel_host

* test_get_langfuse_otel_config_with_otel_host_priority

* docs: LANGFUSE_OTEL_HOST
 2025-08-27 14:43:47 -07:00
Ifta Khairul Alam Adil 75aeca14b2 Merge branch 'main' into fix/gemini-imagen-model-name-validation 2025-08-27 21:52:47 +02:00
Sam XieandGitHub ee13c65701 [Bug]: /responses endpoint proxy ignores extra_headers in GitHub Copilot (#13775) 
* fix: pass extra_headers parameter through responses API transformation chain

Ensure extra_headers parameter is properly forwarded from the responses() function
through the transformation handler and config to maintain header propagation in
litellm_completion_request dict.

* Add tests
 2025-08-27 12:35:09 -07:00
Ishaan JaffandGitHub 4fff05f1cc [Feature]: Support Gemini requests with only system prompt (#14010) 
* _default_user_message_when_system_message_passed

* test_system_prompt_only_adds_blank_user_message

* test_system_message_with_no_user_message
 2025-08-27 12:12:47 -07:00
Ishaan JaffandGitHub f35ce02475 [Bug Fix] LLM Translation - Allow using dynamic api_key for image generation requests (#14007) 
* fix - allow using dyanmic api key for img gen

* test_aiml_image_generation_with_dynamic_api_key
 2025-08-27 10:51:24 -07:00
Ishaan Jaff ed52b67fcf test_image_edit_array_handling 2025-08-27 09:51:22 -07:00
Ishaan Jaff d5440f9614 test_image_edit_array_handling 2025-08-27 08:27:57 -07:00
Ishaan Jaff 76f78f7b2a fix: LITELLM_LOG_FILE test 2025-08-27 08:25:24 -07:00
Ifta Khairul Alam Adil 8808f78ba6 Merge branch 'main' into fix/gemini-imagen-model-name-validation 2025-08-27 15:45:59 +02:00
Krish DholakiaandGitHub 900bd10905 Merge branch 'main' into feature/enhance_logging_for_containers 2025-08-26 23:21:15 -07:00
Krish DholakiaandGitHub 71af7bcf9c Merge pull request #13374 from blahgeek/fix-count-special-token 
Fix token_counter with special token input
 2025-08-26 23:19:26 -07:00
Krish DholakiaandGitHub 45f465d84f Merge pull request #13572 from Noma-Security/noma-security-guardrails-support 
Add Noma Security guardrail support
 2025-08-26 23:15:17 -07:00
Ifta Khairul Alam Adil 0f5b31fd78 fix: resolve invalid model name error for Gemini Imagen models (#13851) 
- Fix URL construction in Gemini image generation to strip 'gemini/' prefix
- Google AI API expects base model name without the prefix
- Update model references and pricing information for consistency
- Remove outdated image generation pricing entries

Fixes issue where models like 'gemini/imagen-4.0-fast-generate-preview-06-06'
were being rejected by the Google AI API due to incorrect URL formatting.
 2025-08-27 04:17:35 +02:00
Ishaan JaffandGitHub ab7efaa832 test_pre_process_non_default_params (#13990) 2025-08-26 19:15:17 -07:00
Cole FrenchandGitHub 6696955506 [Bug]: Fix tests to reference moved attributes in braintrust_logging module (#13978) 
* Mock instance attribute moved from globals

* Support looking up mock message on mock choice
 2025-08-26 18:58:16 -07:00
Krish DholakiaandGitHub ae678a6642 Merge pull request #13573 from Ivy-Interactive/feature/braintrust-span-name-metadata 
Feature/braintrust span name metadata
 2025-08-25 23:19:56 -07:00
Krish DholakiaandGitHub 3ed1220f75 Merge pull request #13916 from mubashir1osmani/openai-multiple-images 
feat: multiple images in openai images/edits endpoint
 2025-08-25 23:11:13 -07:00
Krish DholakiaandGitHub 22934907ba Merge pull request #13933 from michal-otmianowski/ignore-expired-s3-cache-entries 
Verify if cache entry has expired prior to serving it to client
 2025-08-25 23:10:48 -07:00
Krish DholakiaandGitHub b1bc8e3aa2 Merge pull request #13955 from ckoehler/push-oymwkvxmkqxv 
feat: add image headers for Copilot
 2025-08-25 23:04:24 -07:00
Ishaan JaffandGitHub 6ce1d82970 [Bug] Fix: Vertex Mistral not working for streaming (#13952) 
* fix OpenAI like chat handler

* fix MockResponse

* test_partner_models_httpx_streaming

* test_partner_models_httpx_streaming
 2025-08-25 17:39:40 -07:00
Christoph Koehler 04cca1e7f3 feat: add image headers for Copilot 
Fixes #13696. See issue for details.
 2025-08-25 18:15:26 -06:00
Ishaan JaffandGitHub c7b0c57b1e [Bug Fix] Azure Passthrough request with streaming (#13831) 
* fix: _update_stream_param_based_on_request_body

* test_update_stream_param_based_on_request_body

* test_pass_through_request_stream_param_override
 2025-08-25 14:11:17 -07:00
Ishaan JaffandGitHub 433d1a4947 [Bug fix] - Fix /messages fallback from Anthropic API -> Bedrock API (#13946) 
* use helper get_provider_specific_headers

* fix get_provider_specific_headers

* test_anthropic_messages_fallbacks

* bedrock/us.anthropic.claude-sonnet-4

* fix: get_provider_specific_headers

* TestProviderSpecificHeaderUtils

* test_anthropic_messages_fallbacks
 2025-08-25 13:44:54 -07:00
0x5751 8dd04c4eee fix: test error 2025-08-26 01:58:04 +08:00
0x5751 d6cd50dfdb feat: Add support for custom Anthropic-compatible API endpoints 
This commit adds support for custom Anthropic-compatible API endpoints
that don't follow the standard /v1/messages or /v1/complete path convention.

## Changes
- Added LITELLM_ANTHROPIC_DISABLE_URL_SUFFIX environment variable
- When set to true, prevents automatic appending of /v1/messages (for anthropic)
- When set to true, prevents automatic appending of /v1/complete (for anthropic_text)
- Added debug logging to indicate when suffix is being skipped
- Maintained full backward compatibility - existing deployments are unaffected
 2025-08-26 01:22:51 +08:00
Michal Otmianowski c444263e7d verify expires field prior to serving cache entry 2025-08-25 10:42:12 +02:00
Steven Anton d95578464e Add support for AWS assume_role with a session token 2025-08-23 22:37:21 -07:00
Ishaan Jaff e0e356e328 test mapped test fixes 2025-08-23 17:04:23 -07:00
mubashir1osmani 2fa8f971e0 feat: multiple images in openai images/edits endpoint 2025-08-23 19:13:32 -04:00
Ishaan Jaff 0fccd619ea test_vertex_ai_deepseek 2025-08-23 14:13:03 -07:00
Ishaan Jaff cf64033044 test fix 2025-08-23 14:04:41 -07:00
Ishaan Jaff f1c710a608 test_e2e_generate_cold_storage_object_key_not_configured 2025-08-23 13:33:33 -07:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
 2025-08-23 13:13:23 -07:00