tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-05 08:23:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
24,346 Commits 2,072 Branches 1,425 Tags
6cd5afa8b11ea3654a2c26d3d64cd2a2e3eff3f9
Commit Graph
24346 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
mubashir1osmani (aider) 6cd5afa8b1 feat: add persistence and management for guardrails on virtual keys 2025-09-01 00:56:22 -04:00
Ishaan JaffandGitHub 04dc1a5351 [Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
 2025-08-27 16:16:19 -07:00
Xingyao WangGitHubopenhands
 		9acf80b8ad Fix issue #13995: Handle None metadata in batch requests (#13996) 
* Fix issue #13995: Handle None metadata in batch requests

- Added null check in add_key_level_controls method to prevent NoneType error
- Updated type hint to Optional[dict] for better type safety
- Added comprehensive test suite to verify the fix works correctly
- All existing tests pass, confirming no regression

Fixes #13995

Co-authored-by: openhands <openhands@all-hands.dev>

* Move test file to tests/test_litellm/proxy/ directory

- Moved test_batch_metadata_none_fix.py from tests/ to tests/test_litellm/proxy/
- Updated import structure to match existing test patterns
- This ensures the test runs in GitHub Actions as requested by @krrishdholakia

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
 2025-08-27 14:51:09 -07:00
Ishaan JaffandGitHub 165242e31f [Feat] langfuse_otel logger - allow using LANGFUSE_OTEL_HOST for configuring host (#14013) 
* feat - add _get_langfuse_otel_host

* test_get_langfuse_otel_config_with_otel_host_priority

* docs: LANGFUSE_OTEL_HOST
 2025-08-27 14:43:47 -07:00
Sam XieandGitHub ee13c65701 [Bug]: /responses endpoint proxy ignores extra_headers in GitHub Copilot (#13775) 
* fix: pass extra_headers parameter through responses API transformation chain

Ensure extra_headers parameter is properly forwarded from the responses() function
through the transformation handler and config to maintain header propagation in
litellm_completion_request dict.

* Add tests
 2025-08-27 12:35:09 -07:00
Ishaan JaffandGitHub 4fff05f1cc [Feature]: Support Gemini requests with only system prompt (#14010) 
* _default_user_message_when_system_message_passed

* test_system_prompt_only_adds_blank_user_message

* test_system_message_with_no_user_message
 2025-08-27 12:12:47 -07:00
Ishaan JaffandGitHub f35ce02475 [Bug Fix] LLM Translation - Allow using dynamic api_key for image generation requests (#14007) 
* fix - allow using dyanmic api key for img gen

* test_aiml_image_generation_with_dynamic_api_key
 2025-08-27 10:51:24 -07:00
Ishaan Jaff ed52b67fcf test_image_edit_array_handling 2025-08-27 09:51:22 -07:00
Ishaan Jaff d5440f9614 test_image_edit_array_handling 2025-08-27 08:27:57 -07:00
Ishaan Jaff 76f78f7b2a fix: LITELLM_LOG_FILE test 2025-08-27 08:25:24 -07:00
Krish DholakiaandGitHub 539b94ad4e Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers 
Enhance logging for containers to log on files both with usual format and json format
 2025-08-26 23:21:34 -07:00
Krish DholakiaandGitHub 900bd10905 Merge branch 'main' into feature/enhance_logging_for_containers 2025-08-26 23:21:15 -07:00
Krish DholakiaandGitHub 71af7bcf9c Merge pull request #13374 from blahgeek/fix-count-special-token 
Fix token_counter with special token input
 2025-08-26 23:19:26 -07:00
Krish DholakiaandGitHub ddec6c69c8 Merge pull request #13534 from cometapi-dev/main 
docs: Add CometAPI documentation with authentication, usage examples, and error handling
 2025-08-26 23:17:04 -07:00
Krish DholakiaandGitHub 4613f7821d Merge pull request #13536 from edwardsamuel/feat/openrouter-gpt-5-pricing 
Add openrouter gpt-5 family models pricing
 2025-08-26 23:16:53 -07:00
Krish DholakiaandGitHub 45f465d84f Merge pull request #13572 from Noma-Security/noma-security-guardrails-support 
Add Noma Security guardrail support
 2025-08-26 23:15:17 -07:00
Krish DholakiaandGitHub 201d96b2d2 Merge pull request #13917 from Toy-97/patch-1 
Deepinfra Metadata Update 24082025
 2025-08-26 23:13:46 -07:00
Krish DholakiaandGitHub 28f7df8da8 Merge pull request #13965 from manascb1344/feat/nebius-models-update 
feat(constants): expand Nebius provider models and normalize model IDs
 2025-08-26 23:13:06 -07:00
Krish DholakiaandGitHub 682fb697d0 Merge pull request #13969 from dttran-glo/patch-1 
bump `orjson` version to "3.11.2"
 2025-08-26 23:09:57 -07:00
Ishaan Jaff ee324943d7 bump: version 1.76.0 → 1.76.1 2025-08-26 20:02:57 -07:00
Ishaan Jaff c839b2c751 installing_litellm_on_python 2025-08-26 19:51:02 -07:00
Ishaan JaffandGitHub 80d4cc1283 [Perf] Use fastuuid for fast UUID generations - 2.1x Faster (#13992) 
* use fastuuid

* add fastuuid

* add fastuuid==0.12.0
 2025-08-26 19:47:25 -07:00
Ishaan JaffandGitHub ab7efaa832 test_pre_process_non_default_params (#13990) 2025-08-26 19:15:17 -07:00
Cole FrenchandGitHub 6696955506 [Bug]: Fix tests to reference moved attributes in braintrust_logging module (#13978) 
* Mock instance attribute moved from globals

* Support looking up mock message on mock choice
 2025-08-26 18:58:16 -07:00
codeflash-ai[bot]GitHubcodeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
 		5dba5822f2 ⚡️ Speed up function _is_debugging_on by 45% (#13988) 
The optimization eliminates unnecessary conditional branching by replacing the explicit `if-else` structure with a direct return of the boolean expression. Instead of evaluating the condition and then branching to return `True` or `False`, the optimized version directly returns the result of the boolean expression `verbose_logger.isEnabledFor(logging.DEBUG) or set_verbose is True`.

This change removes Python's conditional jump overhead and reduces the number of executed bytecode instructions per function call. The line profiler shows the original version required 3 lines of execution (condition check, conditional return True, fallback return False) while the optimized version executes only 1 line.

The 45% speedup is achieved by:
- **Eliminating branching overhead**: No conditional jumps needed
- **Reducing bytecode instructions**: From ~3 instructions to 1 instruction per call
- **Leveraging Python's short-circuit evaluation**: The `or` operator still evaluates left-to-right and stops early when the first condition is True

The optimization is particularly effective for this logging utility function which is likely called frequently throughout the application. All test cases show consistent 40-75% improvements across different scenarios (debug on/off, verbose flag variations, edge cases with different logging levels), demonstrating the optimization works well regardless of the boolean expression's outcome.

Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
 2025-08-26 17:25:45 -07:00
Ishaan JaffandGitHub 5647757ab3 [Feat] New model gemini-2.5-flash-image-preview (#13979) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview
 2025-08-26 15:04:02 -07:00
Duc TranandGitHub e3d947abeb bump orjson version to "3.11.2" 2025-08-26 23:21:53 +07:00
manascb1344 1150b93e8f feat(constants): expand nebius_models set with DeepSeek, Llama, Qwen, NVIDIA, ZAI, and others; normalize model IDs (remove -fast variants, add new instruct/thinking variants) 2025-08-26 18:20:29 +05:30
Krish DholakiaandGitHub 55c3fd62ac Add Claude Opus 4 and 4.1 via OpenRouter and Fix Claude Sonnet 4 and 3.7 via OpenRouter in Prices and Context Window Config 
Update model prices and context window
 2025-08-25 23:28:39 -07:00
Krish DholakiaandGitHub 3803af5ff5 Merge pull request #13566 from kankute-sameer/fix/azure-gpt4-web-search-support 
fix: remove incorrect web search support for azure/gpt-4.1 family
 2025-08-25 23:28:15 -07:00
Krish DholakiaandGitHub ae678a6642 Merge pull request #13573 from Ivy-Interactive/feature/braintrust-span-name-metadata 
Feature/braintrust span name metadata
 2025-08-25 23:19:56 -07:00
Krish DholakiaandGitHub 3ed1220f75 Merge pull request #13916 from mubashir1osmani/openai-multiple-images 
feat: multiple images in openai images/edits endpoint
 2025-08-25 23:11:13 -07:00
Krish DholakiaandGitHub 22934907ba Merge pull request #13933 from michal-otmianowski/ignore-expired-s3-cache-entries 
Verify if cache entry has expired prior to serving it to client
 2025-08-25 23:10:48 -07:00
Krish DholakiaandGitHub b1bc8e3aa2 Merge pull request #13955 from ckoehler/push-oymwkvxmkqxv 
feat: add image headers for Copilot
 2025-08-25 23:04:24 -07:00
Krish DholakiaandGitHub 51b491e7a1 Merge pull request #13958 from TeddyAmkie/pricing/deepseek-v3p1 
Add DeepSeek-v3.1 pricing for Fireworks AI provider
 2025-08-25 23:03:39 -07:00
Teddy Amkie 97e9502f4a Add DeepSeek-v3.1 pricing for Fireworks AI provider 
- Add fireworks_ai/accounts/fireworks/models/deepseek-v3p1 model configuration
- Set context window: 128K input, 8K output tokens
- Pricing: /bin/zsh.56/1M input tokens, .68/1M output tokens
- Supports response schema and tool choice
- Based on DeepSeek API unified pricing effective Sept 2025
 2025-08-25 18:41:49 -07:00
Ishaan JaffandGitHub 6ce1d82970 [Bug] Fix: Vertex Mistral not working for streaming (#13952) 
* fix OpenAI like chat handler

* fix MockResponse

* test_partner_models_httpx_streaming

* test_partner_models_httpx_streaming
 2025-08-25 17:39:40 -07:00
Christoph Koehler 04cca1e7f3 feat: add image headers for Copilot 
Fixes #13696. See issue for details.
 2025-08-25 18:15:26 -06:00
Ishaan JaffandGitHub c7b0c57b1e [Bug Fix] Azure Passthrough request with streaming (#13831) 
* fix: _update_stream_param_based_on_request_body

* test_update_stream_param_based_on_request_body

* test_pass_through_request_stream_param_override
 2025-08-25 14:11:17 -07:00
Ishaan JaffandGitHub 433d1a4947 [Bug fix] - Fix /messages fallback from Anthropic API -> Bedrock API (#13946) 
* use helper get_provider_specific_headers

* fix get_provider_specific_headers

* test_anthropic_messages_fallbacks

* bedrock/us.anthropic.claude-sonnet-4

* fix: get_provider_specific_headers

* TestProviderSpecificHeaderUtils

* test_anthropic_messages_fallbacks
 2025-08-25 13:44:54 -07:00
Ishaan JaffandGitHub c1ee8c26af [UI QA] - Allow setting Team Member RPM/TPM limits when creating a team (#13943) 
* allow setting team_member_rpm_limit on creating

* create_team_member_rate_limits

* docs fix

* fix img
 2025-08-25 09:12:50 -07:00
Ishaan Jaff 7c5e9b666b update release notes 2025-08-25 08:17:30 -07:00
Michal Otmianowski 3b6462236c clean imports 2025-08-25 11:09:06 +02:00
Michal Otmianowski c444263e7d verify expires field prior to serving cache entry 2025-08-25 10:42:12 +02:00
Krish DholakiaandGitHub 68aaeb4f4a Merge pull request #13914 from uc4w6c/docs/fix-custom-guardrail-imports 
Fix missing and unused imports in custom_guardrail docs example
 2025-08-25 00:57:46 -07:00
Krish DholakiaandGitHub 0d285c8b4c Merge pull request #13919 from stevenmanton/fix-aws-assume-role-with-token 
Add support for AWS assume_role with a session token
 2025-08-25 00:55:02 -07:00
Krish DholakiaandGitHub a7bb5bc814 Merge pull request #13922 from ArthurRenault/patch-1 
[BUG] Add back supervisor to non-root image
 2025-08-25 00:54:18 -07:00
ArthurandGitHub 6c97a31c9c bug: add supervisor to non-root image 2025-08-24 15:43:57 +02:00
Steven Anton d95578464e Add support for AWS assume_role with a session token 2025-08-23 22:37:21 -07:00
Toy-97andGitHub f2bb1ce31e Deepinfra Metadata Update 24082025 
Deepseek v3.1 Price Dropped
 2025-08-24 08:30:25 +08:00