tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-05 20:23:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,117 Commits 2,072 Branches 1,425 Tags
6721189b7cbbc95ecbbf33c1de7cacbb60853d02
Commit Graph
3188 Commits
Author SHA1 Message Date
Michael Tan 6721189b7c feat: add Asia/Bangkok timezone support for budget resets 
- Add Asia/Bangkok (UTC+7) to timezone_map in duration_parser.py
- Update documentation to include Bangkok in common timezone values
- Add test case to verify Bangkok timezone functionality
 2025-09-18 14:56:12 +07:00
Krish DholakiaandGitHub bfaab8ad7e Merge pull request #14557 from timelfrink/fix/issue-14478-bedrock-count-tokens-endpoint 
Implement AWS Bedrock CountTokens API support
 2025-09-17 23:51:06 -07:00
Krish DholakiaandGitHub ff36dfdc76 Merge pull request #14637 from akraines/feature/middle-truncate-spend-logs 
feat: implement middle-truncation for spend log payloads
 2025-09-17 23:47:04 -07:00
Krish DholakiaandGitHub e7bc700b20 Merge pull request #14640 from Sameerlite/litellm_gardrail_default_latest_message 
[Feat]Add last message as default in gaurdrail
 2025-09-17 23:42:40 -07:00
Tim Elfrink c234b13275 Apply code formatting and linting fixes 
- Apply Black formatting to all Bedrock CountTokens files
- Clean up imports and remove unused variables in tests
- Fix indentation and simplify test structure
- Fix pyright type error with type ignore annotation
- All tests continue to pass after cleanup
 2025-09-18 08:28:17 +02:00
Tim Elfrink e74ac35b5d Add comprehensive tests for Bedrock CountTokens functionality 
- Add endpoint integration test in test_proxy_token_counter.py
- Add unit tests for transformation logic in bedrock/count_tokens/
- Test model extraction from request body vs endpoint path
- Test input format detection (converse vs invokeModel)
- Test request transformation from Anthropic to Bedrock format
- All tests follow existing codebase patterns and pass successfully
 2025-09-18 08:16:56 +02:00
Sameerlite edf95966c9 Handle consecutive user messages 2025-09-18 10:35:25 +05:30
Alexsander HamirandGitHub 8d96626044 fix: iscoroutine removed from hot path +50 RPS (#14649) 
* fix: iscoroutine removed from hot path

* fix: replace all instances & separate concerns

1. Replaced all instances of iscoroutine with is_async_callable
2. Place the coroutine checker in its own file

* fix: PR comment changes

* fix: missing config setting declaration

* fix: revert non-performance related changes

* fix: revert to initial implementation

* fix: remove dead const
 2025-09-17 19:01:43 -07:00
Krrish Dholakia 00d8dedd16 test: update test 2025-09-17 17:44:14 -07:00
Krish DholakiaandGitHub 44e0c730b9 Merge pull request #14653 from ARajan1084/in-memory-guardrail-fix 
fix: In Memory Guardrail fails to update
 2025-09-17 17:42:23 -07:00
= 268b984cb6 added test for updating in memory guardrails 2025-09-17 17:16:49 -07:00
Mubashir OsmaniGitHubIshaan Jaff
 		8b804303ed fix: ci/cd tests + lint errors (#14646) 
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-09-17 17:06:43 -07:00
Krish DholakiaandGitHub f51003538a Merge pull request #14650 from BerriAI/litellm_dev_09_17_2025_p1 
Bedrock Guardrails - support setting bedrock runtime endpoint + Protect `/health/test_connect` to prevent users without model creation permissions from calling it
 2025-09-17 16:55:53 -07:00
Krrish Dholakia 1954bbffc0 test: fix test on ci/cd 2025-09-17 16:53:07 -07:00
Krish DholakiaandGitHub 895c41efa3 Merge pull request #14619 from BerriAI/litellm_dev_09_16_2025_p1 
UI - allow team member to view service account keys they create + Anthropic - include cache creation tokens in prompt token total (separate out during cost tracking)
 2025-09-17 15:43:04 -07:00
Krrish Dholakia 530662d735 test: add unit testing 2025-09-17 14:54:44 -07:00
Sameer KankuteandGitHub 69c01488bd remove not needed names (#14641) 2025-09-17 14:26:48 -07:00
Krrish Dholakia 08ba38adeb test: update unit test 2025-09-17 14:23:13 -07:00
Sameerlite 1371abf880 add last message as default in gaurdrail 2025-09-17 21:08:19 +05:30
Carlos MarchalandGitHub e168161e64 Feat/add posthog observability (#14610) 
* feat: add posthog observability

* docs: add posthog logging docs

* docs: posthog integration in proxy mode
 2025-09-17 08:24:04 -07:00
Akiva Kraines 115a3e9ded feat: implement middle-truncation for spend log payloads 
- Change truncation strategy from head-only to middle-truncation (35% start, 65% end)
- Preserve both beginning and end of long strings for better debugging context
- Apply same sanitization to response payloads when store_prompts_in_spend_logs is enabled
- Increase default MAX_STRING_LENGTH_PROMPT_IN_DB from 1000 to 2048 characters
- Update tests to verify new truncation behavior with 35%-65% split

This provides better diagnostic value by keeping the more important end context
while still maintaining storage limits.
 2025-09-17 16:30:25 +03:00
Krish DholakiaandGitHub bf0dd4a284 Merge pull request #14418 from iabhi4/deep-copy-issue 
fix: avoid deepcopy crash with non-pickleables in Gemini/Vertex
 2025-09-16 22:55:31 -07:00
Krish DholakiaandGitHub 2563c0ab2d Merge pull request #14534 from hula-la/fix/response-api-bugfixes 
fix: improve response api handling and cold storage configuration
 2025-09-16 22:46:03 -07:00
Krish DholakiaandGitHub fcf84027e8 Merge pull request #14555 from mubashir1osmani/dd_spend_metric 
DataDog shows spend metrics
 2025-09-16 22:44:47 -07:00
Krish DholakiaandGitHub 635dc72211 Merge pull request #14604 from Sameerlite/litellm_gemini_api_base_update 
Litellm gemini api base update
 2025-09-16 22:38:44 -07:00
Krish DholakiaandGitHub 6bccdcd530 Merge pull request #14620 from BerriAI/litellm_dev_09_16_2025_p2 
(feat) Anthropic - document pricing for cache creation tokens above 1hr
 2025-09-16 22:33:11 -07:00
Krrish Dholakia 8273630e08 test: update tests 2025-09-16 22:30:40 -07:00
Mubashir OsmaniandGitHub 0fde408e35 added langfuse logging for responses api (#14597) 
* added langfuse logging for responses api

* tests added
 2025-09-16 21:43:19 -07:00
mubashir1osmani e56b11cb54 added tests 2025-09-17 00:19:03 -04:00
Krrish Dholakia 0e747aaaf1 test: fix test 2025-09-16 19:20:12 -07:00
Ishaan JaffandGitHub 93524cf19b [Feat] Batches - Add bedrock retrieve endpoint support (#14618) 
* feat: add bedrock retrieve endpoint

* feat: feat: add bedrock retrieve endpoint

* test: batches mocked transform

* ruff fix

* refactor

* fix transform

* fix: parse_timestamp
 2025-09-16 19:19:02 -07:00
Krrish Dholakia 22d3e492f9 fix(test_utils.py): fix test 2025-09-16 19:17:44 -07:00
Krrish Dholakia e735808614 test(test_utils.py): fix cache creation tokens 2025-09-16 19:03:16 -07:00
Krrish Dholakia 1c855385c9 build(model_cost): add cache_creation_input_token_cost_above_1hr pricing 2025-09-16 18:43:57 -07:00
Krrish Dholakia 0341e7fc09 fix: fix test 2025-09-16 18:34:24 -07:00
Krrish Dholakia e488312873 fix(utils.py): log cache_creation_tokens in prompt token details 
Closes LIT-907
 2025-09-16 18:24:10 -07:00
Krrish Dholakia 1162c52692 fix(anthropic/chat/transformation.py): include cache creation token count in final prompt token value 
Closes LIT-907
 2025-09-16 17:52:12 -07:00
Krrish Dholakia 32f634e305 test: add unit testing 2025-09-16 16:54:03 -07:00
Sameer KankuteandGitHub ab1fb2b2e7 Add Support for Bedrock Guardrails to supportive selective Guarding (#14575) 
* Add Support for Bedrock Guardrails to supportive selective Guarding

* Add method for better handling

* Add guarded_text content type

* Add guarded_text content type

* Update Dockerfile

* Update Dockerfile
 2025-09-16 16:43:34 -07:00
Ishaan Jaffer 7de8811c4c test 2025-09-16 16:33:10 -07:00
Alexsander HamirandGitHub 02db2e8ae8 [Performance] RPS Improvement +500 RPS when sending the user field (#14616) 
* perf tool

* fix: cache type issue

* fix: exception hanging & cache setting

1. Removed unhandled exceptions
2. Set cache value to dict
 2025-09-16 16:18:23 -07:00
Ishaan Jaffer 68105ce1a7 fix type 2025-09-16 15:41:52 -07:00
Sameerlite f08fc45a0f add base url support for gemini 2025-09-16 15:15:24 +05:30
Krish DholakiaandGitHub f34bbd1f35 Merge pull request #14569 from talesofai/volcengine-fix-thinking-parameters-missing-if-not-set 
fix volcengine thinking parameters missing when it set disable
 2025-09-15 21:33:31 -07:00
Krish DholakiaandGitHub 3f90fe938d Merge pull request #14514 from iabhi4/fix-13581 
feat(proxy): Assign default budget to auto-generated JWT teams
 2025-09-15 21:22:25 -07:00
Krish DholakiaandGitHub f4e2870490 Merge pull request #14532 from timelfrink/feat/issue-14476-compactifai-provider 
Add CompactifAI provider support
 2025-09-15 21:15:00 -07:00
mubashir1osmani f5f00043ce fixed tests 2025-09-16 00:14:48 -04:00
LingXuanYin f8c9009fe5 add more test 2025-09-16 12:11:10 +08:00
Krish DholakiaandGitHub 784eb18366 Merge pull request #14558 from iabhi4/fix-14536 
fix(proxy): Correctly parse multi-part MCP server aliases from URL paths
 2025-09-15 21:05:22 -07:00
Ishaan JaffandGitHub 8e22cf5d65 [Fix] /responses API - add cancel endpoint + allow non-admins to use this as an llm api endpoint (#14594) 
* fix: ensure /responses/cancel works for non admins

* test: cancel endpoint

* fix responses API  cancel endpoint

* test fix

* TestGoogleAIStudioResponsesAPITest
 2025-09-15 18:49:54 -07:00