tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-20 10:18:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
29,804 Commits 2,072 Branches 1,425 Tags
ebf09218d5cecdf139bb62ecfe776141f26f883f
Commit Graph
97 Commits
Author SHA1 Message Date
Ishaan Jaffer ebf09218d5 TestManusResponsesAPITest 2026-01-08 18:59:46 +05:30
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 		b482d336b3 [Feat] New provider - Manus API on /responses, GET /responses (#18804) 
* init ManusResponsesAPIConfig

* init MANUS ApI

* init MANUS create responses

* init MANUS

* test_extract_agent_profile

* transform_get_response_api_request

* test fix

* fixes non stream

* fix streaming

* add MANUSConfig

* test_multiturn_responses_api

* code QA check

* add manus

* Potential fix for code scanning alert no. 3961: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2026-01-08 18:37:42 +05:30
Ishaan JaffandGitHub 76eda472be [Feat] New API Endpoint - Responses API (v1/responses/compact) (#18697) 
* init transform_compact_response_api_request

* init acompact_responses

* init async_compact_response_api_handler in llm http handler

* init transform_compact_response_api_request for openai

* init acompact_responses

* fix acompact_responses

* add OAI Compact API

* docs responses API Compact

* code qa checks

* test_openai_compact_responses_api

* fix mypy linting
 2026-01-06 16:24:04 +05:30
LingXuanYin 499fc9f48a Align responses API streaming hooks with chat pipeline 2026-01-06 10:54:19 +08:00
Alexsander HamirandGitHub 5a76aada3c [Fix] CI/CD - responses_api_testing (#18295) 2025-12-20 10:27:58 -08:00
Alexsander HamirandGitHub 0c48826cdc Revert "[Fix] CI/CD - local_testing & mapped tests (#18222)" (#18223) 
This reverts commit dc7f500c47.
 2025-12-18 14:46:09 -08:00
Alexsander HamirandGitHub dc7f500c47 [Fix] CI/CD - local_testing & mapped tests (#18222) 2025-12-18 14:34:48 -08:00
Alexsander HamirandGitHub 5230e97448 [Fix] CI/CD - Add missing documentation for ENVs & Clean Functionality (#18211) 2025-12-18 13:16:15 -08:00
Sameer Kankute 85907cebb5 Add custom headers in responses API 2025-12-16 11:31:15 +05:30
Ishaan Jaffer 50328d15d4 test_process_chunk_with_response_completed_event 2025-11-26 11:52:05 -08:00
Sameer KankuteandGitHub 82dc0354ce Litellm sameer nov 3 stable branch (#16963) 
* Add openai metadata filed in the request

* Add docs related to openai metadata

* Add utils

* test_completion_openai_metadata[True]

* Added support for though signature for gemini 3 in responses api (#16872)

* Added support for though signature for gemini 3

* Update docs with all supported endpoints and cost tracking

* Added config based routing support for batches and files

* fix lint errors

* Litellm anthropic image url support (#16868)

* Add image as url support to anthropic

* fix mypy errors

* fix tests

* Fix: Populate spend_logs_metadata in batch and files endpoints (#16921)

* Add spend-logs-metadata to the metadata

* Add tests for spend logs metadata in batches

* use better names

* Remove support for penalty param for gemini 3 (#16907)

* Remove support for penalty param

* remove halucinated model names

* fix mypy/test errors

* fix tests

* fix too many lines error

* fix too many lines error

* Add config for cicd test case

* Fix final tests

* fix batch tests

* fix batch tests
 2025-11-22 09:35:05 -08:00
Ishaan Jaffer 1d2bdaebb6 test_openai_streaming_logging 2025-11-08 11:49:36 -08:00
Ishaan Jaffer 4fb521c251 test_basic_openai_responses_api_non_streaming_with_logging 2025-11-08 11:12:35 -08:00
Ishaan Jaffer bbcdf6f996 test_basic_openai_responses_api_non_streaming_with_logging 2025-11-08 10:47:48 -08:00
Ishaan Jaffer 0148b8d2f7 test_basic_openai_responses_api_non_streaming_with_logging 2025-11-08 10:36:47 -08:00
Ishaan Jaffer 89157f4b5c test_basic_openai_responses_api_streaming_with_logging 2025-11-08 10:30:41 -08:00
Ishaan Jaffer 6bb963dca2 test_basic_openai_responses_api_non_streaming_with_logging 2025-11-08 10:13:27 -08:00
Ishaan Jaffer f9d95b71bb fix _get_assembled_streaming_response 2025-11-08 10:08:03 -08:00
Ishaan Jaffer 61cf169926 test_basic_openai_responses_api_streaming 2025-11-06 16:18:30 -08:00
Cesar GarciaandGitHub 1fec48499f fix: Pass extra_body to provider in Responses API requests (#16320) 
## Problem
The `extra_body` parameter in `litellm.responses()` and `litellm.aresponses()`
was being accepted but never passed to the HTTP request sent to the LLM provider.
This prevented users from sending custom/experimental parameters to provider APIs.

## Changes
- Added `data.update(extra_body)` in `async_response_api_handler` (line 2138)
- Added `data.update(extra_body)` in `response_api_handler` (line 2012)
- Added tests to `test_openai_responses_api.py` for extra_body functionality

## Testing
- Tests verify extra_body params are passed in both sync and async modes
- Existing Responses API tests continue to pass
- Manually verified with OpenAI API that custom params are sent correctly

## Impact
Users can now pass custom/experimental parameters via extra_body:
```python
litellm.aresponses(
    model="gpt-4o",
    input="hello",
    extra_body={"custom_param": "value"}  # Now works!
)
```

This aligns with the OpenAI SDK pattern and matches behavior in other
LiteLLM endpoints (completion, embedding, etc.) that already support extra_body.
 2025-11-06 14:54:40 -08:00
Ishaan Jaffer 4a83ae0695 test_aresponses_service_tier_and_safety_identifier 2025-11-04 18:05:30 -08:00
Cesar GarciaandGitHub 78ed5126a5 fix: Fix Responses API streaming tests usage field names and cost (#16236) 
This commit fixes two bugs in Responses API streaming tests:

1. **Usage field naming bug**: Tests were using `input_tokens` and
   `output_tokens` but the Usage object uses `prompt_tokens` and
   `completion_tokens`.

2. **Missing cost in streaming usage**: When `include_cost_in_streaming_usage`
   was enabled, the cost was calculated and added to ResponseAPIUsage, but was
   lost during the transformation to the Usage object.

Changes:
- Updated test assertions to use correct field names (prompt_tokens, completion_tokens)
- Added cost preservation logic in FakeStreamerResponsesAPIIterator
- Modified _transform_response_api_usage_to_chat_usage() to preserve cost attribute

All streaming tests now pass successfully.
 2025-11-04 15:57:59 -08:00
Krish DholakiaandGitHub 74ae7aed44 build: Squashed commit of the following: (#16176) 
commit bb0b050fb01633d83c1c2932f8e9c11432911847
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Sat Nov 1 20:00:01 2025 -0700

    test: update tests

commit b2da4bdac23868e69a9452805b231f8830e49912
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:58:01 2025 -0700

    fix(langfuse_otel_attributes.py): log tools and other optional params

commit 75bee1f2748f32b230467de0b085c55bf1d687a9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:42:05 2025 -0700

    feat(langfuse_otel/): working request/response logging on spans

    Closes https://github.com/BerriAI/litellm/issues/13764

commit a3e4fa5b81e82f71c74fb9e7dc859c6cb40495f5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 14:20:39 2025 -0700

    fix: initial commit fixing langfuse request/response logging with OTEL

commit 09fc9deac844004104822810e42975cd9c68f0e3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Oct 22 13:33:52 2025 -0700

    fix(litellm_logging.py): for responses api - return a unified usage object for logging

    ensures logging integrations all pull the right usage information
 2025-11-02 09:46:40 -08:00
Sameer KankuteandGitHub f804ab6de5 Add LLM provider response headers to Responses API (#16091) 
* Add llm headers to responses api

* fix mock test
 2025-11-01 13:25:56 -07:00
Ishaan Jaffer 33371d18f4 test fix claude-sonnet-4-5-20250929 2025-10-28 19:05:13 -07:00
Ishaan Jaffer 1b49dba1dd fix claude-sonnet-4-5 2025-10-28 17:37:08 -07:00
Ishaan Jaffer a1d3790198 TestAzureResponsesAPITest 2025-10-25 16:22:52 -07:00
Ishaan Jaffer f0ae2bef4f TestAzureResponsesAPITest 2025-10-25 16:09:04 -07:00
Ishaan Jaffer 667f2613de TestAzureOpenAIVectorStore 2025-10-25 14:06:28 -07:00
Ishaan Jaffer eb425550a2 test responses API fixes 2025-10-25 10:58:39 -07:00
Ishaan JaffandGitHub d91efa7a7b [Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure (#15804) 
* add ErrorEventError nested field

* test_openai_responses_api_token_limit_error

* test_openai_responses_api_token_limit_error
 2025-10-22 14:18:46 -07:00
Sameer KankuteandGitHub 44495c0117 fix encrypted content error (#15782) 2025-10-21 23:29:48 -07:00
Ishaan Jaffer 97626f2d02 fix test 2025-10-11 11:30:19 -07:00
Ishaan Jaffer ce6102d54f test_azure_responses_api_status_error 2025-10-11 10:41:15 -07:00
Ishaan Jaffer 86881a8fc1 test_azure_responses_api_status_error 2025-10-11 10:21:07 -07:00
Krrish Dholakia 5336fcc000 fix(azure/responses): always remove status 
unsupported parameter
 2025-10-06 18:08:57 -07:00
Ishaan JaffandGitHub 708c0bd78d [Feat] Return Cost for Responses API Streaming requests (#15053) 
* test_basic_openai_responses_api_streaming

* _transform_chat_completion_usage_to_responses_usage

* ResponseAPIUsage.cost

* test fixes for anthropic cost with /responses

* fix mypy typng
 2025-09-29 19:47:04 -07:00
Alexsander HamirandGitHub eaa04cd8ce fix: use fastuuid helper (#14903) 
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
 2025-09-25 15:47:01 -07:00
Ishaan JaffandGitHub 8e22cf5d65 [Fix] /responses API - add cancel endpoint + allow non-admins to use this as an llm api endpoint (#14594) 
* fix: ensure /responses/cancel works for non admins

* test: cancel endpoint

* fix responses API  cancel endpoint

* test fix

* TestGoogleAIStudioResponsesAPITest
 2025-09-15 18:49:54 -07:00
Sameer KankuteandGitHub 110ce543c2 [Feat]Add cancel endpoint support for openai and azure (#14561) 
* Add cancel endpoint support for openai
 and azure

* fix lint error

* fix cancel url contruction azure

* readd changes
 2025-09-15 07:08:56 -07:00
Sameer Kankute ad9f54a192 Move test to test_litellm/ folder 2025-09-05 10:08:34 +05:30
Krish DholakiaandGitHub 94c1b21ae7 Merge branch 'main' into litellm_responses_structured_output 2025-09-04 12:23:26 -07:00
Sameer KankuteandGitHub fc9560573b [BUG] Fix response api for reasoning item in input for litellm proxy (#14200) 
* fix response api for litellm proxy

* Add test for checking if status is getting removed

* add test in correct file

* remove hardcoded fields

* Make the handling simpler

* fix lint error:
 2025-09-04 10:36:48 -07:00
Sameer KankuteandGitHub 8d67392e99 Merge branch 'main' into litellm_responses_structured_output 2025-09-04 22:35:30 +05:30
Sameer KankuteandGitHub 5f79e8aac6 Litellm passthrough cost tracking chat completion (#14256) 
* feat: add structured output for sdk

* Add support for cost tracking for chat completion in passthrough

* remove not required changes
 2025-09-04 09:57:48 -07:00
Sameer Kankute 8e363fe78c feat: add structured output for sdk 2025-09-03 17:58:55 +05:30
Ishaan JaffandGitHub 37d885e46a [Fix] LiteLLM does not support new web_search tool (Responses API) (#14083) 
* test_basic_openai_responses_with_websearch

* fix: ResponsesAPIResponse

* fix: StandardBuiltInToolCostTracking
 2025-08-29 18:33:52 -07:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
 2025-08-23 13:13:23 -07:00
Krrish Dholakia cbb161f10b test: handle internal server errors 2025-08-23 11:00:18 -07:00
Ishaan JaffandGitHub 76f1064229 [Bug Fix] litellm incompatible with newest release of openAI v1.100.0 (#13728) 
* fix imports OpenAI SDK

* ResponseText fixes

* fixes ResponseText

* fix imports

* catch AttributeError

* fix import

* use openai==1.100.1

* fix build from PIP

* fix lint test

* Print OpenAI version

* fix Install dependencies
 2025-08-18 18:26:17 -07:00