tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-05 14:23:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,256 Commits 2,072 Branches 1,425 Tags
dfdba527e2ed3fc51dd236cf0b5ea7c8a6528d1d
Commit Graph
54 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub 51d5255452 [Bug]: Azure OpenAI & AI Foundry Reject Image Generation Payload Due to extra_body Injection in LiteLLM v1.76.3 (#14475) 
* add request body azure img gen

* fix test_get_optional_params_image_gen_filters_empty_values

* test_azure_image_generation_request_body

* test_azure_image_generation_request_body
 2025-09-11 19:39:06 -07:00
Ishaan Jaff 87f116bded test fix 2025-09-08 18:46:20 -07:00
Krrish Dholakia 7a083750ce refactor: remove 'input_fidelity' from image gen -> it's a image_edit param 2025-09-06 12:14:27 -07:00
Ifta Khairul Alam Adil 75aeca14b2 Merge branch 'main' into fix/gemini-imagen-model-name-validation 2025-08-27 21:52:47 +02:00
Ishaan JaffandGitHub f35ce02475 [Bug Fix] LLM Translation - Allow using dynamic api_key for image generation requests (#14007) 
* fix - allow using dyanmic api key for img gen

* test_aiml_image_generation_with_dynamic_api_key
 2025-08-27 10:51:24 -07:00
Ishaan Jaff ed52b67fcf test_image_edit_array_handling 2025-08-27 09:51:22 -07:00
Ishaan Jaff d5440f9614 test_image_edit_array_handling 2025-08-27 08:27:57 -07:00
Ifta Khairul Alam Adil 0f5b31fd78 fix: resolve invalid model name error for Gemini Imagen models (#13851) 
- Fix URL construction in Gemini image generation to strip 'gemini/' prefix
- Google AI API expects base model name without the prefix
- Update model references and pricing information for consistency
- Remove outdated image generation pricing entries

Fixes issue where models like 'gemini/imagen-4.0-fast-generate-preview-06-06'
were being rejected by the Google AI API due to incorrect URL formatting.
 2025-08-27 04:17:35 +02:00
Krish DholakiaandGitHub 3ed1220f75 Merge pull request #13916 from mubashir1osmani/openai-multiple-images 
feat: multiple images in openai images/edits endpoint
 2025-08-25 23:11:13 -07:00
mubashir1osmani 2fa8f971e0 feat: multiple images in openai images/edits endpoint 2025-08-23 19:13:32 -04:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
 2025-08-23 13:13:23 -07:00
Ishaan JaffandGitHub 9750374081 [Feat] New LLM API - AI/ML API for Image Gen (#13893) 
* add LlmProviders.AIML

* add AIMLChatConfig

* add aiml

* fix AimlImageGenerationConfig

* add AimlImageGenerationConfig

* add cost_calculator

* fixes for AI ML API

* add known AIML Flux image models

* add AIML Cost Calc

* add AI/ML API

* add aiml_models
 2025-08-23 13:12:44 -07:00
Krrish Dholakia 385bb15219 test: skip test 2025-08-23 11:27:46 -07:00
Krrish Dholakia b5c4ee60bb fix(openai/image_edits): Support 'mask' parameter for openai image edits 
Closes https://github.com/BerriAI/litellm/issues/13528
 2025-08-16 01:40:24 -07:00
Ishaan JaffandGitHub 76d25926d4 [Feat] New provider - Azure AI Flux Image Generation (#13592) 
* init files

* add AzureFoundryModelInfo

* fix api_version property

* add azure_ai img gen

* use AzureFoundryModelInfo

* get_base_image_generation_call_args

* add azure_ai/FLUX-1.1-pro

* add util for route_image_generation_cost_calculator

* docs azure ai flux

* fixes for flux

* fixes for AzureFoundryFluxImageGenerationConfig

* ruff fix
 2025-08-13 17:20:30 -07:00
Ishaan JaffandGitHub a8371d2cb1 [Feat] Add Google AI Studio Imagen4 model family (#13065) 
* add gemini

* add init files

* add get_gemini_image_generation_config

* refactor transform

* TestGoogleImageGen

* fix transform

* fix transform

* add gemini_image_cost_calculator

* add cost tracking for gemini/imagen models

* docs image gen

* docs image gen

* test_get_model_info_gemini
 2025-07-28 21:25:40 -07:00
Ishaan JaffandGitHub d5ee93aa0c [Feat] Add Recraft API - Image Edits Support (#12874) 
* test_recraft_image_edit_api

* add RecraftImageEditConfig

* complete RecraftImageEditConfig

* add RecraftImageEditRequestParams in types

* update RecraftImageEditRequestParams

* working

* transform_image_edit_request

* Image Edit docs recraft

* working transform_image_edit_request

* TestRecraftImageEditTransformation
 2025-07-22 15:03:08 -07:00
Ishaan Jaff 03baf23ad1 [Feat] Add Recraft Image Generation API Support - New LLM Provider (#12832) 
* add recraft

* init RecraftImageGenerationConfig

* add get_complete_url + validate_environment

* add image_generation_handler in llm http clas

* fixes for transform

* working recraft request

* fixed img gen transform

* fixes for llm http handler

* test: TestRecraftImageGeneration

* fixes for llm_http_handler

* fix RecraftImageGenerationConfig

* TestRecraftImageGenerationTransformation

* add recraft API

* docs recraft API

* fix code QA

* map_openai_params

* fix recraft

* cost tracking for recraft/recraftv3

* fix code qa check
 2025-07-21 22:19:58 -07:00
Krish DholakiaandGitHub e4e10aa4ed Litellm dev 07 21 2025 p1 (#12848) 
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang

* test(tests/): update tests
 2025-07-21 22:09:39 -07:00
Ishaan JaffandGitHub 2941a555a8 [Feat] Add Recraft Image Generation API Support - New LLM Provider (#12832) 
* add recraft

* init RecraftImageGenerationConfig

* add get_complete_url + validate_environment

* add image_generation_handler in llm http clas

* fixes for transform

* working recraft request

* fixed img gen transform

* fixes for llm http handler

* test: TestRecraftImageGeneration

* fixes for llm_http_handler

* fix RecraftImageGenerationConfig

* TestRecraftImageGenerationTransformation

* add recraft API

* docs recraft API

* fix code QA

* map_openai_params

* fix recraft

* cost tracking for recraft/recraftv3

* fix code qa check
 2025-07-21 15:01:32 -07:00
Cole McIntoshandGitHub b2080ec9af feat: add input_fidelity parameter for OpenAI image generation (#12662) 
* feat: add input_fidelity parameter for OpenAI image generation

- Add input_fidelity to OpenAIImageGenerationOptionalParams type
- Update image_generation function signature to accept input_fidelity
- Add input_fidelity to default_params in get_optional_params_image_gen
- Include input_fidelity in openai_params list for proper handling
- Update documentation with input_fidelity parameter description
- Add test for input_fidelity parameter functionality

This enables control over how closely the model follows the input prompt
for gpt-image-1 model, improving prompt adherence and image quality.

* feat: add input_fidelity to optional parameters for image generation

- Include input_fidelity in the list of OpenAIImageGenerationOptionalParams
- This addition enhances the flexibility of image generation by allowing control over input fidelity.

* test: enhance test for gpt-image-1 with input_fidelity parameter

- Update test_gpt_image_1_with_input_fidelity to include mocking of OpenAI response
- Validate that the OpenAI client is called with correct parameters, including input_fidelity
- Improve response validation to ensure expected output structure and values
 2025-07-16 16:56:05 -07:00
Ishaan JaffandGitHub ce2934349f [Feat] Add XInference Image Generation API Provider (#12439) 
* add XInferenceImageGenerationConfig

* add get_xinference_image_generation_config

* test_xinference_image_generation

* docs Image Generation xinference

* docs inference

* docs xinference

* fix xinference img gen
 2025-07-08 21:17:38 -07:00
Ishaan JaffandGitHub a76d991952 [Fix]: Add cost tracking for image edits endpoint [OpenAI, Azure] (#11186) 
* fix: add cost tracking for image edits

* fix: add cost tracking for azure image edits

* fix: fix linting error

* fix: fixes for background param

* fix: image edit test fixes

* fix: openai image edit cost tracking
 2025-05-27 17:52:15 -07:00
Ishaan JaffandGitHub 1009defbdc [Feat] Add /image/edits support for Azure (#11160) 
* feat: add image edits on litellm router

* feat: add image edits endpoint

* fix: use pure async for image edits

* fix: base_process_llm_request

* fix: get_image_content_type

* feat: add image edits endpoint

* add image edits on UI

* test: image edits support

* fix: linting errors

* fix: linting errors

* test fix img gen

* feat: azure image edits

* fix: fix url construction of azure image edits

* fix: mock endpoints for azure images
 2025-05-26 10:37:48 -07:00
Ishaan JaffandGitHub 702b5fec22 [Feat] Add /image/edits on LiteLLM (#11123) 
* feat: add image edits on litellm router

* feat: add image edits endpoint

* fix: use pure async for image edits

* fix: base_process_llm_request

* fix: get_image_content_type

* feat: add image edits endpoint

* add image edits on UI

* test: image edits support

* fix: linting errors

* fix: linting errors

* test fix img gen
 2025-05-24 10:18:01 -07:00
Ishaan JaffandGitHub 8d7e234efd [Feat] Add content policy violation error mapping for image editd (#11113) 
* feat: add image edit mapping for content policy violations

* test fix
 2025-05-23 21:57:43 -07:00
Ishaan Jaff ba4148100f test: test_openai_image_edit_litellm_sdk 2025-05-23 17:34:03 -07:00
Ishaan JaffandGitHub dd4a65b83a Feat: add MCP to Responses API and bump openai python sdk (#11029) 
* feat: add MCP to responses API

* feat: bump openai version to 1.75.0

* docs MCP + responses API

* fixes: type checking

* fixes: type checking

* build: use latest openai 1.81.0

* fix: linting error

* fix: linting error

* fix: test

* fix: linting errors

* fix: test

* fix: test

* fix: linting

* Revert "fix: linting"

This reverts commit ebb19ff8cb1f8fcc3e224390e351676daccb33de.

* fix: linting
 2025-05-22 07:24:10 -07:00
Ishaan JaffandGitHub e5d278c454 [Feat] Add Image Edits Support to LiteLLM (#11020) 
* refactor: use 1 file for image methods

* refactor: use 1 file for image methods

* feat: add stubs for image edits

* fix: types for image edits

* feat: add async image edits

* feat: add base config for image edits

* feat: add basic structure for image edits

* feat: add ImageEditRequestUtils

* feat: complete instrumentation of image edits

* tes: test_openai_image_edit_litellm_sdk

* tets: test_openai_image_edit_litellm_sdk

* feat: get_provider_image_edit_config

* feat: add OpenAIImageEditConfig

* feat: working image edits

* fixes: working image edits

* fix: code qa

* fix: using image edits

* fix: linting errors
 2025-05-21 14:03:09 -07:00
Krrish Dholakia 058503ea11 test: skip test - model EOL 2025-05-20 15:57:12 -07:00
Krish DholakiaandGitHub ff68cbda8e Fix azure dall e 3 call with custom model name + Handle Bearer $LITELLM_API_KEY in x-litellm-api-key custom header (#10776) 
* fix(main.py): use base model instead of user model if given

Fixes https://github.com/BerriAI/litellm/issues/10760

* feat(azure/image_generation/__init__.py): make azure image gen check more robust

Fixes https://github.com/BerriAI/litellm/issues/10760

* fix(user_api_key_auth.py): support bearer token auth for `x-litellm-api-key` header

Fixes earlier regression on vertex ai passthrough auth

* fix(user_api_key_auth.py): refactor get api key into separate function

enables easier testing

* fix: cleanup

* fix: fix linting error

* fix: cleanup

* test: update tests
 2025-05-12 22:49:35 -07:00
Krish DholakiaandGitHub beae5cfea9 Litellm staging 05 10 2025 - openai pdf url support + sagemaker chat content length error fix (#10724) 
* Support pdf url's to openai (#10640)

* fix(gpt_transformation.py): support pdf url input to openai

pass as base64 as openai doesn't support image url's

* fix(openai.py): support async message transformation

allows async get request to convert url to base64

* fix(gpt_transformation.py): fix linting errrors and use common components across sync + async flows

* fix: fix linting errors

* fix(openai.py): pop correct var

* Fix sagemaker chat calls - content length error  (#10607)

* fix(sagemaker_chat/): support passing dynamic aws params

previously being ignored

* refactor(sagemaker/chat): more refactoring

* fix(sagemaker_chat/): make sure streaming is correctly handled post-refactor

* refactor: more refactoring to support using signed json str

* fix(sagemaker/chat): working sync streaming post refactor

* fix(sagemaker/chat): support async streaming post refactor

* fix(llm_http_handler.py): await async function

* fix: remove print statements

* test: update test

* test: update test

* fix(llm_http_handler.py): retain passing in data as json str

* test: update test

* fix(base_model_iterator.py): fix linting error

* test: test auth

* fix: fix linting error

* test: update test

* test: update translation test

* fix(gpt_transformation.py): handle awaitable/non-awaitable object

* fix: handle async flow for message transformation on openai compatible api's

* test: cleanup testing

* test: update test

* test(test_router.py): use model with higher quota

* test: simplify test

* test: update test
 2025-05-10 17:41:57 -07:00
Krrish Dholakia 7934610f63 test: update test to handle rate limit error 2025-05-10 16:15:48 -07:00
Ishaan JaffandGitHub 2e58e47b43 [Bug Fix] Add Cost Tracking for gpt-image-1 when quality is unspecified (#10247) 
* TestOpenAIGPTImage1

* fixes for cost calc

* fix ImageGenerationRequestQuality.MEDIUM
 2025-04-23 15:16:40 -07:00
Ishaan Jaff 20d84ddef1 get_base_image_generation_call_args 2025-04-02 21:04:06 -07:00
Ishaan Jaff 74550df197 get_base_image_generation_call_args 2025-04-02 20:52:16 -07:00
Ishaan Jaff 4ed0ab5b1c Revert "remove google dns for img tests" 
This reverts commit d3fc8b563c.
 2025-04-02 20:42:29 -07:00
Ishaan Jaff d3fc8b563c remove google dns for img tests 2025-04-02 20:34:47 -07:00
Krish DholakiaGitHubomrishiv
 		801ecb6517 Nova Canvas complete image generation tasks (#9177) (#9525) 
* Nova Canvas complete image generation tasks (#9177)

* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add color guided generation task for Nova Canvas

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix merge

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas image generation documentation

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas unit tests

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* ci(config.yml): bump ci config

* test: fix test

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: omrishiv <327609+omrishiv@users.noreply.github.com>
 2025-03-26 11:28:20 -07:00
Krrish Dholakia 2cf8dcaad2 fix(base_image_gen_test.py): weaken assertion, working locally failing on ci/cd 2025-03-11 22:49:09 -07:00
Krrish Dholakia 7015be8957 test: update test for correct aws region 2025-03-11 21:14:40 -07:00
omrishivGitHubKrish Dholakia
 		0674491386 add support for Amazon Nova Canvas model (#7838) 
* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2025-03-10 08:02:00 -07:00
Krish DholakiaandGitHub c69ec66dc5 fix(base_aws_llm.py): remove region name before sending in args (#8998) 
* fix(base_aws_llm.py): remove region name before sending in args

* fix(base_aws_llm.py): fix optional param pop position

* fix: fix linting error
 2025-03-04 23:05:28 -08:00
Ishaan JaffandGitHub 8a235e7d38 (Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112) 
* LoggingCallbackManager

* add logging_callback_manager

* use logging_callback_manager

* add add_litellm_failure_callback

* use add_litellm_callback

* use add_litellm_async_success_callback

* add_litellm_async_failure_callback

* linting fix

* fix logging callback manager

* test_duplicate_multiple_loggers_test

* use _reset_all_callbacks

* fix testing with dup callbacks

* test_basic_image_generation

* reset callbacks for tests

* fix check for _add_custom_logger_to_list

* fix test_amazing_sync_embedding

* fix _get_custom_logger_key

* fix batches testing

* fix _reset_all_callbacks

* fix _check_callback_list_size

* add callback_manager_test

* fix test gemini-2.0-flash-thinking-exp-01-21
 2025-01-30 19:35:50 -08:00
Krish DholakiaandGitHub 866fffb50d Litellm dev 01 21 2025 p1 (#7898) 
* fix(utils.py): don't pass 'anthropic-beta' header to vertex - will cause request to fail

* fix(utils.py): add flag to allow user to disable filtering invalid headers

ensure user can control behaviour

* style(utils.py): cleanup message

* test(test_utils.py): add unit test to cover invalid header filtering

* fix(proxy_server.py): fix custom openapi schema generation

* fix(utils.py): pass extra headers if set

* fix(main.py): fix image variation to use 'client' param
 2025-01-21 20:36:11 -08:00
Krish DholakiaandGitHub ad2f66b3e3 [BETA] Add OpenAI /images/variations + Topaz API support (#7700) 
* feat(main.py): initial commit for `/image/variations` endpoint support

* refactor(base_llm/): introduce new base llm base config for image variation endpoints

* refactor(openai/image_variations/transformation.py): implement openai image variation transformation handler

* fix: test

* feat(openai/): working openai `/image/variation` endpoint calls via sdk

* feat(topaz/): topaz sync image variation call support

Addresses https://github.com/BerriAI/litellm/issues/7593

'

* fix(topaz/transformation.py): fix linting errors

* fix(openai/image_variations/handler.py): fix passing json data

* fix(main.py): image_variation/

support async image variation route - `aimage_variation`

* fix(test_get_model_info.py): fix test

* fix: cleanup unused imports

* feat(openai/): add async `/image/variations` endpoint support

* feat(topaz/): support async `/image/variations` calls

* fix: test

* fix(utils.py): fix get_model_info_helper for no model info w/ provider config

handles situation where model info is not known but provider config exists

* test(test_router_fallbacks.py): mark flaky test

* fix: fix unused imports

* test: bump otel load test perf threshold - accounts for current load tests hitting same server
 2025-01-11 23:27:46 -08:00
Ishaan Jaff 137879ffea vertex testing use pathrise-convert-1606954137718 2025-01-05 14:00:17 -08:00
Ishaan Jaff ef8812d150 ci/cd update vertex acct 2025-01-05 13:43:32 -08:00
Ishaan JaffandGitHub 11e5960462 use helper for image gen tests (#7343) 2024-12-20 21:28:32 -08:00
Ishaan JaffGitHubKrrish Dholakia
 		920f4c9f82 (fix) add linting check to ban creating AsyncHTTPHandler during LLM calling (#6855) 
* fix triton

* fix TEXT_COMPLETION_CODESTRAL

* fix REPLICATE

* fix CLARIFAI

* fix HUGGINGFACE

* add test_no_async_http_handler_usage

* fix PREDIBASE

* fix anthropic use get_async_httpx_client

* fix vertex fine tuning

* fix dbricks get_async_httpx_client

* fix get_async_httpx_client vertex

* fix get_async_httpx_client

* fix get_async_httpx_client

* fix make_async_azure_httpx_request

* fix check_for_async_http_handler

* test: cleanup mistral model

* add check for AsyncClient

* fix check_for_async_http_handler

* fix get_async_httpx_client

* fix tests using in_memory_llm_clients_cache

* fix langfuse import

* fix import

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
 2024-11-21 19:03:02 -08:00