Commit Graph

32 Commits

Author SHA1 Message Date
Ishaan Jaff a76d991952 [Fix]: Add cost tracking for image edits endpoint [OpenAI, Azure] (#11186)
* fix: add cost tracking for image edits

* fix: add cost tracking for azure image edits

* fix: fix linting error

* fix: fixes for background param

* fix: image edit test fixes

* fix: openai image edit cost tracking
2025-05-27 17:52:15 -07:00
Ishaan Jaff 1009defbdc [Feat] Add /image/edits support for Azure (#11160)
* feat: add image edits on litellm router

* feat: add image edits endpoint

* fix: use pure async for image edits

* fix: base_process_llm_request

* fix: get_image_content_type

* feat: add image edits endpoint

* add image edits on UI

* test: image edits support

* fix: linting errors

* fix: linting errors

* test fix img gen

* feat: azure image edits

* fix: fix url construction of azure image edits

* fix: mock endpoints for azure images
2025-05-26 10:37:48 -07:00
Ishaan Jaff 702b5fec22 [Feat] Add /image/edits on LiteLLM (#11123)
* feat: add image edits on litellm router

* feat: add image edits endpoint

* fix: use pure async for image edits

* fix: base_process_llm_request

* fix: get_image_content_type

* feat: add image edits endpoint

* add image edits on UI

* test: image edits support

* fix: linting errors

* fix: linting errors

* test fix img gen
2025-05-24 10:18:01 -07:00
Ishaan Jaff 8d7e234efd [Feat] Add content policy violation error mapping for image editd (#11113)
* feat: add image edit mapping for content policy violations

* test fix
2025-05-23 21:57:43 -07:00
Ishaan Jaff ba4148100f test: test_openai_image_edit_litellm_sdk 2025-05-23 17:34:03 -07:00
Ishaan Jaff dd4a65b83a Feat: add MCP to Responses API and bump openai python sdk (#11029)
* feat: add MCP to responses API

* feat: bump openai version to 1.75.0

* docs MCP + responses API

* fixes: type checking

* fixes: type checking

* build: use latest openai 1.81.0

* fix: linting error

* fix: linting error

* fix: test

* fix: linting errors

* fix: test

* fix: test

* fix: linting

* Revert "fix: linting"

This reverts commit ebb19ff8cb1f8fcc3e224390e351676daccb33de.

* fix: linting
2025-05-22 07:24:10 -07:00
Ishaan Jaff e5d278c454 [Feat] Add Image Edits Support to LiteLLM (#11020)
* refactor: use 1 file for image methods

* refactor: use 1 file for image methods

* feat: add stubs for image edits

* fix: types for image edits

* feat: add async image edits

* feat: add base config for image edits

* feat: add basic structure for image edits

* feat: add ImageEditRequestUtils

* feat: complete instrumentation of image edits

* tes: test_openai_image_edit_litellm_sdk

* tets: test_openai_image_edit_litellm_sdk

* feat: get_provider_image_edit_config

* feat: add OpenAIImageEditConfig

* feat: working image edits

* fixes: working image edits

* fix: code qa

* fix: using image edits

* fix: linting errors
2025-05-21 14:03:09 -07:00
Krrish Dholakia 058503ea11 test: skip test - model EOL 2025-05-20 15:57:12 -07:00
Krish Dholakia ff68cbda8e Fix azure dall e 3 call with custom model name + Handle Bearer $LITELLM_API_KEY in x-litellm-api-key custom header (#10776)
* fix(main.py): use base model instead of user model if given

Fixes https://github.com/BerriAI/litellm/issues/10760

* feat(azure/image_generation/__init__.py): make azure image gen check more robust

Fixes https://github.com/BerriAI/litellm/issues/10760

* fix(user_api_key_auth.py): support bearer token auth for `x-litellm-api-key` header

Fixes earlier regression on vertex ai passthrough auth

* fix(user_api_key_auth.py): refactor get api key into separate function

enables easier testing

* fix: cleanup

* fix: fix linting error

* fix: cleanup

* test: update tests
2025-05-12 22:49:35 -07:00
Krish Dholakia beae5cfea9 Litellm staging 05 10 2025 - openai pdf url support + sagemaker chat content length error fix (#10724)
* Support pdf url's to openai (#10640)

* fix(gpt_transformation.py): support pdf url input to openai

pass as base64 as openai doesn't support image url's

* fix(openai.py): support async message transformation

allows async get request to convert url to base64

* fix(gpt_transformation.py): fix linting errrors and use common components across sync + async flows

* fix: fix linting errors

* fix(openai.py): pop correct var

* Fix sagemaker chat calls - content length error  (#10607)

* fix(sagemaker_chat/): support passing dynamic aws params

previously being ignored

* refactor(sagemaker/chat): more refactoring

* fix(sagemaker_chat/): make sure streaming is correctly handled post-refactor

* refactor: more refactoring to support using signed json str

* fix(sagemaker/chat): working sync streaming post refactor

* fix(sagemaker/chat): support async streaming post refactor

* fix(llm_http_handler.py): await async function

* fix: remove print statements

* test: update test

* test: update test

* fix(llm_http_handler.py): retain passing in data as json str

* test: update test

* fix(base_model_iterator.py): fix linting error

* test: test auth

* fix: fix linting error

* test: update test

* test: update translation test

* fix(gpt_transformation.py): handle awaitable/non-awaitable object

* fix: handle async flow for message transformation on openai compatible api's

* test: cleanup testing

* test: update test

* test(test_router.py): use model with higher quota

* test: simplify test

* test: update test
2025-05-10 17:41:57 -07:00
Krrish Dholakia 7934610f63 test: update test to handle rate limit error 2025-05-10 16:15:48 -07:00
Ishaan Jaff 2e58e47b43 [Bug Fix] Add Cost Tracking for gpt-image-1 when quality is unspecified (#10247)
* TestOpenAIGPTImage1

* fixes for cost calc

* fix ImageGenerationRequestQuality.MEDIUM
2025-04-23 15:16:40 -07:00
Ishaan Jaff 20d84ddef1 get_base_image_generation_call_args 2025-04-02 21:04:06 -07:00
Ishaan Jaff 74550df197 get_base_image_generation_call_args 2025-04-02 20:52:16 -07:00
Ishaan Jaff 4ed0ab5b1c Revert "remove google dns for img tests"
This reverts commit d3fc8b563c.
2025-04-02 20:42:29 -07:00
Ishaan Jaff d3fc8b563c remove google dns for img tests 2025-04-02 20:34:47 -07:00
Krish Dholakia 801ecb6517 Nova Canvas complete image generation tasks (#9177) (#9525)
* Nova Canvas complete image generation tasks (#9177)

* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add color guided generation task for Nova Canvas

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix merge

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas image generation documentation

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas unit tests

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* ci(config.yml): bump ci config

* test: fix test

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-26 11:28:20 -07:00
Krrish Dholakia 2cf8dcaad2 fix(base_image_gen_test.py): weaken assertion, working locally failing on ci/cd 2025-03-11 22:49:09 -07:00
Krrish Dholakia 7015be8957 test: update test for correct aws region 2025-03-11 21:14:40 -07:00
omrishiv 0674491386 add support for Amazon Nova Canvas model (#7838)
* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-03-10 08:02:00 -07:00
Krish Dholakia c69ec66dc5 fix(base_aws_llm.py): remove region name before sending in args (#8998)
* fix(base_aws_llm.py): remove region name before sending in args

* fix(base_aws_llm.py): fix optional param pop position

* fix: fix linting error
2025-03-04 23:05:28 -08:00
Ishaan Jaff 8a235e7d38 (Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112)
* LoggingCallbackManager

* add logging_callback_manager

* use logging_callback_manager

* add add_litellm_failure_callback

* use add_litellm_callback

* use add_litellm_async_success_callback

* add_litellm_async_failure_callback

* linting fix

* fix logging callback manager

* test_duplicate_multiple_loggers_test

* use _reset_all_callbacks

* fix testing with dup callbacks

* test_basic_image_generation

* reset callbacks for tests

* fix check for _add_custom_logger_to_list

* fix test_amazing_sync_embedding

* fix _get_custom_logger_key

* fix batches testing

* fix _reset_all_callbacks

* fix _check_callback_list_size

* add callback_manager_test

* fix test gemini-2.0-flash-thinking-exp-01-21
2025-01-30 19:35:50 -08:00
Krish Dholakia 866fffb50d Litellm dev 01 21 2025 p1 (#7898)
* fix(utils.py): don't pass 'anthropic-beta' header to vertex - will cause request to fail

* fix(utils.py): add flag to allow user to disable filtering invalid headers

ensure user can control behaviour

* style(utils.py): cleanup message

* test(test_utils.py): add unit test to cover invalid header filtering

* fix(proxy_server.py): fix custom openapi schema generation

* fix(utils.py): pass extra headers if set

* fix(main.py): fix image variation to use 'client' param
2025-01-21 20:36:11 -08:00
Krish Dholakia ad2f66b3e3 [BETA] Add OpenAI /images/variations + Topaz API support (#7700)
* feat(main.py): initial commit for `/image/variations` endpoint support

* refactor(base_llm/): introduce new base llm base config for image variation endpoints

* refactor(openai/image_variations/transformation.py): implement openai image variation transformation handler

* fix: test

* feat(openai/): working openai `/image/variation` endpoint calls via sdk

* feat(topaz/): topaz sync image variation call support

Addresses https://github.com/BerriAI/litellm/issues/7593

'

* fix(topaz/transformation.py): fix linting errors

* fix(openai/image_variations/handler.py): fix passing json data

* fix(main.py): image_variation/

support async image variation route - `aimage_variation`

* fix(test_get_model_info.py): fix test

* fix: cleanup unused imports

* feat(openai/): add async `/image/variations` endpoint support

* feat(topaz/): support async `/image/variations` calls

* fix: test

* fix(utils.py): fix get_model_info_helper for no model info w/ provider config

handles situation where model info is not known but provider config exists

* test(test_router_fallbacks.py): mark flaky test

* fix: fix unused imports

* test: bump otel load test perf threshold - accounts for current load tests hitting same server
2025-01-11 23:27:46 -08:00
Ishaan Jaff 137879ffea vertex testing use pathrise-convert-1606954137718 2025-01-05 14:00:17 -08:00
Ishaan Jaff ef8812d150 ci/cd update vertex acct 2025-01-05 13:43:32 -08:00
Ishaan Jaff 11e5960462 use helper for image gen tests (#7343) 2024-12-20 21:28:32 -08:00
Ishaan Jaff 920f4c9f82 (fix) add linting check to ban creating AsyncHTTPHandler during LLM calling (#6855)
* fix triton

* fix TEXT_COMPLETION_CODESTRAL

* fix REPLICATE

* fix CLARIFAI

* fix HUGGINGFACE

* add test_no_async_http_handler_usage

* fix PREDIBASE

* fix anthropic use get_async_httpx_client

* fix vertex fine tuning

* fix dbricks get_async_httpx_client

* fix get_async_httpx_client vertex

* fix get_async_httpx_client

* fix get_async_httpx_client

* fix make_async_azure_httpx_request

* fix check_for_async_http_handler

* test: cleanup mistral model

* add check for AsyncClient

* fix check_for_async_http_handler

* fix get_async_httpx_client

* fix tests using in_memory_llm_clients_cache

* fix langfuse import

* fix import

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2024-11-21 19:03:02 -08:00
Ishaan Jaff 73c7b73aa0 (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716)
* add BaseImageGenTest

* use 1 class for unit testing

* add debugging to BaseImageGenTest

* TestAzureOpenAIDalle3

* fix response_cost_calculator

* test_basic_image_generation

* fix img gen basic test

* fix _select_model_name_for_cost_calc

* fix test_aimage_generation_bedrock_with_optional_params

* fix undo changes cost tracking

* fix response_cost_calculator

* fix test_cost_azure_gpt_35
2024-11-12 20:02:16 -08:00
Ishaan Jaff 25bae4cc23 (feat) add cost tracking stable diffusion 3 on Bedrock (#6676)
* add cost tracking for sd3

* test_image_generation_bedrock

* fix get model info for image cost

* add cost_calculator for stability 1 models

* add unit testing for bedrock image cost calc

* test_cost_calculator_with_no_optional_params

* add test_cost_calculator_basic

* correctly allow size Optional

* fix cost_calculator

* sd3 unit tests cost calc
2024-11-11 20:21:44 -08:00
Ishaan Jaff 979dfe8ab2 (feat) Add Bedrock Stability.ai Stable Diffusion 3 Image Generation models (#6673)
* add bedrock image gen async support

* added async support for bedrock image gen

* move image gen testing

* add AmazonStability3Config

* add AmazonStability3Config config

* update AmazonStabilityConfig

* update get_optional_params_image_gen

* use 1 helper for _get_request_body

* add transform_response_dict_to_openai_response for stability3

* test sd3-large-v1:0

* unit testing for bedrock image gen

* fix load_vertex_ai_credentials

* fix test_aimage_generation_vertex_ai

* add stability.sd3-large-v1:0 to model cost map

* add stability.stability.sd3-large-v1:0 to docs
2024-11-08 19:26:03 -08:00
Ishaan Jaff 086e1ac5bc move image gen testing 2024-11-08 14:10:08 -08:00