Commit Graph

512 Commits

Author SHA1 Message Date
Krish Dholakia c0845fec1f Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Krrish Dholakia 109add7946 build(model_prices_and_context_window.json): add gemini multimodal embedding cost 2025-03-26 23:04:24 -07:00
Krish Dholakia 4351c77253 Support Gemini audio token cost tracking + fix openai audio input token cost tracking (#9535)
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object

enables accurate cost tracking

* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it

Google has moved away from this for gemini-2.0 models

* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough

* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token

enables vertex ai cost tracking to work with audio tokens

* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set

* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token

more consistent behaviour across providers

* test: add unit test for gemini audio token cost calculation

* ci: bump ci config

* test: fix test
2025-03-26 17:26:25 -07:00
Ishaan Jaff 8eaf4c55c0 test_gemini_fine_tuned_model_request_consistency 2025-03-26 14:18:11 -07:00
Ishaan Jaff da9d849348 test_gemini_fine_tuned_model_request_consistency 2025-03-26 14:10:32 -07:00
Ishaan Jaff baa9b34950 Merge branch 'main' into litellm_fix_vertex_ai_ft_models 2025-03-26 11:11:54 -07:00
Ishaan Jaff bbe69a47a9 _is_model_gemini_gemini_spec_model 2025-03-26 10:53:23 -07:00
Ishaan Jaff e7a8b5a809 run ci/cd again 2025-03-26 08:12:51 -07:00
Ishaan Jaff efce84815a test_gemini_fine_tuned_model_request_consistency 2025-03-25 23:54:06 -07:00
Krish Dholakia 6fd18651d1 Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation (#9516)
* test(tests): add unit testing for litellm_proxy integration

* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy

* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes

* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion

* feat(vertex_ai/): test

* fix: fix linting error

* test: set api base as None before starting loadtest
2025-03-25 23:46:20 -07:00
Ishaan Jaff 9aec7c3878 test_create_delete_assistants 2025-03-25 22:08:06 -07:00
Krrish Dholakia 75994d0bf0 test: improve flaky test 2025-03-24 23:15:04 -07:00
Tyler Hutcherson 7864cd1f76 update redisvl dependency 2025-03-24 08:42:11 -04:00
Ishaan Jaff 69c9a782b2 add supports_web_search 2025-03-22 13:32:22 -07:00
Ishaan Jaff 78c371d2e8 search_context_cost_per_query test 2025-03-22 13:08:57 -07:00
Ishaan Jaff 1bdb94a314 add search_context_cost_per_1k_calls to model cost map spec 2025-03-22 12:56:21 -07:00
Krrish Dholakia 48e6a7036b test: mock sagemaker tests 2025-03-21 16:21:18 -07:00
Krrish Dholakia 46d68a61c8 fix: fix testing 2025-03-20 14:37:58 -07:00
Krish Dholakia 706bcf4432 Merge pull request #9366 from JamesGuthrie/jg/vertex-output-dimensionality
fix: VertexAI outputDimensionality configuration
2025-03-20 13:55:33 -07:00
Ishaan Jaff 247e4d09ee Merge branch 'main' into litellm_fix_ssl_verify 2025-03-19 21:03:06 -07:00
James Guthrie 437dbe7246 fix: VertexAI outputDimensionality configuration
VertexAI's API documentation [1] is an absolute mess. In it, they
describe the parameter to configure output dimensionality as
`output_dimensionality`. In the API example, they switch to using snake
case `outputDimensionality`, which is the correct variant.

[1]: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#generative-ai-get-text-embedding-drest
2025-03-19 11:07:36 +01:00
Ishaan Jaff e32aee9124 Merge pull request #9353 from BerriAI/litellm_arize_dynamic_logging
[Feat] - API - Allow using dynamic Arize AI Spaces on LiteLLM
2025-03-18 23:35:28 -07:00
Krish Dholakia 6347b694ee Merge pull request #9335 from BerriAI/litellm_dev_03_17_2025_p3
Contributor PR: Fix sagemaker too little data for content error
2025-03-18 23:24:07 -07:00
Ishaan Jaff 57e5c94360 Merge branch 'main' into litellm_arize_dynamic_logging 2025-03-18 22:13:35 -07:00
Ishaan Jaff c101fe9b5d Merge pull request #9352 from BerriAI/litellm_arize_mar_18
[Bug Fix] Arize AI Logging Integration with LiteLLM
2025-03-18 22:12:46 -07:00
Ishaan Jaff 412ad0d64e test_arize_callback 2025-03-18 20:21:23 -07:00
Ishaan Jaff 19a7bfa6b5 test_arize_callback 2025-03-18 18:49:06 -07:00
Krrish Dholakia a34cc2031d fix(response_metadata.py): log the litellm_model_name
make it easier to track the model sent to the provider
2025-03-18 17:46:33 -07:00
Krrish Dholakia 8ed3483adb test(test_tpm_rpm_routing_v2.py): initial test, for asserting async pre call check works as expected 2025-03-18 17:36:55 -07:00
Ishaan Jaff c010cdef59 test_dynamic_azure_params 2025-03-18 17:26:23 -07:00
Krrish Dholakia 39ac9e3eca fix(lowest_tpm_rpm_v2.py): fix updating limits 2025-03-18 17:10:17 -07:00
Ishaan Jaff 41d9cc14be test_async_dynamic_arize_config 2025-03-18 15:02:22 -07:00
Ishaan Jaff 3e0df0fc1e test_arize_set_attributes 2025-03-18 14:31:34 -07:00
Ishaan Jaff 959299c8ea Merge pull request #9338 from nate-mar/arize-integration-fixes
Arize integration Fix
2025-03-18 14:28:25 -07:00
Krrish Dholakia 267084a1af test(test_get_llm_provider.py): cover scenario where xai not in model name 2025-03-18 11:04:59 -07:00
Krrish Dholakia aeec703c4e test(test_get_llm_provider.py): Minimal repro for https://github.com/BerriAI/litellm/issues/9291 2025-03-18 10:35:50 -07:00
Nate Mar 9f9e555342 Update test_arize_ai.py 2025-03-18 04:04:44 -07:00
Nate Mar 6b821fbbc0 restore previous formatting 2025-03-18 04:02:43 -07:00
Nate Mar a1d188ba5e Fix test and add comments 2025-03-18 03:46:53 -07:00
Nate Mar 434e262b8c revert space_key change and add tests for arize integration 2025-03-18 01:40:10 -07:00
Krrish Dholakia b341b788a5 test: update testing 2025-03-17 22:34:08 -07:00
Krish Dholakia bcbb88d802 Merge pull request #9326 from andjsmi/main
Modify completion handler for SageMaker to use payload from `prepared_request`
2025-03-17 22:16:43 -07:00
Krish Dholakia cd5024f3b1 Merge pull request #9333 from BerriAI/litellm_dev_03_17_2025_p2
fix(ollama/completions/transformation.py): pass prompt, untemplated o…
2025-03-17 21:48:30 -07:00
Nate Mar 35e0856f11 Fix wrong import and use space_id instead of space_key for Arize integration 2025-03-17 20:37:28 -07:00
Krrish Dholakia 22faf7d232 fix(ollama/completions/transformation.py): pass prompt, untemplated on /completions request
Fixes https://github.com/BerriAI/litellm/issues/6900
2025-03-17 18:35:44 -07:00
Krrish Dholakia c4b2e0ae3d fix(streaming_handler.py): support logging complete streaming response on cache hit 2025-03-17 18:10:39 -07:00
Krrish Dholakia dd9e79adbd fix(streaming_handler.py): emit deep copy of completed chunk 2025-03-17 17:26:21 -07:00
Andrew Smith d5274efe62 Update test_sagemaker.py to match changed parameters 2025-03-18 11:09:52 +11:00
Krrish Dholakia 8618295911 test: loosen test 2025-03-17 09:44:22 -07:00
Krrish Dholakia d01361747d test: make test less flaky 2025-03-17 09:00:15 -07:00