tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-14 10:25:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
20,928 Commits 2,072 Branches 1,425 Tags
87f0201f8400bee3550e1c82befccfc637fc4436
Commit Graph
510 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub 4351c77253 Support Gemini audio token cost tracking + fix openai audio input token cost tracking (#9535) 
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object

enables accurate cost tracking

* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it

Google has moved away from this for gemini-2.0 models

* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough

* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token

enables vertex ai cost tracking to work with audio tokens

* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set

* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token

more consistent behaviour across providers

* test: add unit test for gemini audio token cost calculation

* ci: bump ci config

* test: fix test
 2025-03-26 17:26:25 -07:00
Ishaan Jaff 8eaf4c55c0 test_gemini_fine_tuned_model_request_consistency 2025-03-26 14:18:11 -07:00
Ishaan Jaff da9d849348 test_gemini_fine_tuned_model_request_consistency 2025-03-26 14:10:32 -07:00
Ishaan Jaff baa9b34950 Merge branch 'main' into litellm_fix_vertex_ai_ft_models 2025-03-26 11:11:54 -07:00
Ishaan Jaff bbe69a47a9 _is_model_gemini_gemini_spec_model 2025-03-26 10:53:23 -07:00
Ishaan Jaff e7a8b5a809 run ci/cd again 2025-03-26 08:12:51 -07:00
Ishaan Jaff efce84815a test_gemini_fine_tuned_model_request_consistency 2025-03-25 23:54:06 -07:00
Krish DholakiaandGitHub 6fd18651d1 Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation (#9516) 
* test(tests): add unit testing for litellm_proxy integration

* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy

* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes

* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion

* feat(vertex_ai/): test

* fix: fix linting error

* test: set api base as None before starting loadtest
 2025-03-25 23:46:20 -07:00
Ishaan Jaff 9aec7c3878 test_create_delete_assistants 2025-03-25 22:08:06 -07:00
Krrish Dholakia 75994d0bf0 test: improve flaky test 2025-03-24 23:15:04 -07:00
Tyler Hutcherson 7864cd1f76 update redisvl dependency 2025-03-24 08:42:11 -04:00
Ishaan Jaff 69c9a782b2 add supports_web_search 2025-03-22 13:32:22 -07:00
Ishaan Jaff 78c371d2e8 search_context_cost_per_query test 2025-03-22 13:08:57 -07:00
Ishaan Jaff 1bdb94a314 add search_context_cost_per_1k_calls to model cost map spec 2025-03-22 12:56:21 -07:00
Krrish Dholakia 48e6a7036b test: mock sagemaker tests 2025-03-21 16:21:18 -07:00
Krrish Dholakia 46d68a61c8 fix: fix testing 2025-03-20 14:37:58 -07:00
Krish DholakiaandGitHub 706bcf4432 Merge pull request #9366 from JamesGuthrie/jg/vertex-output-dimensionality 
fix: VertexAI outputDimensionality configuration
 2025-03-20 13:55:33 -07:00
Ishaan JaffandGitHub 247e4d09ee Merge branch 'main' into litellm_fix_ssl_verify 2025-03-19 21:03:06 -07:00
James Guthrie 437dbe7246 fix: VertexAI outputDimensionality configuration 
VertexAI's API documentation [1] is an absolute mess. In it, they
describe the parameter to configure output dimensionality as
`output_dimensionality`. In the API example, they switch to using snake
case `outputDimensionality`, which is the correct variant.

[1]: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#generative-ai-get-text-embedding-drest
 2025-03-19 11:07:36 +01:00
Ishaan JaffandGitHub e32aee9124 Merge pull request #9353 from BerriAI/litellm_arize_dynamic_logging 
[Feat] - API - Allow using dynamic Arize AI Spaces on LiteLLM
 2025-03-18 23:35:28 -07:00
Krish DholakiaandGitHub 6347b694ee Merge pull request #9335 from BerriAI/litellm_dev_03_17_2025_p3 
Contributor PR: Fix sagemaker too little data for content error
 2025-03-18 23:24:07 -07:00
Ishaan Jaff 57e5c94360 Merge branch 'main' into litellm_arize_dynamic_logging 2025-03-18 22:13:35 -07:00
Ishaan JaffandGitHub c101fe9b5d Merge pull request #9352 from BerriAI/litellm_arize_mar_18 
[Bug Fix] Arize AI Logging Integration with LiteLLM
 2025-03-18 22:12:46 -07:00
Ishaan Jaff 412ad0d64e test_arize_callback 2025-03-18 20:21:23 -07:00
Ishaan Jaff 19a7bfa6b5 test_arize_callback 2025-03-18 18:49:06 -07:00
Krrish Dholakia a34cc2031d fix(response_metadata.py): log the litellm_model_name 
make it easier to track the model sent to the provider
 2025-03-18 17:46:33 -07:00
Krrish Dholakia 8ed3483adb test(test_tpm_rpm_routing_v2.py): initial test, for asserting async pre call check works as expected 2025-03-18 17:36:55 -07:00
Ishaan Jaff c010cdef59 test_dynamic_azure_params 2025-03-18 17:26:23 -07:00
Krrish Dholakia 39ac9e3eca fix(lowest_tpm_rpm_v2.py): fix updating limits 2025-03-18 17:10:17 -07:00
Ishaan Jaff 41d9cc14be test_async_dynamic_arize_config 2025-03-18 15:02:22 -07:00
Ishaan Jaff 3e0df0fc1e test_arize_set_attributes 2025-03-18 14:31:34 -07:00
Ishaan JaffandGitHub 959299c8ea Merge pull request #9338 from nate-mar/arize-integration-fixes 
Arize integration Fix
 2025-03-18 14:28:25 -07:00
Krrish Dholakia 267084a1af test(test_get_llm_provider.py): cover scenario where xai not in model name 2025-03-18 11:04:59 -07:00
Krrish Dholakia aeec703c4e test(test_get_llm_provider.py): Minimal repro for https://github.com/BerriAI/litellm/issues/9291 2025-03-18 10:35:50 -07:00
Nate Mar 9f9e555342 Update test_arize_ai.py 2025-03-18 04:04:44 -07:00
Nate Mar 6b821fbbc0 restore previous formatting 2025-03-18 04:02:43 -07:00
Nate Mar a1d188ba5e Fix test and add comments 2025-03-18 03:46:53 -07:00
Nate Mar 434e262b8c revert space_key change and add tests for arize integration 2025-03-18 01:40:10 -07:00
Krrish Dholakia b341b788a5 test: update testing 2025-03-17 22:34:08 -07:00
Krish DholakiaandGitHub bcbb88d802 Merge pull request #9326 from andjsmi/main 
Modify completion handler for SageMaker to use payload from `prepared_request`
 2025-03-17 22:16:43 -07:00
Krish DholakiaandGitHub cd5024f3b1 Merge pull request #9333 from BerriAI/litellm_dev_03_17_2025_p2 
fix(ollama/completions/transformation.py): pass prompt, untemplated o…
 2025-03-17 21:48:30 -07:00
Nate Mar 35e0856f11 Fix wrong import and use space_id instead of space_key for Arize integration 2025-03-17 20:37:28 -07:00
Krrish Dholakia 22faf7d232 fix(ollama/completions/transformation.py): pass prompt, untemplated on /completions request 
Fixes https://github.com/BerriAI/litellm/issues/6900
 2025-03-17 18:35:44 -07:00
Krrish Dholakia c4b2e0ae3d fix(streaming_handler.py): support logging complete streaming response on cache hit 2025-03-17 18:10:39 -07:00
Krrish Dholakia dd9e79adbd fix(streaming_handler.py): emit deep copy of completed chunk 2025-03-17 17:26:21 -07:00
Andrew SmithandGitHub d5274efe62 Update test_sagemaker.py to match changed parameters 2025-03-18 11:09:52 +11:00
Krrish Dholakia 8618295911 test: loosen test 2025-03-17 09:44:22 -07:00
Krrish Dholakia d01361747d test: make test less flaky 2025-03-17 09:00:15 -07:00
Krish DholakiaandGitHub d4caaae1be Merge pull request #9274 from BerriAI/litellm_contributor_rebase_branch 
Litellm contributor rebase branch
 2025-03-14 21:57:49 -07:00
Krrish Dholakia c2f01b0fdc fix(router.py): add new test 2025-03-14 14:23:45 -07:00