* Add new model provider Novita AI (#7582)
* feat: add new model provider Novita AI
* feat: use deepseek r1 model for examples in Novita AI docs
* fix: fix tests
* fix: fix tests for novita
* fix: fix novita transformation
* ci: fix ci yaml
* fix: fix novita transformation and test (#10056)
---------
Co-authored-by: Jason <ggbbddjm@gmail.com>
* Support pdf url's to openai (#10640)
* fix(gpt_transformation.py): support pdf url input to openai
pass as base64 as openai doesn't support image url's
* fix(openai.py): support async message transformation
allows async get request to convert url to base64
* fix(gpt_transformation.py): fix linting errrors and use common components across sync + async flows
* fix: fix linting errors
* fix(openai.py): pop correct var
* Fix sagemaker chat calls - content length error (#10607)
* fix(sagemaker_chat/): support passing dynamic aws params
previously being ignored
* refactor(sagemaker/chat): more refactoring
* fix(sagemaker_chat/): make sure streaming is correctly handled post-refactor
* refactor: more refactoring to support using signed json str
* fix(sagemaker/chat): working sync streaming post refactor
* fix(sagemaker/chat): support async streaming post refactor
* fix(llm_http_handler.py): await async function
* fix: remove print statements
* test: update test
* test: update test
* fix(llm_http_handler.py): retain passing in data as json str
* test: update test
* fix(base_model_iterator.py): fix linting error
* test: test auth
* fix: fix linting error
* test: update test
* test: update translation test
* fix(gpt_transformation.py): handle awaitable/non-awaitable object
* fix: handle async flow for message transformation on openai compatible api's
* test: cleanup testing
* test: update test
* test(test_router.py): use model with higher quota
* test: simplify test
* test: update test
* fix(auth_checks.py): enforce auth checks on target model names
ensures user has access to models they are trying to call
* test(test_auth_utils.py): add unit tests for auth check
* fix(exception_mapping_utils.py): handle mistral 429 exception
* fix: fix linting error
* fix(auth_checks.py): add max fallback depth
* fix(caching_handler.py): fix embedding str caching result
Fixes issue where str caching results were not being correctly assembled on str input
* feat(azure/image_generation): Support dropping response_format for azure gpt-image-1
Fixes LIT-118
* test(test_utils.py): add unit testing
* test: rename file to avoid testing conflict
* feat(team_info.tsx): allow user to reassign team to another org
* style(team_info.tsx): fix org id styling
* feat(team_endpoints.py): add validation check before migrating team to another org
ensure model access, budgets and membership is respected
* fix(team_endpoints.py): update model migration validation to check if org has 'all-proxy-models' access
* fix(organization_view.tsx): show teams belonging to org
* feat(team_endpoints.py): handle wildcard model check on org migration
* fix(team_endpoints.py): nest router check
* test: update testing - use model with higher quota
* build: update poetry lock
* added tests
messages_with_counts: Made tolerance explicit for each test. But they match the new implementation(which beats the old)
* new token counter impl
* compare old and new implementation in test
* delete old token counter
* moved tests to /tests/litellm/litellm_core_utils
* use existing types
* docstrings
* warn about using default params on unknown model.
* created type for the token_counter_function
* check key == "content"
* throw error on invalid detail-type, ignore type-warning.
* fix imports
* Update docs for OpenAI compatible providers, add Llamafile docs, include Llamafile in the sidebar
* Add Llamafile as an LlmProviders enum
* Add llamafile as a OpenAI compatible provider (in the list of compatible providers)
* Add Llamafile chat config and tests
* Wire up Llamafile
Co-authored-by: Peter Wilson <peter@mozilla.ai>
* fix(exception_mapping_utils.py): correctly pass through 504 status code
openai also raises a 504 status code
* build(model_prices_and_context_window.json): add gpt-4o-mini-tts to model cost map
Fixes https://github.com/BerriAI/litellm/issues/9591
* fix(cost_calculator.py): fix input cost calculation for gpt-4o-mini-tts
Fixes https://github.com/BerriAI/litellm/issues/9591
* test: testing updates
* fix: initial commit of v2 parallel request limiter hook
enables multi-instance rate limiting to work
* fix: subsequent commit with additional refactors
* fix(parallel_request_limiter_v2.py): cleanup initial call hook
simplify it
* fix(parallel_request_limiter_v2.py): working v2 parallel request limiter
* fix: more updates - still not passing testing
* fix(test_parallel_request_limiter_v2.py): update test + add conftest
* fix: fix ruff checks
* fix(parallel_request_limiter_v2.py): use pull via pattern method to load in keys instance wouldn't have seen yet
Fixes issue where redis syncing was not pulling key until instance had seen it
* test: update testing to cover tpm and rpm
* fix(parallel_request_limiter_v2.py): fix ruff errors
* fix(proxy/hooks/__init__.py): feature flag export
* fix(proxy/hooks/__init_.py): fix linting error
* ci(config.yml): add tests/enterprise to ci/cd
* fix: fix ruff check
* test: update testing
* build(model_prices_and_context_window.json): add fireworks ai new 0-4b pricing tier
* build(model_prices_and_context_window.json): add more fireworks ai models
* test: update testing
* fix(caching_handler.py): handle str + list cache
Fixes issue on cache hits for embedding when initial cached input was str
* test(test_caching.py): add e2e test on caching with individual item and then list
* fix(caching_handler.py): set usage tokens for cache hits
enables token counting to work
* fix(caching_handler.py): combine usage between cached result and embedding response
Handles case of new input to embedding response
* fix: cleanup
* test: move to gpt-4o-new-test
* test: update test
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly
Fixes https://github.com/BerriAI/litellm/issues/7209
* fix(utils.py): handle none type in message
* fix: fix model name in test
* fix(utils.py): fix validate check for openai messages
* fix: fix model returned
* fix(main.py): fix text completion routing
* test: update testing
* test: skip test - cohere having RBAC issues
Ishaan Jaff