* build(model_prices_and_context_window.json): add bedrock llama4 models to model cost map
* fix template conversion for Llama 4 models in Bedrock (#10557)
* test: add testing to repro https://github.com/BerriAI/litellm/pull/10557
* test: add unit testing
* test(test_main.py): refactor where test is kept
---------
Co-authored-by: aswny <87371411+aswny@users.noreply.github.com>
* Add `litellm-proxy` CLI (#10478)
* First cut at a Python client module for proxy
* Add UnauthorizedError + add_model method
* Add delete_model method
* Add example model_id to delete_model docstring
* Make delete_model raise NotFoundError
* Add get_model
* Add get_all_model_info
* Rename models.list_models to models.list
* Rename models.get_all_model_info to models.info
* Move ModelsManagementClient.get_all_model_group_info to ModelGroupsManagementClient.info
* Rename get_model to get
* Rename add_model to new
* Rename delete_model to delete
* In client classes, rename base_url attribute to _base_url and api_key attribute to _api_key
* Add ModelsManagementClient.updae method
* Add client.chat.completions (ChatClient)
* ruff format litellm/proxy/client
* ruff format tests/litellm/proxy/client/*.py
* Add latest changes
* Rename KeysManagementClient.create to KeysManagementClient.generate
* Add new parameters to KeysManagementClient.generate
* Add CredentialsManagementClient
* Remove api_key parameter from KeysManagementClient.generate
* Fix lint errors
* Add litellm/proxy/client/README.md
* README.md: Remove api_key param to client.keys.generate
* Fix mypy errors
* First cut at litellm-proxy cli
* Add test for `litellm-proxy models list`
* Nicer get_models_info
* get_models_info: --columns option
* Use format_timestamp in list_models
* ruff format litellm/proxy/client
* Simpler JSON printing with rich.print_json
* Move models-related commands to separate file
From `cli.py` to `groups/models.py`
* Improve directory structure
* Cleanup cli/groups/models.py - esp. usage of rich
* Refactoring
* Refactor mocking in cli/test_main.py
* Dedup models commands tests
* Update poetry.lock
* Fix mypy errors
* ruff format litellm/proxy/client/cli
* ruff format tests/litellm/proxy/client/*.py
* Fix timezone issue in test_models_list_table_format
* Add cli/README.md
* Small README.md tweaks
* README.md enhancements
* Add credentials commands
* Add chat commands
* Add http commands
* ruff format litellm/proxy/client/cli
* Fix lint errors in credentials and http commands
* json => json_lib
* test-key => sk-test-key
* Mock HTTP responses so http command tests pass
* Fix mypy error in credentials.py
* bump: version 1.67.5 → 1.67.6
* build: update litellm version
* cli/main.py: show_envvar=True
* Increase test job timeout to 8 minutes
because it looks like maybe the job is getting canceled because it takes
too long with the additional tests?
This probably could be reverted once #10484 is merged, since that speeds
up pytest runs greatly.
* Add keys functionality to library/CLI
* Add info about keys commands to litellm/proxy/client/cli/README.md
* Move Model Information section in CLI README
* Make Model Information a level 4 heading
* Move rich to extras
as suggested by @ishaan-jaff
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* pin rich=13.7.1
---------
Co-authored-by: Marc Abramowitz <abramowi@adobe.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Fixes:
Error generating conversation title: litellm.BadRequestError: VertexAIException BadRequestError - {
"error": {
"code": 400,
"message": "Unable to submit request because it has a maxOutputTokens value of 65536 but the supported range is from 1 (inclusive) to 65536 (exclusive). Update the value and try again.",
"status": "INVALID_ARGUMENT"
}
}
* fixes for generic api logger
* tests for generic api logger
* test_generic_api_callback_multiple_logs
* allow health checking generic api endpoints
* docs generic api endpoint for logging
* allow setting headers for generic api callback
* fix for test_init_custom_logger_compatible_class_as_callback
* fix linting
* fix(converse_transformation.py): handle meta llama tool call response
Fixes issue where bedrock meta llama would return tool call response as content str
* test(test_converse_transformation.py): add unit testing for new function
* fix: fix linting error
* fix: fix linting error
* fix(model_management_endpoints.py): allow team admin to update model via `/model/{model_id}/update` route
Fixes ui regression where team admin could not modify their own models
* fix(provider_specific_fields.tsx): style fix
* fix(table.tsx): allow expanding multiple rows
* fix(organization_endpoints.py): more robust check if user can give org model access
handle when user has models=["all-proxy-models"]
* fix(organization_endpoints.py): enable proxy admin with 'all-proxy-model' access to create new org with specific models
Fixes LIT-135
* fix: fix linting error
* fix: fix ui linting error
* fix(index.tsx): fix linting errors
* Support Llama-api as an LLM provider (#10451)
* init: support llama-api as a llm provider
* docs: fix endpoint url
* fix: rename meta dir to meta-llama
* docs: add meta-llama info
* fix: mv LlamaAPIConfig under chat directory
* feat: add LlamaAPIConfig in ProviderConfigManager
* fix: provider_config from ProviderConfigManager
* feat: add supports_tool_choice param
* fix: remove optional_params using model_info
* fix: rename meta-llama to meta_llama
* init: test for meta_llama
* fix: model names
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix file naming convention
* fix file naming convention for meta_llama
* docs meta llama api litellm
---------
Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat(team_info.tsx): allow user to reassign team to another org
* style(team_info.tsx): fix org id styling
* feat(team_endpoints.py): add validation check before migrating team to another org
ensure model access, budgets and membership is respected
* fix(team_endpoints.py): update model migration validation to check if org has 'all-proxy-models' access
* fix(organization_view.tsx): show teams belonging to org
* feat(team_endpoints.py): handle wildcard model check on org migration
* fix(team_endpoints.py): nest router check
* test: update testing - use model with higher quota
* build: update poetry lock
* refactor KB implementation to use central registry
* allow passing tools when making KB calls
* test fixes
* linting fix
* fix kb tests
* QA for KB stored in DB
* fix, use litellm_credential_name when adding KB on litellm UI
* QA list endpoint vector stores
* allow using UI creds with KBs
* feat(provider_specific_fields.tsx): Use correct form instance
Fixes https://github.com/BerriAI/litellm/issues/10115
* Fix broken pagination by correctly passing page and pageSize to keyListCall (#10498)
* [Organization] Include litellm_budget_table in /organization/list response (#10488)
* join litellm_budget_table in /organization/list endpoint
* update test
---------
Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
Fixes: GH-9641
This is a Cursor-generated fix for the following warning from pytest:
```
litellm/caching/llm_caching_handler.py:17
/Users/abramowi/Code/OpenSource/litellm/litellm/caching/llm_caching_handler.py:17: DeprecationWarning: There is no current event loop
event_loop = asyncio.get_event_loop()
```
Cursor prompt:
Fix this pytest warning
```
litellm/caching/llm_caching_handler.py:17
/Users/abramowi/Code/OpenSource/litellm/litellm/caching/llm_caching_handler.py:17: DeprecationWarning: There is no current event loop
event_loop = asyncio.get_event_loop()
```
Fixes https://github.com/BerriAI/litellm/issues/9641
Krish Dholakia