* feat(team_info.tsx): allow user to reassign team to another org
* style(team_info.tsx): fix org id styling
* feat(team_endpoints.py): add validation check before migrating team to another org
ensure model access, budgets and membership is respected
* fix(team_endpoints.py): update model migration validation to check if org has 'all-proxy-models' access
* fix(organization_view.tsx): show teams belonging to org
* feat(team_endpoints.py): handle wildcard model check on org migration
* fix(team_endpoints.py): nest router check
* test: update testing - use model with higher quota
* build: update poetry lock
* refactor KB implementation to use central registry
* allow passing tools when making KB calls
* test fixes
* linting fix
* fix kb tests
* QA for KB stored in DB
* fix, use litellm_credential_name when adding KB on litellm UI
* QA list endpoint vector stores
* allow using UI creds with KBs
* feat(provider_specific_fields.tsx): Use correct form instance
Fixes https://github.com/BerriAI/litellm/issues/10115
* Fix broken pagination by correctly passing page and pageSize to keyListCall (#10498)
* [Organization] Include litellm_budget_table in /organization/list response (#10488)
* join litellm_budget_table in /organization/list endpoint
* update test
---------
Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
* ensure vector store results are logged in SLP
* fix tests
* fix tests with vector_store_request_metadata
* fix linting
* track duration of vector store, only log content when user opts into it
* working vector store viewer
* fix custom llm provider - Vector Store Requests
* fix vector store viewer
* fix logging redacted vector searches
* testing for storing KB queries in DB
* fix slack alerting with webhooks
* emit correct event group/entity on webhooks
* refactor to use a common class of alerts with abc methods
* fixes for tests
* refactor to use a common class of alerts with abc methods
* Send a budget alert on slack or webhook
* unit test slack alerting
* fix code qa
* added tests
messages_with_counts: Made tolerance explicit for each test. But they match the new implementation(which beats the old)
* new token counter impl
* compare old and new implementation in test
* delete old token counter
* moved tests to /tests/litellm/litellm_core_utils
* use existing types
* docstrings
* warn about using default params on unknown model.
* created type for the token_counter_function
* check key == "content"
* throw error on invalid detail-type, ignore type-warning.
* fix imports
* fix(vertex_and_google_ai_studio.py): fix finish reason to be 'tool_calls' when tool call returned
Vertex returns 'Stop', openai format is 'tool calls'
* test(base_llm_unit_tests.py): bump test to assert tool calls in finish reason
* Update docs for OpenAI compatible providers, add Llamafile docs, include Llamafile in the sidebar
* Add Llamafile as an LlmProviders enum
* Add llamafile as a OpenAI compatible provider (in the list of compatible providers)
* Add Llamafile chat config and tests
* Wire up Llamafile
Co-authored-by: Peter Wilson <peter@mozilla.ai>
* fix(exception_mapping_utils.py): correctly pass through 504 status code
openai also raises a 504 status code
* build(model_prices_and_context_window.json): add gpt-4o-mini-tts to model cost map
Fixes https://github.com/BerriAI/litellm/issues/9591
* fix(cost_calculator.py): fix input cost calculation for gpt-4o-mini-tts
Fixes https://github.com/BerriAI/litellm/issues/9591
* test: testing updates
The client provides access to a low-level HTTP client for making direct
requests to the LiteLLM proxy server. This is useful when you need more
control or when working with endpoints that don't yet have a high-level
interface.
```python
In [2]: client.http.request(
...: method="POST",
...: uri="/health/test_connection",
...: json={
...: "litellm_params": {
...: "model": "gpt-4",
...: "custom_llm_provider": "azure_ai",
...: "litellm_credential_name": None,
...: "api_key": "6xxxxxxx",
...: "api_base": "https://litellm8397336933...",
...: },
...: "mode": "chat",
...: },
...: )
Out[2]:
{'status': 'error',
'result': {'model': 'gpt-4',
'custom_llm_provider': 'azure_ai',
'litellm_credential_name': None,
'api_base': 'https://litellm8397336933...',
...
```
* init vector store configs
* working kb init
* add vector store endpoints
* use litellm_credential_name
* working CRUD vector stores litellm
* working creds with vector DB
* ui cleanup
* clean up vector store id
* fix delete button
* refactored vector store component
* working selector for KBs
* ui vector stores
* add vector store tool calls usage on chat ui
* fixes for vector stores litellm
* test fix
* docs Knowledge Bases
* fixes for vector stores litellm
* fix linting
* add managed vectorstores
* fix orjson ci/cd test
* fix linting
* add types.tsx file
* fix: initial commit of v2 parallel request limiter hook
enables multi-instance rate limiting to work
* fix: subsequent commit with additional refactors
* fix(parallel_request_limiter_v2.py): cleanup initial call hook
simplify it
* fix(parallel_request_limiter_v2.py): working v2 parallel request limiter
* fix: more updates - still not passing testing
* fix(test_parallel_request_limiter_v2.py): update test + add conftest
* fix: fix ruff checks
* fix(parallel_request_limiter_v2.py): use pull via pattern method to load in keys instance wouldn't have seen yet
Fixes issue where redis syncing was not pulling key until instance had seen it
* test: update testing to cover tpm and rpm
* fix(parallel_request_limiter_v2.py): fix ruff errors
* fix(proxy/hooks/__init__.py): feature flag export
* fix(proxy/hooks/__init_.py): fix linting error
* ci(config.yml): add tests/enterprise to ci/cd
* fix: fix ruff check
* test: update testing
* build(model_prices_and_context_window.json): add fireworks ai new 0-4b pricing tier
* build(model_prices_and_context_window.json): add more fireworks ai models
* test: update testing
* fix(caching_handler.py): handle str + list cache
Fixes issue on cache hits for embedding when initial cached input was str
* test(test_caching.py): add e2e test on caching with individual item and then list
* fix(caching_handler.py): set usage tokens for cache hits
enables token counting to work
* fix(caching_handler.py): combine usage between cached result and embedding response
Handles case of new input to embedding response
* fix: cleanup
* test: move to gpt-4o-new-test
* test: update test
* Schedule budget resets at expectable times (#10331)
* Enhance budget reset functionality with timezone support and standardized reset times
- Added `get_next_standardized_reset_time` function to calculate budget reset times based on specified durations and timezones.
- Introduced `timezone_utils.py` to manage timezone retrieval and budget reset time calculations.
- Updated budget reset logic in `reset_budget_job.py`, `internal_user_endpoints.py`, `key_management_endpoints.py`, and `team_endpoints.py` to utilize the new timezone-aware reset time calculations.
- Added unit tests for the new reset time functionality in `test_duration_parser.py`.
- Updated `.gitignore` to include `test.py` and made minor formatting adjustments in `docker-compose.yml` for consistency.
* Fixed linting
* Fix for mypy
* Fixed testcase for reset
* fix(duration_parser.py): move off zoneinfo - doesn't work with python 3.8
* test: update test
* refactor: improve budget reset time calculation and update related tests for accuracy
* clean up imports in team_endpoints.py
* test: update budget remaining hours assertions to reflect new reset time logic
* build(model_prices_and_context_window.json): update model
---------
Co-authored-by: Prathamesh Saraf <pratamesh1867@gmail.com>
* fix(user_api_key_auth.py): fix passing `x-litellm-api-key` to user api key auth
Support using this when given, or bearer token when given
Fixes issue with auth on vertex passthrough
* test(test_user_api_key_auth.py): use new fastapi.security check
* fix(user_api_key_auth.py): allow key at budget, to still call non-llm api endpoints
Fixes issue where key at budget, couldn't call `/key/info`
* fix(provider_info_helpers.tsx): fix together ai provider name
* fix(user_edit_view.tsx): all admin to edit user's personal models
* fix(model_dashboard.tsx): fix model filtering by team id check
* fix(proxy_server.py): support returning available models for a user on `/model_group/info`
Fixes issue where personal key with 'all team models' would see all proxy models (not just the ones user can call)
* test(test_models.py): add unit test for model group info check on personal key
* fix(cost_calculator.py): support custom pricing for image gen
Allow user to set custom pricing on azure image gen models
* test(test_cost_calculator.py): add unit test
* test(test_litellm_logging.py): add more unit testing
* fix(litellm_logging.py): fix ruff check
* build(litellm-proxy-extras/utils.py): correctly generate baseline migration for non-empty db
* fix(litellm-proxy-extras/utils.py): Fix issue in migration, where if a migration fails during baselining, all are still marked as applied
* fix(prisma_client.py): don't pass separate schema.prisma to litellm-proxy-extras
use the one in litellm-proxy-extras
* fix(litellm-proxy-extras/utils.py): support passing custom dir for baselining db in read-only fs
Fixes https://github.com/BerriAI/litellm/issues/9885
* fix(utils.py): give helpful warning message when permission denied error raised in fs
* add session id in spendLogs
* fix log proxy server request as independant field
* use trace id for SpendLogs
* add _ENTERPRISE_ResponsesSessionHandler
* use _ENTERPRISE_ResponsesSessionHandler
* working session_ids
* working session management
* working session_ids
* test_async_gcs_pub_sub_v1
* test_spend_logs_payload_e2e
* working session_ids
* test_get_standard_logging_payload_trace_id
* test_get_standard_logging_payload_trace_id
* test_gcs_pub_sub.py
* fix all linting errors
* test_spend_logs_payload_with_prompts_enabled
* _ENTERPRISE_ResponsesSessionHandler
* _ENTERPRISE_ResponsesSessionHandler
* expose session id on ui
* get spend logs by session
* add sessionSpendLogsCall
* add session handling
* session logs
* ui session details
* fix on rowExpandDetails
* ui working sessions
Ishaan Jaff