* Fix user-team association issues in LiteLLM proxy
- Update list_team function to properly filter teams using user's teams array instead of only checking members_with_roles field
- Add Field descriptions and docstring to TeamMemberAddRequest and related models for better Swagger/OpenAPI documentation
- Maintain backward compatibility with fallback to members_with_roles if user lookup fails
This ensures users created with teams parameter appear correctly in team views and improves API documentation.
* Fix duplicate member checking in team_member_add endpoint
- Enhanced team_member_add_duplication_check to check both user_id and user_email
- Added additional duplicate prevention logic after user creation/lookup
- Fixed issue where users added by email could be duplicated in teams
- Added logging for debugging duplicate detection
This addresses the bug where adding the same user by email multiple times would create duplicate entries in the team's members_with_roles array.
* Improve duplicate member prevention in team_member_add
- Enhanced early duplicate check to handle both user_id and user_email
- Added late-stage duplicate prevention after user lookup/creation
- Fixed issue where users could be added multiple times by email
- Cleaned up debug logging
Note: There's still an edge case where the duplicate prevention may not work
correctly in all scenarios. This needs further investigation and testing.
* Refactor team_member_add endpoint for improved member management
- Split team_member_add functionality into smaller, dedicated functions for permission validation, member processing, and team member list updates.
- Enhanced permission checks to ensure only authorized users can add members.
- Streamlined member addition logic to reduce redundancy and improve readability.
- Maintained existing functionality while improving code structure and maintainability.
* Add tests for team_member_add helper functions
- Add test for _validate_team_member_add_permissions with admin user
- Add test for _validate_team_member_add_permissions with non-admin user
- Add test for _process_team_members with single member
- Add test for _process_team_members with multiple members
- Add test for _update_team_members_list with new member
- Add test for _update_team_members_list duplicate prevention
These tests ensure the refactored helper functions work correctly
after fixing the PLR0915 linting error.
* fix(llm_http_handler.py): raise clearer error on anthropic unified route
Fixes https://github.com/BerriAI/litellm/issues/12063
* fix(key_management_endpoints.py): add new param `new_key` for setting the regenerated key value
user request
* test: add unit tests
* fix(pass_through_endpoints.py): use data instead of json for passthrough requests
fixes bedrock latency issue
* Revert "fix(pass_through_endpoints.py): use data instead of json for passthrough requests"
This reverts commit 021dfd9165f837d37e6aad247ccbd0d2e8ca6043.
The test was failing because the managed_files hook was not configured.
This fix adds a DummyManagedFiles class and mocks the create_file
function as an AsyncMock to ensure proper test execution.
* test(test_router.py): initial unit test confirming router.afile_content uses dynamic api key / api base
* fix(managed_files.py): filter deployments for only those within file id mapping
ensure call works - only route to models where the file was written
* fix(proxy_server.py): fix loading in model ids from config, if config id is int
* fix(router.py): return all model file id mappings on create_file
if multiple deployments - this ensures all the file id mappings are bubbled up
Fixes issue when trying to use loadbalanced deployments - only 1 file id mapping was being stored
* fix(team_endpoints.py): support setting default key expiry
allows admin to set key expiry on all team member keys
makes it easier to setup default team for experimentation
* feat(key_management_endpoints.py): allows admin to set duration for keys created by team members
* feat(team_endpoints.py): support team_member_key_duration on `/team/update`
allows setting max time team member keys are valid for
* fix(team_info.tsx): ui component to update team member key duration
* fix(team_info.tsx): support updating team member key duration, if set
* feat(teams.tsx): add team member key duration param ui component
allow admin to set this on UI
* feat(ui_sso.py): support restricting ui access by sso group
allows controlling who can/can't access the UI
* feat(ssomodals.tsx): add initial commit adding sso group access to admin ui
* feat(proxy_server.py): support reading + writing ui_access_mode from db
allows admin to configure allowed sso groups from UI
* feat(ui_sso.py): support enforcing all teams on sso jwt handler
if ui access mode set via ui, support reading the value and enforcing it
* feat(ui/): ui component for controlling sso access group
allow admin to only allow users within specific sso group to log into UI
* fix(uiaccesscontrolform.tsx): fix field names
* feat(ui_sso.py): return received sso response in the clientside error message - enables easier debugging
* test: add unit tests
* fix: minor fixes
* SCIM fix new user roles
* test_create_user_defaults_to_viewer
* test_create_user_uses_default_internal_user_params_role
* fix default user for SCIM
* fix linting error
* fix(internal_user_endpoints.py): don't apply default internal user params if role is admin
prevent internal user restrictions from being applied to admin
* fix(proxy_server.py): fix model info v2 endpoint check - handle user_id being none
* fix(team_endpoints.py): ensure team doesn't lose all model access if set as empty string and new team model added
* fix(proxy_server.py): ensure model with team id is only added as valid for team which has that id
* fix(proxy_cli.py): check for module not found error on proxy import
Fixes https://github.com/BerriAI/litellm/issues/11836
* feat(proxy_server.py): utility function to get all models across all teams user is in
Allows user to see all team models on UI
* feat(proxy_server.py): return models accessible via team id's in `/v2/model/info`
Allows UI to tell user which team they can use to access model
* feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub
allows user to know what teams they can access a model through
* Revert "feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub"
This reverts commit f844c79383ec6739ed712f59e33a524a26b3d35a.
* fix(proxy_server.py): backend model info endpoint improvements
* UI Improvements for Default User access (#11952)
* feat(ui/): add a 'current team' and 'view' filters to the models page
allow user to see what all models they have access to within a specific team
* feat: working ui for seeing models in teams
* fix(model_dashboard.tsx): make current team filter more prominent
* style(model_dashboard.tsx): add a helpful note telling user how to create a model for the team they've selected
* style(model_dashboard.tsx): only show helpful note when current view is team, not for global
* fix(team_dropdown.tsx): allow searching by team id on create key modal
* feat(create_key_button.tsx): add helpful message when team selection is required
* fix: fix linting checks
* fix: fix ui linting error
* docs(team_endpoints.py): document new param
commit 440bc027251d8180174d762d83d271d0f7b68cc5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 23:04:11 2025 -0700
fix: fix check
commit 89a7451cb9ee26ff9f642335714dcc6f449d1fc2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 22:42:30 2025 -0700
fix: fix test
commit 1322e3b3497e5d334fdcaa18f0cf7a98ea758df4
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 20:52:40 2025 -0700
style: add more tooltips
commit 172738b98b7864aabcacf3334a394098b300283f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 20:51:09 2025 -0700
feat(team_member_view.tsx): add a tooltip
commit 895eb28deb9127985e30b5e859e5bca8530951c9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:46:49 2025 -0700
fix(teams.tsx): support setting team member budget on create
commit 003cc54a6dd0f65030c4f39a8487adc771b62e11
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:40:49 2025 -0700
fix(team_member_view.tsx): style improvements
commit a627a044f21df788f80d92a4081212072be91632
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:40:01 2025 -0700
fix(team_member_view.tsx): handle scientific notation in string
commit c5a3b7bd8419f6394e1b490849555d02d473baed
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:34:25 2025 -0700
feat(team_membership_view.tsx): show team member spend + max budget on UI
commit e986d12ad5b07c676f4cac5e16745939d7473dee
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:28:06 2025 -0700
feat(team_member_view.tsx): show team member spend + budget on team info
commit 8e398607b25f8a8f0bab41964810b5dd27c5e3f2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:18:16 2025 -0700
feat(team_info.tsx): show team member budget on team info
commit 1f56886b5913dafefc0c00fbe741c0c9c01144a6
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:15:30 2025 -0700
feat(team_endpoints.py): get team budget table on team info
allows user to see max budget set for team members
commit 0a4320bbfa406c24ad32a420f82152da7bdd7323
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:10:06 2025 -0700
feat(team_endpoints.py): return team member budget on team info
allows ui to display this to admin / team member
commit 6a4e29f87b333ae9977e8f878960e63becd89150
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 17:57:20 2025 -0700
fix(team_endpoints.py): support updating team budget on UI
commit 53f0fff34032977433dfe6935ce0a684a4141fd8
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 17:38:17 2025 -0700
feat(proxy/_types.py): return team member spend
update pydantic object to include spend
Allows showing spend of team member within team on UI
commit ef2a1a43ecf7fecfb904042cbf47b3d56246edcb
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 16:31:42 2025 -0700
feat(team_endpoints.py): support 'team_member_budget' param on `/team/update`
enables budget working across all team members
commit 512999f1249b00a02a30f049a0cfa36e829ff989
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 16:20:04 2025 -0700
test: add unit tests for default team member budget
commit 90fa3f61a2d63e12b9f3e1da9775f5c8b7294b5f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 15:37:51 2025 -0700
feat(team_endpoints.py): support using default team member budget id, if set
allows all team members to use the same budget id
commit acef5324b1a0935a482c71060f610c3d8823e8c3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 15:22:30 2025 -0700
feat(team_endpoints.py): support `team_member_budget` param on `/team/new`
Allow creating 1 budget for all users within team (makes it easier to increase/reduce budget if needed for all team members)
commit 2e867ac70fbd8768e7c27cf3b078e6dc10e566b9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 13:45:06 2025 -0700
fix(ui_sso.py): ensure user is added to team, if set via default internal settings
allows users signed up via SSO to be added to default team
* use ID for pass through management
* use id for pass through
* fix columns
* fix PassThroughInfoView
* cleanup
* working edit and delete pass through
* fix rendering id for pt row
* fixes for pt info view
* working delete pass through
* fix use NumericalInput
* fix alignment
* qa - creating pt
* show route preview
* fix show just 1 msg
* test_create_pass_through_endpoint
* fix ui linting
* feat(delete callback): add api and ui for deleting logging callbacks
* hotfix(delete callback): hotfix lint error
* move test case to test_litellm folder
* fix test case to use mock prisma client
* fix(factory.py): handle qs:.. in mime type
Fixes https://github.com/BerriAI/litellm/issues/11839
* feat(litellm_proxy/): don't transform messages client-side
leave litellm proxy messages untouched - allow proxy to handle transformation
prevents double transformation
* feat(tag_management_endpoints.py): support adding models to tag by adding model_name
Closes https://github.com/BerriAI/litellm/issues/11884
* test(test_tag_management_endpoints.py): add unit tests for adding new model by public model name
* test: update test
* feat(internal_user_endpoints.py): allow passing team-specific details when creating new user with team
allows assigning users to teams on creation
* feat(internal_user_endpoints.py): support for setting team values in default internal user params
Ensures admin can assign all new users to a default team
* test(test_internal_user_endpoints.py): add unit tests
* feat(ui/): allow setting default team on admin UI
easier onboarding for users
* fix(internal_user_endpoints.py): fix exception check
* fix SCIM memberships Patch
* fixes for SCIM updates
* fixes for SCIM
* working provisioning for teams on SCIM
* working user patch / PUT ops SCIM
* fixes SCIM
* test_scim_v2_endpoints.py
* handle_existing_user_by_email
* fixes for provisioning SCIMUser
* fixes SCIM provisioning
* test scim v2
* fixes for linting
* fix _apply_patch_ops
* fixes code QA check for team membership checks
* use cost_per_request
* fix cost_per_request
* fixes cost_per_request
* fixes for cost per request for pass through
* ui fix param name
* fixes for _set_cost_per_request
* test cost per request pass through endpoints
* Update tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* tests pass through endpoints
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: add summarize parameter to /spend/logs endpoint for individual transaction logs
- Introduced a new `summarize` parameter to control data format when querying spend logs.
- `summarize=true` (default) returns aggregated data, while `summarize=false` provides individual transaction logs.
- Updated documentation and added tests to validate the new functionality.
* fix: wrap boto3.Session() with tracer for observability - Add tracer.trace wrapper around boto3.Session() call in _get_aws_region_name method - Ensures all boto3 initializations in base_aws_llm.py are properly instrumented - Fixes test_boto3_init_tracer_wrapping test failure - Maintains consistency with other boto3 calls in the same file
* fix(litellm_pre_call_utils.py): add user agent tags to spend logs in standard logging payload logic
avoid clash when tag based routing is enabled
* test: remove redundant test
* test: rename oidc test to run earlier
quicker debuging
* fix(azure.py): return more detailed error message
* fix(azure/common_utils.py): use default scope, if scope is none
fixes oidc test
* fix: always default to cognitiveservices.azure.com
* test: update test
* feat: Add health check functionality and endpoints
- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.
* test: Add unit tests for health check functionality
- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.
* Refactor health endpoint model ID handling and improve logging
- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.
* Refactor utility functions in proxy module for improved readability and error handling
- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.
* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness
- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.
* Fix disable_spend_updates method to handle None return value gracefully
- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.
* Refactor join_paths function in utils.py for improved path handling
- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.
* Enhance health check functionality and improve error handling
- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.
* Add health check methods for database interaction
- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.
* Refactor health check result saving to use typed arguments
- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.
* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.
* Implement response time validation and details cleaning in health check result saving
- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.
* Add health check utility functions and refactor existing endpoints
- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.
* Refactor health check tests for improved clarity and coverage
- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.
* Implement helper function for health check and database saving
- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.
* refactor: rename target_model parameter to model in health check function
* fix(handle_jwt.py): check user object, if jwt user is proxy admin
correctly return user role - if jwt user has role updated in UI
* test(test_handle_jwt.py): add unit test for passing correct user role
* feat(model_info_view.tsx): separate UI component for updating edit model component
* feat(model_info_view.tsx): allow updating model access group on UI
show all available access groups in ui component
* docs: minor fixes
* feat(ui_sso.py): allow admin to specify additional headers for sso provider
some sso providers require special headers to return a json response
* test(test_ui_sso.py): add unit tests to ensure custom headers are respect3ed
* docs(config_settings.md): document new header param
* fix(litellm_pre_call_utils.py): add spend tag tracking by user agent
allows checking spend for cli tools like claude code
* feat(litellm_pre_call_utils.py): track spend by user agent part if user agent contains "/"
allows tracking spend across user agent versions
Better cost tracking for claude cod
* test(test_litellm_pre_call_utils.py): add testing for pre call utils, user agent parsing
* fix: fix linting check
* feat: Add health check functionality and endpoints
- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.
* test: Add unit tests for health check functionality
- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.
* Refactor health endpoint model ID handling and improve logging
- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.
* Refactor utility functions in proxy module for improved readability and error handling
- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.
* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness
- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.
* Fix disable_spend_updates method to handle None return value gracefully
- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.
* Refactor join_paths function in utils.py for improved path handling
- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.
* Enhance health check functionality and improve error handling
- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.
* Add health check methods for database interaction
- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.
* Refactor health check result saving to use typed arguments
- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.
* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.
* Implement response time validation and details cleaning in health check result saving
- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.
* Add health check utility functions and refactor existing endpoints
- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.
* Refactor health check tests for improved clarity and coverage
- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.
* Implement helper function for health check and database saving
- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.
* feat(parallel_request_limiter_v3.py): allows admin to enforce token rate limit based on just output tokens
Useful when trying to rate limit for primarily self hosted model use-cases
* test(test_parallel_request_limiter_v3.py): add unit test for token rate limit type
* feat(parallel_request_limiter_v3.py): return remaining token limits in header
* feat: return rate limit headers in response
* feat(parallel_request_limiter_v3.py): working rate limit response headers
* feat(parallel_request_limiter_v3.py): fix rate limit tracking for tpm when rpm also set
* feat(parallel_request_limiter_v3.py): show headers for key/user/team
* feat(parallel_request_limiter_v3.py): decrement max parallel request limiter on failure event
* feat(parallel_request_limiter_v3.py): add in-memory cache implementation of parallel request rate limiter
allows rate limiter to work even without redis cache setup
Work for GA of parallel request limiter v3
* refactor(proxy/hooks/__init__.py): replace with new parallel request handler
* test: update testing
* fix: fix ruff check
* fix: revert ga of multi instance rate limiting - needs more work to pass testing
* fix(internal_user_endpoints.py): support user with `+` in email on user info
ensures user is correctly parsed from input
* fix(factory.py): support vertex function call args as None
handles empty string in args for vertex gemini calls
* docs(langfuse_integration.md): pin langfuse sdk version on docs
* fix(vertex_ai/): return empty dict, instead of none when empty string given
* refactor: reduce function size
* fix: fix linting errors
* fix: revert check
* fix(internal_user_endpoints.py): fix check
* test: update tests
* test: update tests
* docs(deploy.md): move docker recommendation to `main-stable`
* feat(enterprise/internal_user_endpoints.py): expose endpoint for checking available premium users
* feat(usage_indictor.tsx): add new element to help track remaining premium users
* feat(usage_indicator.tsx): show premium user remaining usage
allows users with user caps to know how much is left
* fix(vertex_and_google_ai_studio_gemini.py): bubble up stream is not finished, even if stop reason is given
prevents early completion of stream
Closes https://github.com/BerriAI/litellm/issues/11549
* fix(streaming_handler.py): respect is_finished = False in hidden params
internal logic for preventing ending stream early
* fix(litellm_license.py): add function to check if user is over limit
* fix(internal_user_endpoints.py): add function to check if user is over limit
* refactor: move test
* docs(customer_endpoints.py): document new param
