* Enable Tool Calling for `meta_llama` (#11825)
* feat: enable tools and function_call features
* fix: ignore pydantic warnings for StreamingChoices from llama-api
* docs: add tool calling examples
* docs: change default models to Maverick
* docs: fix output of tool use
* test_map_openai_params
---------
Co-authored-by: Young Han <110819238+seyeong-han@users.noreply.github.com>
* feat(internal_user_endpoints.py): allow passing team-specific details when creating new user with team
allows assigning users to teams on creation
* feat(internal_user_endpoints.py): support for setting team values in default internal user params
Ensures admin can assign all new users to a default team
* test(test_internal_user_endpoints.py): add unit tests
* feat(ui/): allow setting default team on admin UI
easier onboarding for users
* fix(internal_user_endpoints.py): fix exception check
* fix(spend_tracking_utils.py): add user agent tags from standard logging payload, in spend logs payload
* feat(litellm_logging.py): identify user agent tags as `User-Agent: ..` and allow admin to disable storing user agent as tag
* fix(azure_ai/): pass content type header in azure ai request
Fixes https://github.com/BerriAI/litellm/issues/11227
* test: add unit test
* fix(router.py): fix passing dynamic credentials to retrieve batch
Fixes batch retrieval when using router
* test: add more unit tests
* use cost_per_request
* fix cost_per_request
* fixes cost_per_request
* fixes for cost per request for pass through
* ui fix param name
* fixes for _set_cost_per_request
* test cost per request pass through endpoints
* Update tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* tests pass through endpoints
* rename left nav
* move pass through to models + endpoints
* move pass through under models
* add pt info view
* add get/update pt
* fixes for PT ui
* allow updating pt endpoints
* fix pass through quick actions
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix SCIM memberships Patch
* fixes for SCIM updates
* fixes for SCIM
* working provisioning for teams on SCIM
* working user patch / PUT ops SCIM
* fixes SCIM
* test_scim_v2_endpoints.py
* handle_existing_user_by_email
* fixes for provisioning SCIMUser
* fixes SCIM provisioning
* test scim v2
* fixes for linting
* fix _apply_patch_ops
* fixes code QA check for team membership checks
* use cost_per_request
* fix cost_per_request
* fixes cost_per_request
* fixes for cost per request for pass through
* ui fix param name
* fixes for _set_cost_per_request
* test cost per request pass through endpoints
* Update tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* tests pass through endpoints
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: add summarize parameter to /spend/logs endpoint for individual transaction logs
- Introduced a new `summarize` parameter to control data format when querying spend logs.
- `summarize=true` (default) returns aggregated data, while `summarize=false` provides individual transaction logs.
- Updated documentation and added tests to validate the new functionality.
* fix: wrap boto3.Session() with tracer for observability - Add tracer.trace wrapper around boto3.Session() call in _get_aws_region_name method - Ensures all boto3 initializations in base_aws_llm.py are properly instrumented - Fixes test_boto3_init_tracer_wrapping test failure - Maintains consistency with other boto3 calls in the same file
* fix(litellm_pre_call_utils.py): add user agent tags to spend logs in standard logging payload logic
avoid clash when tag based routing is enabled
* test: remove redundant test
* test: rename oidc test to run earlier
quicker debuging
* fix(azure.py): return more detailed error message
* fix(azure/common_utils.py): use default scope, if scope is none
fixes oidc test
* fix: always default to cognitiveservices.azure.com
* test: update test
* feat: Implement health check functionality for models
- Added individual model health check API call and integrated it into the dashboard.
- Introduced health check history and latest health checks retrieval.
- Enhanced model dashboard to display health status, last check time, and error details.
- Created a new health check columns component for better organization of health data.
- Added error pattern handling for meaningful error messages.
- Updated UI components to support health check actions and display loading states.
* refactor: extract health check functionality into a new component
- Removed individual health check logic from ModelDashboard and encapsulated it within HealthCheckComponent for better modularity and maintainability.
- Updated ModelDashboard to utilize the new HealthCheckComponent, streamlining the health check process and improving code organization.
- Cleaned up unused state variables and imports related to health checks in ModelDashboard.
- Enhanced error handling and status management within the new component, ensuring a more robust health check experience.
* fix: add access token check in HealthCheckComponent to prevent unauthorized API calls
- Implemented a check for the presence of an access token before making API calls to fetch the latest health checks, enhancing security and preventing potential errors when the token is missing.
* Fix: #11853
Updated alerting.md to call the latest model for adding metadata.
* Fix: #11856
Updated billing docs to call newer GPT model for Langchain and CURL
* feat: Add health check functionality and endpoints
- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.
* test: Add unit tests for health check functionality
- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.
* Refactor health endpoint model ID handling and improve logging
- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.
* Refactor utility functions in proxy module for improved readability and error handling
- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.
* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness
- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.
* Fix disable_spend_updates method to handle None return value gracefully
- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.
* Refactor join_paths function in utils.py for improved path handling
- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.
* Enhance health check functionality and improve error handling
- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.
* Add health check methods for database interaction
- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.
* Refactor health check result saving to use typed arguments
- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.
* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.
* Implement response time validation and details cleaning in health check result saving
- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.
* Add health check utility functions and refactor existing endpoints
- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.
* Refactor health check tests for improved clarity and coverage
- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.
* Implement helper function for health check and database saving
- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.
* refactor: rename target_model parameter to model in health check function
Added the `aws_region_name` and `aws_role_name` keys to the definition of a guardrail config.
Without these you will get the error `{\"message\":\"The guardrail identifier or version provided in the request does not exist.\"}`
