* feat(parallel_request_limiter_v3.py): allows admin to enforce token rate limit based on just output tokens
Useful when trying to rate limit for primarily self hosted model use-cases
* test(test_parallel_request_limiter_v3.py): add unit test for token rate limit type
* feat(parallel_request_limiter_v3.py): return remaining token limits in header
* feat: return rate limit headers in response
* feat(parallel_request_limiter_v3.py): working rate limit response headers
* feat(parallel_request_limiter_v3.py): fix rate limit tracking for tpm when rpm also set
* feat(parallel_request_limiter_v3.py): show headers for key/user/team
* feat(parallel_request_limiter_v3.py): decrement max parallel request limiter on failure event
* feat(parallel_request_limiter_v3.py): add in-memory cache implementation of parallel request rate limiter
allows rate limiter to work even without redis cache setup
Work for GA of parallel request limiter v3
* refactor(proxy/hooks/__init__.py): replace with new parallel request handler
* test: update testing
* fix: fix ruff check
* fix: revert ga of multi instance rate limiting - needs more work to pass testing
* Checkpoint before follow-up message
* Add comprehensive tests for Deepgram transcription functionality
* clean up transform
* just use 1 test
* test cleanup
* test fix get_complete_url
* test rename file
* refactor deepgram URL construction
* add logging_obj.pre_call
* fix unused imports
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Add Deepgram provider to supported providers list and mappings
* add logo
* Add deepgram to model cost map
* ui - require api key for deepgram
* fix logo path
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* fix: passing litellm_params_dict instead of litellm_params which is a list of strings
* updated custom llm embedding params and litellm_params_dict
---------
Co-authored-by: Ganesh Jagadeesan <cjagadeesan@dataminr.com>
* fix(internal_user_endpoints.py): support user with `+` in email on user info
ensures user is correctly parsed from input
* fix(factory.py): support vertex function call args as None
handles empty string in args for vertex gemini calls
* docs(langfuse_integration.md): pin langfuse sdk version on docs
* fix(vertex_ai/): return empty dict, instead of none when empty string given
* refactor: reduce function size
* fix: fix linting errors
* fix: revert check
* fix(internal_user_endpoints.py): fix check
* test: update tests
* test: update tests
* fix(vertex_ai/anthropic): extract vertex credentials from correct dict
Fixes passing credentials for vertex anthropic via `/v1/messages`
* fix(vertex_ai/anthropic): working e2e vertex ai anthropic messages via `/v1/messages`
Ensures vertex anthropic models can be called via /v1/messages
* fix(vertex_passthrough_logging_handler.py): fix vertex anthropic logging to emit the model name on spend logs
enables observability of passthrough route
* fix(vertex_passthrough_logging_handler.py): don't redefine variable
* fix(vertex_passthrough_logging_handler.py): log cost + token usage for anthropic passthrough requests on vertex ai
* fix(vertex_ai/anthropic): extract vertex credentials from correct dict
Fixes passing credentials for vertex anthropic via `/v1/messages`
* fix(vertex_ai/anthropic): working e2e vertex ai anthropic messages via `/v1/messages`
Ensures vertex anthropic models can be called via /v1/messages
* fix(vertex_passthrough_logging_handler.py): fix vertex anthropic logging to emit the model name on spend logs
enables observability of passthrough route
* fix(vertex_passthrough_logging_handler.py): don't redefine variable
* build(model_prices_and_context_window.json): add o3-pro pricing
* build(model_prices_and_context_window.json): add updated o3 model pricing
* build(model_prices_and_context_window.json): add new o3-pro model version
* chore(pangea-guardrail): Fix typo in debug message.
* docs(pangea-guardrail): Fix YAML example in pangea.md (README)."
* docs(pangea-guardrail): Update pangea.md (README).
* chore(pangea-guardrail): Format with Black.
* build(model_prices_and_context_window.json): add 'claude-opus-4' on vertexai (no @)
* build(model_prices_and_context_window.json): add claude sonnet 4 without 2
*@
* feat(internal_user_endpoints.py): assign user to orgs on user creation
allows user to be a member of orgs on creation - work to enable default orgs on UI
* fix(internal_user_endpoints.py): fix http_request
* docs(deploy.md): move docker recommendation to `main-stable`
* feat(enterprise/internal_user_endpoints.py): expose endpoint for checking available premium users
* feat(usage_indictor.tsx): add new element to help track remaining premium users
* feat(usage_indicator.tsx): show premium user remaining usage
allows users with user caps to know how much is left
* fix(vertex_and_google_ai_studio_gemini.py): bubble up stream is not finished, even if stop reason is given
prevents early completion of stream
Closes https://github.com/BerriAI/litellm/issues/11549
* fix(streaming_handler.py): respect is_finished = False in hidden params
internal logic for preventing ending stream early
* fix(litellm_license.py): add function to check if user is over limit
* fix(internal_user_endpoints.py): add function to check if user is over limit
* refactor: move test
* docs(customer_endpoints.py): document new param
* Feature/lasso guardrail (#9002)
* first version of lasso guardrail in litellm
* update to the new Lasso API
* change prod api_base and kill the request when lasso detect issue.
* change test for now api, local test pass
* add async tests
* all tests pass
* add docs for the new lasso guardrail
* Remove support for modes other than pre_call in Lasso guardrail
* code structure and naming
* only pre_call docs
* fix lint errors
* move test to the new location follows the same directory structure as litellm/.
* add lasso guard
* docs lasso docs
* add lasso guardrail
* fix lasso guardrail
---------
Co-authored-by: oroxenberg <oro@lasso.security>
Krrish Dholakia