Krish Dholakia 711601e22a Add key-level multi-instance tpm/rpm/max parallel request limiting (#10458) ...

* fix: initial commit of v2 parallel request limiter hook

enables multi-instance rate limiting to work

* fix: subsequent commit with additional refactors

* fix(parallel_request_limiter_v2.py): cleanup initial call hook

simplify it

* fix(parallel_request_limiter_v2.py): working v2 parallel request limiter

* fix: more updates - still not passing testing

* fix(test_parallel_request_limiter_v2.py): update test + add conftest

* fix: fix ruff checks

* fix(parallel_request_limiter_v2.py): use pull via pattern method to load in keys instance wouldn't have seen yet

Fixes issue where redis syncing was not pulling key until instance had seen it

* test: update testing to cover tpm and rpm

* fix(parallel_request_limiter_v2.py): fix ruff errors

* fix(proxy/hooks/__init__.py): feature flag export

* fix(proxy/hooks/__init_.py): fix linting error

* ci(config.yml): add tests/enterprise to ci/cd

* fix: fix ruff check

* test: update testing

2025-04-30 21:32:31 -07:00

..

example_config_yaml

test: update tests to new deployment model (#10142)

2025-04-18 14:22:12 -07:00

test_configs

test: update tests to new deployment model (#10142)

2025-04-18 14:22:12 -07:00

test_model_response_typing

…

adroit-crow-413218-bc47f303efc9.json

vertex testing use pathrise-convert-1606954137718

2025-01-05 14:00:17 -08:00

azure_fine_tune.jsonl

…

batch_job_results_furniture.jsonl

…

conftest copy.py

…

conftest.py

…

data_map.txt

…

eagle.wav

…

gettysburg.wav

…

large_text.py

…

messages_with_counts.py

…

model_cost.json

…

openai_batch_completions_router.jsonl

…

openai_batch_completions.jsonl

…

speech_vertex.mp3

…

test_aproxy_startup.py

(Security fix) - Upgrade to fastapi==0.115.5 (#7447)

2024-12-28 17:08:19 -08:00

test_audit_logs_proxy.py

(docs) add docstrings for all /key, /user, /team, /customer endpoints (#6804)

2024-11-18 19:44:06 -08:00

test_auth_checks.py

handle failed db connections

2025-03-25 23:14:44 -07:00

test_banned_keyword_list.py

…

test_custom_callback_input.py

…

test_db_schema_changes.py

test: initial test to enforce all functions in user_api_key_auth.py h… (#7797)

2025-01-15 21:52:45 -08:00

test_db_schema_migration.py

install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637)

2025-03-29 15:27:09 -07:00

test_deployed_proxy_keygen.py

…

test_e2e_pod_lock_manager.py

[Reliability fix] Redis transaction buffer - ensure all redis queues are periodically flushed (#10393)

2025-04-28 21:36:54 -07:00

test_jwt.py

feat(handle_jwt.py): support multiple jwt url's

2025-03-06 23:05:54 -08:00

test_key_generate_dynamodb.py

…

test_key_generate_prisma.py

Add key-level multi-instance tpm/rpm/max parallel request limiting (#10458)

2025-04-30 21:32:31 -07:00

test_proxy_config_unit_test.py

(fix) - proxy reliability, ensure duplicate callbacks are not added to proxy (#8067)

2025-01-28 21:01:56 -08:00

test_proxy_custom_auth.py

…

test_proxy_custom_logger.py

test: update tests to new deployment model (#10142)

2025-04-18 14:22:12 -07:00

test_proxy_encrypt_decrypt.py

…

test_proxy_exception_mapping.py

…

test_proxy_gunicorn.py

…

test_proxy_pass_user_config.py

test: update tests to new deployment model (#10142)

2025-04-18 14:22:12 -07:00

test_proxy_reject_logging.py

…

test_proxy_routes.py

[Bug fix]: Proxy Auth Layer - Allow Azure Realtime routes as llm_api_routes (#7684)

2025-01-10 20:38:06 -08:00

test_proxy_server_caching.py

…

test_proxy_server_keys.py

test: update tests to new deployment model (#10142)

2025-04-18 14:22:12 -07:00

test_proxy_server_langfuse.py

(Security fix) - Upgrade to fastapi==0.115.5 (#7447)

2024-12-28 17:08:19 -08:00

test_proxy_server_spend.py

…

test_proxy_server.py

test: update tests to new deployment model (#10142)

2025-04-18 14:22:12 -07:00

test_proxy_setting_guardrails.py

…

test_proxy_token_counter.py

Litellm dev 12 12 2024 (#7203)

2024-12-13 08:54:03 -08:00

test_proxy_utils.py

Merge branch 'main' into litellm_fix_service_account_behavior

2025-04-01 12:04:28 -07:00

test_unit_test_max_model_budget_limiter.py

Support budget/rate limit tiers for keys (#7429)

2024-12-26 19:05:27 -08:00

test_unit_test_proxy_hooks.py

test fixes

2025-03-31 19:33:10 -07:00

test_update_spend.py

test_long_term_spend_accuracy_with_bursts

2025-03-31 19:17:13 -07:00

test_user_api_key_auth.py

Support x-litellm-api-key header param + allow key at max budget to call non-llm api endpoints (#10392)

2025-04-29 18:57:57 -07:00

vertex_key.json

ci/cd update vertex acct

2025-01-05 13:43:32 -08:00