Ishaan JaffandGitHub 163529b40b (feat - Router / Proxy ) Allow setting budget limits per LLM deployment (#7220) ...

* fix test_deployment_budget_limits_e2e_test

* refactor async_log_success_event to track spend for provider + deployment

* fix format

* rename class to RouterBudgetLimiting

* rename func

* rename types used for budgets

* add new types for deployment budgets

* add budget limits for deployments

* fix checking budgets set for provider

* update file names

* fix linting error

* _track_provider_remaining_budget_prometheus

* async_filter_deployments

* fix model list passed to router

* update error

* test_deployment_budgets_e2e_test_expect_to_fail

* fix test case

* run deployment budget limits

2024-12-13 19:15:51 -08:00

..

my-website

(feat - Router / Proxy ) Allow setting budget limits per LLM deployment (#7220)

2024-12-13 19:15:51 -08:00