Files
litellm/tests/proxy_behavior/management/test_team_update.py
T
yuneng-jiang 5e16f20962 test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints (#28681)
* test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints

Extends the Phase 1–3 behavior-pin suite at tests/proxy_behavior/management/
with a second axis: payload-shape pinning. Phase 1–3 held payload minimal
and pinned (actor, target) → status across 37 routes; Phase 4 holds the
caller fixed at an authorized actor, varies the payload shape, and asserts
the observable DB effect (on accept) or the named guard / row-unchanged
(on reject). Faithfulness contract from Phase 1–3 is unchanged.

Six families + one gap-closer (59 new scenarios, 620 → 679 total):

  * F1 — key budget / rate-limit (test_key_budget_limits.py, 18)
  * F2 — key↔team reassignment   (test_key_team_change.py, 6)
  * F3 — team budget / rate-limit (test_team_budget_limits.py, 15)
  * F4 — member-info validation   (test_team_member_info_validation.py, 5)
  * F5 — permission batching      (test_team_permissions_bulk_update.py, 6)
  * F6 — org-scoped team access   (+2 detail-string pins in existing files)
  * F7 — coverage gap-closer      (test_f7_coverage_closeout.py, 7)

Harness extensions in conftest.py (additive only):
  * create_scratch_org() seeder with its own scratch-prefixed budget row
  * budget / limit fields on create_scratch_team()
  * scratch teardown also sweeps litellm_organizationtable

Coverage telemetry (behavior-suite-only):
  * key_management_endpoints.py  60 % → 65 % (+82 lines)
  * team_endpoints.py            62 % → 72 % (+137 lines, crosses 70 % stretch)

Key lands under 70 % per plan §7 escape hatch — the gap is dominated by
routes outside F1–F6 scope (key list/info v2 internals) and structurally
dead org-budget guards (call sites at lines 889 + 2310 + 985 + 1751 load
the org without include_budget_table=True, so org.litellm_budget_table is
None at guard time and the aggregate guard no-ops). Pinned as observed
no-op behavior so a future fix that flips the flag turns these into reds.

Zero source-code changes; pyproject.toml diff is empty;
test_route_coverage.py stays green untouched; G3 grep guards still green;
local wall-time 14 s for the full suite (no coverage), 22 s with coverage.

G4 regression-replay protocol executed against three representative
fix-PR parents (410ce761dc, 0bd49ecb8b, 8bbc61e03c): all Phase 4 tests
PASS at pre-fix SHAs — confirming the F1–F7 layer is a helper-body pin,
not a regression-replay layer for those specific historical bypass
shapes. Targeted RED-bait scenarios for each fix are left for a
follow-up PR.

* test(proxy): push key_management_endpoints.py past the 70% stretch (F7-extension)

Adds 24 more payload-pin scenarios in test_f7_key_coverage_push.py
following the same accepted-effect / rejected-guard pattern. Each
scenario cites the file:line range it pins; same anti-snapshot rules
apply.

Target ranges (all reachable via HTTP-boundary payload variation):
  * 5942-6063  /key/health with metadata.logging → test_key_logging body
  * 4565-4692  /key/reset_spend happy + 404 + non-admin gate + value validation
  * 4421-4533  /key/regenerate ghost-404 + happy + new_key + grace_period
  * 4168-4202  _insert_deprecated_key body via grace_period
  * 6118-6133  _enforce_unique_key_alias duplicate-alias rejection
  * 6148-6169  validate_model_max_budget malformed-payload rejection
  * 4708-4789  validate_key_list_check user/team/org/key_hash branches
  * 2622-2733  /key/bulk_update mixed success/failure + admin gate + size limits
  * 2797-2950  /team/key/bulk_update all-keys path + explicit-keys dedupe + 404
  * 5108-5207  /key/aliases admin + scoped + search-filter branches
  * 3253-3303  /key/info ghost + explicit-key + no-key-uses-auth-header
  * 3427-3436  generate_key_helper_fn budget_limits initialization
  * 1794-1815  prepare_key_update_data duration + budget_duration paths
  * 5280-5388  _build_filter_conditions across include_created_by_keys/team/sort/alias

Coverage telemetry — full PR4 dataset:

  key_management_endpoints.py: 60 % → 71 %  (+11 pts, +194 lines)
  team_endpoints.py:           62 % → 72 %  (+10 pts, +137 lines)

Both files now over the plan §7 PR4.M4 70 % stretch as a side effect of
pinning real payload behavior. 721 tests pass in 19 s local (full suite,
no coverage); 27 s with coverage. Zero source-code changes; pyproject.toml
diff still empty; test_route_coverage.py + G3 grep guards still green.

Honest finding (kept from the prior commit's body): four structurally-dead
org-budget guards remain pinned as observed no-op behavior — they fire
only when get_org_object is called with include_budget_table=True, which
none of the four management-endpoint call sites currently do. Pinned so
a future change that flips the flag turns these into reds.

Two helper guards are honest-ceiling: _validate_reset_spend_value's
isinstance check at line 4568 is unreachable from HTTP because Pydantic
422s non-float before the helper runs; same shape for /team/key/bulk_update's
missing team_id / no-selector pre-handler guards.

* test(proxy): address PR review — try/finally cleanup + loosen 500 envelope pins + Optional annotations

Greptile review feedback on PR #28681:

1. Wrap manual budget-row cleanup in try/finally so an assertion failure
   doesn't leave non-scratch-prefixed budget rows orphaned across CI re-runs
   (test_team_new_with_team_member_budget_creates_budget_row and
   test_team_update_team_member_budget_upserts).
2. Loosen the two 500-status pins to in (400, 422, 500) — the named-guard
   substring is the real pin; the outer ValueError-wrap envelope is an
   implementation detail that a future improvement should be free to fix
   to a proper 400/422 without flipping these tests red.
3. Add missing Optional annotations on _seed_token's max_budget / metadata
   / team_id keyword args (they default to None).

Greptile's typo flag on 'read-world' in the conftest comment is declined —
'read-world' is the project's established term for the immutable seeded
world fixture (see other usages in conftest.py and actors.py).

721 tests still pass in 17 s.
2026-05-23 12:16:29 -07:00

228 lines
8.4 KiB
Python

import pytest
from litellm.proxy._types import LitellmUserRoles
from .actors import Actor
from .conftest import create_scratch_actor, create_scratch_team
pytestmark = pytest.mark.asyncio(loop_scope="session")
# POST /team/update — actor x team-shape matrix (shapes built by _seed_target).
# Each request carries the team's own organization_id so a non-proxy-admin can
# reach the org-scoped branch of the route-permission gate (401 on denial),
# which fronts the handler's _verify_team_access. Only PROXY_ADMIN and an
# ORG_ADMIN of the team's org pass: an internal_user team admin is filtered by
# the route gate before _verify_team_access's team-admin branch is reached.
MARKER_ALIAS = "behavior-pin-update-marker-alias"
_MATRIX = [
("alpha/proxy_admin", Actor.PROXY_ADMIN, "alpha", 200),
("alpha/org_admin", Actor.ORG_ADMIN, "alpha", 200),
("alpha/team_admin", Actor.TEAM_ADMIN, "alpha", 401),
("alpha/internal_user", Actor.INTERNAL_USER, "alpha", 401),
("alpha/owner", Actor.OWNER, "alpha", 401),
("alpha/unrelated_same_org", Actor.UNRELATED_SAME_ORG, "alpha", 401),
("alpha/cross_org_user", Actor.CROSS_ORG_USER, "alpha", 401),
("alpha/service_account", Actor.SERVICE_ACCOUNT, "alpha", 401),
("alpha/org_b_admin", Actor.ORG_B_ADMIN, "alpha", 401),
("beta/proxy_admin", Actor.PROXY_ADMIN, "beta", 200),
("beta/org_admin", Actor.ORG_ADMIN, "beta", 401),
("beta/team_admin", Actor.TEAM_ADMIN, "beta", 401),
("beta/internal_user", Actor.INTERNAL_USER, "beta", 401),
("beta/owner", Actor.OWNER, "beta", 401),
("beta/unrelated_same_org", Actor.UNRELATED_SAME_ORG, "beta", 401),
("beta/cross_org_user", Actor.CROSS_ORG_USER, "beta", 401),
("beta/service_account", Actor.SERVICE_ACCOUNT, "beta", 401),
("beta/org_b_admin", Actor.ORG_B_ADMIN, "beta", 200),
]
async def _seed_target(prisma, world, shape: str, team_id: str) -> str:
"""Raw-seed the scratch target team; returns its organization_id."""
if shape == "alpha":
await create_scratch_team(
prisma,
team_id,
organization_id=world.org_a_id,
admin_user_ids=[world.keys[Actor.TEAM_ADMIN].user_id],
member_user_ids=[
world.keys[Actor.INTERNAL_USER].user_id,
world.keys[Actor.OWNER].user_id,
world.keys[Actor.UNRELATED_SAME_ORG].user_id,
world.keys[Actor.SERVICE_ACCOUNT].user_id,
],
)
return world.org_a_id
if shape == "beta":
await create_scratch_team(
prisma,
team_id,
organization_id=world.org_b_id,
member_user_ids=[world.keys[Actor.CROSS_ORG_USER].user_id],
)
return world.org_b_id
pytest.fail(f"unknown shape={shape}") # pragma: no cover
@pytest.mark.parametrize(
"actor,shape,expected_status",
[(a, sh, s) for (_id, a, sh, s) in _MATRIX],
ids=[s[0] for s in _MATRIX],
)
async def test_team_update_authz_matrix(
actor: Actor,
shape: str,
expected_status: int,
proxy_client,
prisma,
scratch,
world,
):
org_id = await _seed_target(prisma, world, shape, scratch.prefix)
caller = world.keys[actor]
resp = await proxy_client.post(
"/team/update",
headers={"Authorization": f"Bearer {caller.cleartext}"},
json={
"team_id": scratch.prefix,
"team_alias": MARKER_ALIAS,
"organization_id": org_id,
},
)
assert (
resp.status_code == expected_status
), f"{actor.value} {shape}: {resp.status_code} {resp.text}"
row = await prisma.db.litellm_teamtable.find_unique(
where={"team_id": scratch.prefix}
)
assert row is not None
if expected_status == 200:
assert row.team_alias == MARKER_ALIAS
else:
assert row.team_alias != MARKER_ALIAS, "denied but team mutated"
async def test_team_update_requires_proxy_admin_without_org_context(
proxy_client, prisma, scratch, world
):
"""With no organization_id in the body the route gate has no org context
and falls back to proxy-admin-only: an org admin of the team's own org
is 401, PROXY_ADMIN is 200."""
await _seed_target(prisma, world, "alpha", scratch.prefix)
denied = await proxy_client.post(
"/team/update",
headers={"Authorization": f"Bearer {world.keys[Actor.ORG_ADMIN].cleartext}"},
json={"team_id": scratch.prefix, "team_alias": MARKER_ALIAS},
)
assert denied.status_code == 401, denied.text
allowed = await proxy_client.post(
"/team/update",
headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"},
json={"team_id": scratch.prefix, "team_alias": MARKER_ALIAS},
)
assert allowed.status_code == 200, allowed.text
# Relocation gate — moving a team to a different org. The scratch team starts
# in ORG_A; each scenario relocates it to ORG_B. PROXY_ADMIN bypasses;
# ORG_B_ADMIN clears the route gate (dest-org admin) but fails
# _verify_team_access on the source team (403); the rest fail the route gate
# (401). The relocation-*allowed* branch (caller is org admin of both orgs) is
# covered by test_team_update_org_relocation_allowed_for_dual_org_admin below.
_RELOCATION = [
("proxy_admin", Actor.PROXY_ADMIN, 200),
("org_b_admin", Actor.ORG_B_ADMIN, 403),
("org_admin", Actor.ORG_ADMIN, 401),
("team_admin", Actor.TEAM_ADMIN, 401),
("internal_user", Actor.INTERNAL_USER, 401),
]
@pytest.mark.parametrize(
"actor,expected_status",
[(a, s) for (_id, a, s) in _RELOCATION],
ids=[s[0] for s in _RELOCATION],
)
async def test_team_update_org_relocation_gate(
actor: Actor,
expected_status: int,
proxy_client,
prisma,
scratch,
world,
):
await _seed_target(prisma, world, "alpha", scratch.prefix)
caller = world.keys[actor]
resp = await proxy_client.post(
"/team/update",
headers={"Authorization": f"Bearer {caller.cleartext}"},
json={"team_id": scratch.prefix, "organization_id": world.org_b_id},
)
assert (
resp.status_code == expected_status
), f"{actor.value}: {resp.status_code} {resp.text}"
row = await prisma.db.litellm_teamtable.find_unique(
where={"team_id": scratch.prefix}
)
assert row is not None
if expected_status == 200:
assert row.organization_id == world.org_b_id
else:
assert row.organization_id == world.org_a_id, "denied but team relocated"
# Phase 4 F6 — explicit pin on the `_verify_team_access` 403 detail string
# when an org_admin clears the destination route gate but fails the source
# team's org-membership check. The relocation matrix above covers the
# status; this guard turns a silent rename of the helper's exception detail
# into a CI red.
async def test_team_update_org_b_admin_relocation_rejection_detail(
proxy_client, prisma, scratch, world
):
await _seed_target(prisma, world, "alpha", scratch.prefix)
resp = await proxy_client.post(
"/team/update",
headers={"Authorization": f"Bearer {world.keys[Actor.ORG_B_ADMIN].cleartext}"},
json={"team_id": scratch.prefix, "organization_id": world.org_b_id},
)
assert resp.status_code == 403, resp.text
assert "do not have access to this team" in resp.text, resp.text
async def test_team_update_org_relocation_allowed_for_dual_org_admin(
proxy_client, prisma, scratch, world
):
"""Relocation-allowed branch: a caller who is org admin of BOTH the source
and destination org may relocate a team between them. Completes the
_RELOCATION matrix, whose allowed branch PR2 left open — no seeded actor is
a dual-org admin, so one is minted with create_scratch_actor."""
actor = await create_scratch_actor(
prisma,
scratch.prefix,
user_role=LitellmUserRoles.ORG_ADMIN.value,
org_admin_of=(world.org_a_id, world.org_b_id),
)
team_id = await create_scratch_team(
prisma, scratch.tag("team"), organization_id=world.org_a_id
)
resp = await proxy_client.post(
"/team/update",
headers={"Authorization": f"Bearer {actor.cleartext}"},
json={"team_id": team_id, "organization_id": world.org_b_id},
)
assert resp.status_code == 200, resp.text
row = await prisma.db.litellm_teamtable.find_unique(where={"team_id": team_id})
assert row is not None
assert (
row.organization_id == world.org_b_id
), "dual-org admin relocation not applied"