mirror of
https://github.com/tiennm99/litellm.git
synced 2026-06-26 19:08:09 +00:00
9495f4e941
* auth_with_role_name add region_name arg for cross-account sts * update tests to include case with aws_region_name for _auth_with_aws_role * Only pass region_name to STS client when aws_region_name is set * Add optional aws_sts_endpoint to _auth_with_aws_role * Parametrize ambient-credentials test for no opts, region_name, and aws_sts_endpoint * consistently passing region and endpoint args into explicit credentials irsa * fix env var leakage * fix: bedrock openai-compatible imported-model should also have model arn encoded * feat: show proxy url in ModelHub (#21660) * fix(bedrock): correct modelInput format for Converse API batch models (#21656) * fix(proxy): add model_ids param to access group endpoints for precise deployment tagging (#21655) POST /access_group/new and PUT /access_group/{name}/update now accept an optional model_ids list that targets specific deployments by their unique model_id, instead of tagging every deployment that shares a model_name. When model_ids is provided it takes priority over model_names, giving API callers the same single-deployment precision that the UI already has via PATCH /model/{model_id}/update. Backward compatible: model_names continues to work as before. Closes #21544 * feat(proxy): add custom favicon support\n\nAdd ability to configure a custom favicon for the litellm proxy UI.\n\n- Add favicon_url field to UIThemeConfig model\n- Add LITELLM_FAVICON_URL env var support\n- Add /get_favicon endpoint to serve custom favicons\n- Update ThemeContext to dynamically set favicon\n- Add favicon URL input to UI theme settings page\n- Add comprehensive tests\n\nCloses #8323 (#21653) * fix(bedrock): prevent double UUID in create_file S3 key (#21650) In create_file for Bedrock, get_complete_file_url is called twice: once in the sync handler (generating UUID-1 for api_base) and once inside transform_create_file_request (generating UUID-2 for the actual S3 upload). The Bedrock provider correctly writes UUID-2 into litellm_params["upload_url"], but the sync handler unconditionally overwrites it with api_base (UUID-1). This causes the returned file_id to point to a non-existent S3 key. Fix: only set upload_url to api_base when transform_create_file_request has not already set it, preserving the Bedrock provider's value. Closes #21546 * feat(semantic-cache): support configurable vector dimensions for Qdrant (#21649) Add vector_size parameter to QdrantSemanticCache and expose it through the Cache facade as qdrant_semantic_cache_vector_size. This allows users to use embedding models with dimensions other than the default 1536, enabling cheaper/stronger models like Stella (1024d), bge-en-icl (4096d), voyage, cohere, etc. The parameter defaults to QDRANT_VECTOR_SIZE (env var or 1536) for backward compatibility. When creating new collections, the configured vector_size is used instead of the hardcoded constant. Closes #9377 * fix(utils): normalize camelCase thinking param keys to snake_case (#21762) Clients like OpenCode's @ai-sdk/openai-compatible send budgetTokens (camelCase) instead of budget_tokens in the thinking parameter, causing validation errors. Add early normalization in completion(). * feat: add optional digest mode for Slack alert types (#21683) Adds per-alert-type digest mode that aggregates duplicate alerts within a configurable time window and emits a single summary message with count, start/end timestamps. Configuration via general_settings.alert_type_config: alert_type_config: llm_requests_hanging: digest: true digest_interval: 86400 Digest key: (alert_type, request_model, api_base) Default interval: 24 hours Window type: fixed interval Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add blog_posts.json and local backup * feat: add GetBlogPosts utility with GitHub fetch and local fallback Adds GetBlogPosts class that fetches blog posts from GitHub with a 1-hour in-process TTL cache, validates the response, and falls back to the bundled blog_posts_backup.json on any network or validation failure. * test: add cache reset fixture and LITELLM_LOCAL_BLOG_POSTS test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add GET /public/litellm_blog_posts endpoint Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: log fallback warning in blog posts endpoint and tighten test * feat: add disable_show_blog to UISettings Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add useUISettings and useDisableShowBlog hooks * fix: rename useUISettings to useUISettingsFlags to avoid naming collision * fix: use existing useUISettings hook in useDisableShowBlog to avoid cache duplication Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add BlogDropdown component with react-query and error/retry state Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: enforce 5-post limit in BlogDropdown and add cap test * fix: add retry, stable post key, enabled guard in BlogDropdown Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add BlogDropdown to navbar after Docs link * feat: add network_mock transport for benchmarking proxy overhead without real API calls Intercepts at httpx transport layer so the full proxy path (auth, routing, OpenAI SDK, response transformation) is exercised with zero-latency responses. Activated via `litellm_settings: { network_mock: true }` in proxy config. * Litellm dev 02 19 2026 p2 (#21871) * feat(ui/): new guardrails monitor 'demo mock representation of what guardrails monitor looks like * fix: ui updates * style(ui/): fix styling * feat: enable running ai monitor on individual guardrails * feat: add backend logic for guardrail monitoring * fix(guardrails/usage_endpoints.py): fix usage dashboard * fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo (#21754) * fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo * fix(budget): update stale docstring on get_budget_reset_time * fix: add missing return type annotations to iterator protocol methods in streaming_handler (#21750) * fix: add return type annotations to iterator protocol methods in streaming_handler Add missing return type annotations to __iter__, __aiter__, __next__, and __anext__ methods in CustomStreamWrapper and related classes. - __iter__(self) -> Iterator["ModelResponseStream"] - __aiter__(self) -> AsyncIterator["ModelResponseStream"] - __next__(self) -> "ModelResponseStream" - __anext__(self) -> "ModelResponseStream" Also adds AsyncIterator and Iterator to typing imports. Fixes issue with PLR0915 noqa comments and ensures proper type checking support. Related to: BerriAI/litellm#8304 * fix: add ruff PLR0915 noqa for files with too many statements * Add gollem Go agent framework cookbook example (#21747) Show how to use gollem, a production Go agent framework, with LiteLLM proxy for multi-provider LLM access including tool use and streaming. * fix: avoid mutating caller-owned dicts in SpendUpdateQueue aggregation (#21742) * fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870) * server root path regression doc * fixing syntax * fix: replace Zapier webhook with Google Form for survey submission (#21621) * Replace Zapier webhook with Google Form for survey submission * Add back error logging for survey submission debugging --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> * Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth" This reverts commit0e1db3f7e4, reversing changes made to7e2d6f2355. * test_vertex_ai_gemini_2_5_pro_streaming * UI new build * fix rendering * ui new build * docs fix * docs fix * docs fix * docs fix * docs fix * docs fix * docs fix * docs fix * release note docs * docs * adding image * fix(vertex_ai): enable context-1m-2025-08-07 beta header The `context-1m-2025-08-07` Anthropic beta header was set to `null` for vertex_ai, causing it to be filtered out when users set `extra_headers: {anthropic-beta: context-1m-2025-08-07}`. This prevented using Claude's 1M context window feature via Vertex AI, resulting in `prompt is too long: 460500 tokens > 200000 maximum` errors. Fixes #21861 --------- Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: milan-berri <milan@berri.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> * Revert "fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)" (#21876) This reverts commitbce078a796. * docs(ui): add pre-PR checklist to UI contributing guide Add testing and build verification steps per maintainer feedback from @yjiang-litellm. Contributors should run their related tests per-file and ensure npm run build passes before opening PRs. * Fix entries with fast and us/ * Add tests for fast and us * Add support for Priority PayGo for vertex ai and gemini * Add model pricing * fix: ensure arrival_time is set before calculating queue time * Fix: Anthropic model wildcard access issue * Add incident report * Add ability to see which model cost map is getting used * Fix name of title * Readd tpm limit * State management fixes for CheckBatchCost * Fix PR review comments * State management fixes for CheckBatchCost - Address greptile comments * fix mypy issues: * Add Noma guardrails v2 based on custom guardrails (#21400) * Fix code qa issues * Fix mypy issues * Fix mypy issues * Fix test_aaamodel_prices_and_context_window_json_is_valid * fix: update calendly on repo * fix(tests): use counter-based mock for time.time in prisma self-heal test The test used a fixed side_effect list for time.time(), but the number of calls varies by Python version, causing StopIteration on 3.12 and AssertionError on 3.14. Replace with an infinite counter-based callable and assert the timestamp was updated rather than checking for an exact value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tests): use absolute path for model_prices JSON in validation test The test used a relative path 'litellm/model_prices_and_context_window.json' which only works when pytest runs from a specific working directory. Use os.path based on __file__ to resolve the path reliably. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update tests/test_litellm/test_utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(tests): use os.path instead of Path to avoid NameError Path is not imported at module level. Use os.path.join which is already available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * clean up mock transport: remove streaming, add defensive parsing * docs: add Google GenAI SDK tutorial (JS & Python) (#21885) * docs: add Google GenAI SDK tutorial for JS and Python Add tutorial for using Google's official GenAI SDK (@google/genai for JS, google-genai for Python) with LiteLLM proxy. Covers pass-through and native router endpoints, streaming, multi-turn chat, and multi-provider routing via model_group_alias. Also updates pass-through docs to use the new SDK replacing the deprecated @google/generative-ai. * fix(docs): correct Python SDK env var name in GenAI tutorial GOOGLE_GENAI_API_KEY does not exist in the google-genai SDK. The correct env var is GEMINI_API_KEY (or GOOGLE_API_KEY). Also note that the Python SDK has no base URL env var. * fix(docs): replace non-existent GOOGLE_GENAI_BASE_URL env var in interactions.md The Python google-genai SDK does not read GOOGLE_GENAI_BASE_URL. Use http_options={"base_url": "..."} in code instead. * docs: add network mock benchmarking section * docs: tweak benchmarks wording * fix: add auth headers and empty latencies guard to benchmark script * refactor: use method-level import for MockOpenAITransport * fix: guard print_aggregate against empty latencies * fix: add INCOMPLETE status to Interactions API enum and test Google added INCOMPLETE to the Interactions API OpenAPI spec status enum. Update both the Status3 enum in the SDK types and the test's expected values to match. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Guardrail Monitor - measure guardrail reliability in prod (#21944) * fix: fix log viewer for guardrail monitoring * feat(ui/): fix rendering logs per guardrail * fix: fix viewing logs on overview tab of guardrail * fix: log viewer * fix: fix naming to align with metric * docs: add performance & reliability section to v1.81.14 release notes * fix(tests): make RPM limit test sequential to avoid race condition Concurrent requests via run_in_executor + asyncio.gather caused a race condition where more requests slipped through the rate limiter than expected, leading to flaky test failures (e.g. 3 successes instead of 2 with rpm_limit=2). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Singapore guardrail policies (PDPA + MAS AI Risk Management) (#21948) * feat: Singapore PDPA PII protection guardrail policy template Add Singapore Personal Data Protection Act (PDPA) guardrail support: Regex patterns (patterns.json): - sg_nric: NRIC/FIN detection ([STFGM] + 7 digits + checksum letter) - sg_phone: Singapore phone numbers (+65/0065/65 prefix) - sg_postal_code: 6-digit postal codes (contextual) - passport_singapore: Passport numbers (E/K + 7 digits, contextual) - sg_uen: Unique Entity Numbers (3 formats) - sg_bank_account: Bank account numbers (dash format, contextual) YAML policy templates (5 sub-guardrails): - sg_pdpa_personal_identifiers: s.13 Consent - sg_pdpa_sensitive_data: Advisory Guidelines - sg_pdpa_do_not_call: Part IX DNC Registry - sg_pdpa_data_transfer: s.26 overseas transfers - sg_pdpa_profiling_automated_decisions: Model AI Governance Framework Policy template entry in policy_templates.json with 9 guardrail definitions (4 regex-based + 5 YAML conditional keyword matching). Tests: - test_sg_patterns.py: regex pattern unit tests - test_sg_pdpa_guardrails.py: conditional keyword matching tests (100+ cases) * feat: MAS AI Risk Management Guidelines guardrail policy template Add Monetary Authority of Singapore (MAS) AI Risk Management Guidelines guardrail support for financial institutions: YAML policy templates (5 sub-guardrails): - sg_mas_fairness_bias: Blocks discriminatory financial AI (credit/loans/insurance by protected attributes) - sg_mas_transparency_explainability: Blocks opaque/unexplainable AI for consequential financial decisions - sg_mas_human_oversight: Blocks fully automated financial decisions without human-in-the-loop - sg_mas_data_governance: Blocks unauthorized sharing/mishandling of financial customer data - sg_mas_model_security: Blocks adversarial attacks, model poisoning, inversion on financial AI Policy template entry in policy_templates.json with 5 guardrail definitions. Aligned with MAS FEAT Principles, Project MindForge, and NIST AI RMF. Tests: - test_sg_mas_ai_guardrails.py: conditional keyword matching tests (100+ cases) * fix: address SG pattern review feedback - Update NRIC lowercase test for IGNORECASE runtime behavior - Add keyword context guard to sg_uen pattern to reduce false positives * docs: clarify MAS AIRM timeline references - Explicitly mark MAS AIRM as Nov 2025 consultation draft - Add 2018 qualifier for FEAT principles in MAS policy descriptions - Update MAS guardrail wording to avoid release-year ambiguity * chore: commit resolved MAS policy conflicts * test: * chore: * Add OpenAI Agents SDK tutorial with LiteLLM Proxy to docs (#21221) * Add OpenAI Agents SDK tutorial to docs * Update OpenAI Agents SDK tutorial to use LiteLLM environment variables * Enhance OpenAI Agents SDK tutorial with built-in LiteLLM extension details and updated configuration steps. Adjust section headings for clarity and improve the flow of information regarding model setup and usage. * adjust blog posts to fetch from github first * feat(videos): add variant parameter to video content download (#21955) openai videos models support the features to download variants. See more details here: https://developers.openai.com/api/docs/guides/video-generation#use-image-references. Plumb variant (e.g. "thumbnail", "spritesheet") through the full video content download chain: avideo_content → video_content → video_content_handler → transform_video_content_request. OpenAI appends ?variant=<value> to the GET URL; other providers accept the parameter in their signature but ignore it. * fixing path * adjust blog post path * Revert duplicate issue checker to text-based matching, remove duplicate PR workflow Remove the Claude Code-powered duplicate PR detection workflow and revert the duplicate issue checker back to wow-actions/potential-duplicates with text similarity matching. * ui changes * adding tests * adjust default aggregation threshold * fix(videos): pass api_key from litellm_params to video remix handlers (#21965) video_remix_handler and async_video_remix_handler were not falling back to litellm_params.api_key when the api_key parameter was None, causing Authorization: Bearer None to be sent to the provider. This matches the pattern already used by async_video_generation_handler. * adding testing coverage + fixing flaky tests * fix(ollama): thread api_base through get_model_info and add graceful fallback When users pass api_base to litellm.completion() for Ollama, the model info fetch (context window, function_calling support) was ignoring the user's api_base and only reading OLLAMA_API_BASE env var or defaulting to localhost:11434. This caused confusing errors in logs when Ollama runs on a remote server. Thread api_base from litellm_params through the get_model_info call chain so OllamaConfig.get_model_info() uses the correct server. Also return safe defaults instead of raising when the server is unreachable. Fixes #21967 --------- Co-authored-by: An Tang <ta@stripe.com> Co-authored-by: janfrederickk <75388864+janfrederickk@users.noreply.github.com> Co-authored-by: Zhenting Huang <3061613175@qq.com> Co-authored-by: Darien Kindlund <darien@kindlund.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu> Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: LeeJuOh <56071126+LeeJuOh@users.noreply.github.com> Co-authored-by: Monesh Ram <31161039+WhoisMonesh@users.noreply.github.com> Co-authored-by: Trevor Prater <trevor.prater@gmail.com> Co-authored-by: The Mavik <179817126+themavik@users.noreply.github.com> Co-authored-by: Edwin Isac <33712823+edwiniac@users.noreply.github.com> Co-authored-by: milan-berri <milan@berri.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: Harshit Jain <harshitjain0562@gmail.com> Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com> Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com> Co-authored-by: TomAlon <tom@noma.security> Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: Ron Zhong <ron-zhong@hotmail.com> Co-authored-by: Arindam Majumder <109217591+Arindam200@users.noreply.github.com> Co-authored-by: Lei Nie <lenie@quora.com>
1073 lines
41 KiB
Plaintext
1073 lines
41 KiB
Plaintext
datasource client {
|
|
provider = "postgresql"
|
|
url = env("DATABASE_URL")
|
|
}
|
|
|
|
generator client {
|
|
provider = "prisma-client-py"
|
|
binaryTargets = ["native", "debian-openssl-1.1.x", "debian-openssl-3.0.x", "linux-musl", "linux-musl-openssl-3.0.x"]
|
|
}
|
|
|
|
// Budget / Rate Limits for an org
|
|
model LiteLLM_BudgetTable {
|
|
budget_id String @id @default(uuid())
|
|
max_budget Float?
|
|
soft_budget Float?
|
|
max_parallel_requests Int?
|
|
tpm_limit BigInt?
|
|
rpm_limit BigInt?
|
|
model_max_budget Json?
|
|
budget_duration String?
|
|
budget_reset_at DateTime?
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
organization LiteLLM_OrganizationTable[] // multiple orgs can have the same budget
|
|
projects LiteLLM_ProjectTable[] // multiple projects can have the same budget
|
|
keys LiteLLM_VerificationToken[] // multiple keys can have the same budget
|
|
end_users LiteLLM_EndUserTable[] // multiple end-users can have the same budget
|
|
tags LiteLLM_TagTable[] // multiple tags can have the same budget
|
|
team_membership LiteLLM_TeamMembership[] // budgets of Users within a Team
|
|
organization_membership LiteLLM_OrganizationMembership[] // budgets of Users within a Organization
|
|
}
|
|
|
|
// Models on proxy
|
|
model LiteLLM_CredentialsTable {
|
|
credential_id String @id @default(uuid())
|
|
credential_name String @unique
|
|
credential_values Json
|
|
credential_info Json?
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
}
|
|
|
|
// Models on proxy
|
|
model LiteLLM_ProxyModelTable {
|
|
model_id String @id @default(uuid())
|
|
model_name String
|
|
litellm_params Json
|
|
model_info Json?
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
}
|
|
|
|
|
|
// Agents on proxy
|
|
model LiteLLM_AgentsTable {
|
|
agent_id String @id @default(uuid())
|
|
agent_name String @unique
|
|
litellm_params Json?
|
|
agent_card_params Json
|
|
agent_access_groups String[] @default([])
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
}
|
|
|
|
model LiteLLM_OrganizationTable {
|
|
organization_id String @id @default(uuid())
|
|
organization_alias String
|
|
budget_id String
|
|
metadata Json @default("{}")
|
|
models String[]
|
|
spend Float @default(0.0)
|
|
model_spend Json @default("{}")
|
|
object_permission_id String?
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
teams LiteLLM_TeamTable[]
|
|
users LiteLLM_UserTable[]
|
|
keys LiteLLM_VerificationToken[]
|
|
members LiteLLM_OrganizationMembership[] @relation("OrganizationToMembership")
|
|
object_permission LiteLLM_ObjectPermissionTable? @relation(fields: [object_permission_id], references: [object_permission_id])
|
|
}
|
|
|
|
// Model info for teams, just has model aliases for now.
|
|
model LiteLLM_ModelTable {
|
|
id Int @id @default(autoincrement())
|
|
model_aliases Json? @map("aliases")
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
team LiteLLM_TeamTable?
|
|
}
|
|
|
|
|
|
// Assign prod keys to groups, not individuals
|
|
model LiteLLM_TeamTable {
|
|
team_id String @id @default(uuid())
|
|
team_alias String?
|
|
organization_id String?
|
|
object_permission_id String?
|
|
admins String[]
|
|
members String[]
|
|
members_with_roles Json @default("{}")
|
|
metadata Json @default("{}")
|
|
max_budget Float?
|
|
soft_budget Float?
|
|
spend Float @default(0.0)
|
|
models String[]
|
|
max_parallel_requests Int?
|
|
tpm_limit BigInt?
|
|
rpm_limit BigInt?
|
|
budget_duration String?
|
|
budget_reset_at DateTime?
|
|
blocked Boolean @default(false)
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
model_spend Json @default("{}")
|
|
model_max_budget Json @default("{}")
|
|
router_settings Json? @default("{}")
|
|
team_member_permissions String[] @default([])
|
|
access_group_ids String[] @default([])
|
|
policies String[] @default([])
|
|
model_id Int? @unique // id for LiteLLM_ModelTable -> stores team-level model aliases
|
|
allow_team_guardrail_config Boolean @default(false) // if true, team admin can configure guardrails for this team
|
|
litellm_organization_table LiteLLM_OrganizationTable? @relation(fields: [organization_id], references: [organization_id])
|
|
litellm_model_table LiteLLM_ModelTable? @relation(fields: [model_id], references: [id])
|
|
object_permission LiteLLM_ObjectPermissionTable? @relation(fields: [object_permission_id], references: [object_permission_id])
|
|
projects LiteLLM_ProjectTable[]
|
|
}
|
|
|
|
// Projects sit between teams and keys for use-case management
|
|
model LiteLLM_ProjectTable {
|
|
project_id String @id @default(uuid())
|
|
project_alias String?
|
|
description String?
|
|
team_id String?
|
|
budget_id String?
|
|
metadata Json @default("{}")
|
|
models String[]
|
|
spend Float @default(0.0)
|
|
model_spend Json @default("{}")
|
|
model_rpm_limit Json @default("{}")
|
|
model_tpm_limit Json @default("{}")
|
|
blocked Boolean @default(false)
|
|
object_permission_id String?
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String
|
|
|
|
// Relations
|
|
litellm_team_table LiteLLM_TeamTable? @relation(fields: [team_id], references: [team_id])
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
keys LiteLLM_VerificationToken[]
|
|
object_permission LiteLLM_ObjectPermissionTable? @relation(fields: [object_permission_id], references: [object_permission_id])
|
|
}
|
|
|
|
// Audit table for deleted teams - preserves spend and team information for historical tracking
|
|
model LiteLLM_DeletedTeamTable {
|
|
id String @id @default(uuid())
|
|
team_id String // Original team_id
|
|
team_alias String?
|
|
organization_id String?
|
|
object_permission_id String?
|
|
admins String[]
|
|
members String[]
|
|
members_with_roles Json @default("{}")
|
|
metadata Json @default("{}")
|
|
max_budget Float?
|
|
soft_budget Float?
|
|
spend Float @default(0.0)
|
|
models String[]
|
|
max_parallel_requests Int?
|
|
tpm_limit BigInt?
|
|
rpm_limit BigInt?
|
|
budget_duration String?
|
|
budget_reset_at DateTime?
|
|
blocked Boolean @default(false)
|
|
model_spend Json @default("{}")
|
|
model_max_budget Json @default("{}")
|
|
router_settings Json? @default("{}")
|
|
team_member_permissions String[] @default([])
|
|
access_group_ids String[] @default([])
|
|
policies String[] @default([])
|
|
model_id Int? // id for LiteLLM_ModelTable -> stores team-level model aliases
|
|
allow_team_guardrail_config Boolean @default(false)
|
|
|
|
// Original timestamps from team creation/updates
|
|
created_at DateTime? @map("created_at")
|
|
updated_at DateTime? @map("updated_at")
|
|
|
|
// Deletion metadata
|
|
deleted_at DateTime @default(now()) @map("deleted_at")
|
|
deleted_by String? @map("deleted_by") // User who deleted the team
|
|
deleted_by_api_key String? @map("deleted_by_api_key") // API key hash that performed the deletion
|
|
litellm_changed_by String? @map("litellm_changed_by") // From litellm-changed-by header if provided
|
|
|
|
@@index([team_id])
|
|
@@index([deleted_at])
|
|
@@index([organization_id])
|
|
@@index([team_alias])
|
|
@@index([created_at])
|
|
}
|
|
|
|
// Track spend, rate limit, budget Users
|
|
model LiteLLM_UserTable {
|
|
user_id String @id
|
|
user_alias String?
|
|
team_id String?
|
|
sso_user_id String? @unique
|
|
organization_id String?
|
|
object_permission_id String?
|
|
password String?
|
|
teams String[] @default([])
|
|
user_role String?
|
|
max_budget Float?
|
|
spend Float @default(0.0)
|
|
user_email String?
|
|
models String[]
|
|
metadata Json @default("{}")
|
|
max_parallel_requests Int?
|
|
tpm_limit BigInt?
|
|
rpm_limit BigInt?
|
|
budget_duration String?
|
|
budget_reset_at DateTime?
|
|
allowed_cache_controls String[] @default([])
|
|
policies String[] @default([])
|
|
model_spend Json @default("{}")
|
|
model_max_budget Json @default("{}")
|
|
created_at DateTime? @default(now()) @map("created_at")
|
|
updated_at DateTime? @default(now()) @updatedAt @map("updated_at")
|
|
|
|
// relations
|
|
litellm_organization_table LiteLLM_OrganizationTable? @relation(fields: [organization_id], references: [organization_id])
|
|
organization_memberships LiteLLM_OrganizationMembership[]
|
|
invitations_created LiteLLM_InvitationLink[] @relation("CreatedBy")
|
|
invitations_updated LiteLLM_InvitationLink[] @relation("UpdatedBy")
|
|
invitations_user LiteLLM_InvitationLink[] @relation("UserId")
|
|
object_permission LiteLLM_ObjectPermissionTable? @relation(fields: [object_permission_id], references: [object_permission_id])
|
|
}
|
|
|
|
model LiteLLM_ObjectPermissionTable {
|
|
object_permission_id String @id @default(uuid())
|
|
mcp_servers String[] @default([])
|
|
mcp_access_groups String[] @default([])
|
|
mcp_tool_permissions Json? // Tool-level permissions for MCP servers. Format: {"server_id": ["tool_name_1", "tool_name_2"]}
|
|
vector_stores String[] @default([])
|
|
agents String[] @default([])
|
|
agent_access_groups String[] @default([])
|
|
teams LiteLLM_TeamTable[]
|
|
projects LiteLLM_ProjectTable[]
|
|
verification_tokens LiteLLM_VerificationToken[]
|
|
organizations LiteLLM_OrganizationTable[]
|
|
users LiteLLM_UserTable[]
|
|
end_users LiteLLM_EndUserTable[]
|
|
}
|
|
|
|
// Holds the MCP server configuration
|
|
model LiteLLM_MCPServerTable {
|
|
server_id String @id @default(uuid())
|
|
server_name String?
|
|
alias String?
|
|
description String?
|
|
url String?
|
|
spec_path String?
|
|
transport String @default("sse")
|
|
auth_type String?
|
|
credentials Json? @default("{}")
|
|
created_at DateTime? @default(now()) @map("created_at")
|
|
created_by String?
|
|
updated_at DateTime? @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String?
|
|
mcp_info Json? @default("{}")
|
|
mcp_access_groups String[]
|
|
allowed_tools String[] @default([])
|
|
extra_headers String[] @default([])
|
|
static_headers Json? @default("{}")
|
|
// Health check status
|
|
status String? @default("unknown")
|
|
last_health_check DateTime?
|
|
health_check_error String?
|
|
// Stdio-specific fields
|
|
command String?
|
|
args String[] @default([])
|
|
env Json? @default("{}")
|
|
authorization_url String?
|
|
token_url String?
|
|
registration_url String?
|
|
allow_all_keys Boolean @default(false)
|
|
available_on_public_internet Boolean @default(false)
|
|
}
|
|
|
|
// Generate Tokens for Proxy
|
|
model LiteLLM_VerificationToken {
|
|
token String @id
|
|
key_name String?
|
|
key_alias String?
|
|
soft_budget_cooldown Boolean @default(false) // key-level state on if budget alerts need to be cooled down
|
|
spend Float @default(0.0)
|
|
expires DateTime?
|
|
models String[]
|
|
aliases Json @default("{}")
|
|
config Json @default("{}")
|
|
router_settings Json? @default("{}")
|
|
user_id String?
|
|
team_id String?
|
|
project_id String?
|
|
permissions Json @default("{}")
|
|
max_parallel_requests Int?
|
|
metadata Json @default("{}")
|
|
blocked Boolean?
|
|
tpm_limit BigInt?
|
|
rpm_limit BigInt?
|
|
max_budget Float?
|
|
budget_duration String?
|
|
budget_reset_at DateTime?
|
|
allowed_cache_controls String[] @default([])
|
|
allowed_routes String[] @default([])
|
|
policies String[] @default([])
|
|
access_group_ids String[] @default([])
|
|
model_spend Json @default("{}")
|
|
model_max_budget Json @default("{}")
|
|
budget_id String?
|
|
organization_id String?
|
|
object_permission_id String?
|
|
created_at DateTime? @default(now()) @map("created_at")
|
|
created_by String?
|
|
updated_at DateTime? @default(now()) @updatedAt @map("updated_at")
|
|
updated_by String?
|
|
last_active DateTime? // When this key was last used
|
|
rotation_count Int? @default(0) // Number of times key has been rotated
|
|
auto_rotate Boolean? @default(false) // Whether this key should be auto-rotated
|
|
rotation_interval String? // How often to rotate (e.g., "30d", "90d")
|
|
last_rotation_at DateTime? // When this key was last rotated
|
|
key_rotation_at DateTime? // When this key should next be rotated
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
litellm_organization_table LiteLLM_OrganizationTable? @relation(fields: [organization_id], references: [organization_id])
|
|
litellm_project_table LiteLLM_ProjectTable? @relation(fields: [project_id], references: [project_id])
|
|
object_permission LiteLLM_ObjectPermissionTable? @relation(fields: [object_permission_id], references: [object_permission_id])
|
|
|
|
// SELECT COUNT(*) FROM (SELECT "public"."LiteLLM_VerificationToken"."token" FROM "public"."LiteLLM_VerificationToken" WHERE ("public"."LiteLLM_VerificationToken"."user_id" = $1 AND ("public"."LiteLLM_VerificationToken"."team_id" IS NULL OR "public"."LiteLLM_VerificationToken"."team_id" <> $2)) OFFSET $3 ) AS "sub"
|
|
// SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE "public"."LiteLLM_VerificationToken"."user_id" = $1 OFFSET $2
|
|
@@index([user_id, team_id])
|
|
|
|
// SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE "public"."LiteLLM_VerificationToken"."team_id" = $1 OFFSET $2
|
|
@@index([team_id])
|
|
|
|
// SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (("public"."LiteLLM_VerificationToken"."expires" IS NULL OR "public"."LiteLLM_VerificationToken"."expires" > $1) AND "public"."LiteLLM_VerificationToken"."budget_reset_at" < $2) OFFSET $3
|
|
@@index([budget_reset_at, expires])
|
|
}
|
|
|
|
// Deprecated keys during grace period - allows old key to work until revoke_at
|
|
model LiteLLM_DeprecatedVerificationToken {
|
|
id String @id @default(uuid())
|
|
token String // Hashed old key
|
|
active_token_id String // Current token hash in LiteLLM_VerificationToken
|
|
revoke_at DateTime // When the old key stops working
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
|
|
@@unique([token])
|
|
@@index([token, revoke_at])
|
|
@@index([revoke_at])
|
|
}
|
|
|
|
// Audit table for deleted keys - preserves spend and key information for historical tracking
|
|
model LiteLLM_DeletedVerificationToken {
|
|
id String @id @default(uuid())
|
|
token String // Original token (hashed)
|
|
key_name String?
|
|
key_alias String?
|
|
soft_budget_cooldown Boolean @default(false)
|
|
spend Float @default(0.0)
|
|
expires DateTime?
|
|
models String[]
|
|
aliases Json @default("{}")
|
|
config Json @default("{}")
|
|
user_id String?
|
|
team_id String?
|
|
project_id String?
|
|
permissions Json @default("{}")
|
|
max_parallel_requests Int?
|
|
metadata Json @default("{}")
|
|
blocked Boolean?
|
|
tpm_limit BigInt?
|
|
rpm_limit BigInt?
|
|
max_budget Float?
|
|
budget_duration String?
|
|
budget_reset_at DateTime?
|
|
allowed_cache_controls String[] @default([])
|
|
allowed_routes String[] @default([])
|
|
policies String[] @default([])
|
|
access_group_ids String[] @default([])
|
|
model_spend Json @default("{}")
|
|
model_max_budget Json @default("{}")
|
|
router_settings Json? @default("{}")
|
|
budget_id String?
|
|
organization_id String?
|
|
object_permission_id String?
|
|
created_at DateTime? // Original creation timestamp
|
|
created_by String? // Original creator
|
|
updated_at DateTime? // Last update timestamp before deletion
|
|
updated_by String? // Last user who updated before deletion
|
|
last_active DateTime? // When this key was last used before deletion
|
|
rotation_count Int? @default(0)
|
|
auto_rotate Boolean? @default(false)
|
|
rotation_interval String?
|
|
last_rotation_at DateTime?
|
|
key_rotation_at DateTime?
|
|
|
|
// Deletion metadata
|
|
deleted_at DateTime @default(now()) @map("deleted_at")
|
|
deleted_by String? @map("deleted_by") // User who deleted the key
|
|
deleted_by_api_key String? @map("deleted_by_api_key") // API key hash that performed the deletion
|
|
litellm_changed_by String? @map("litellm_changed_by") // From litellm-changed-by header if provided
|
|
|
|
@@index([token])
|
|
@@index([deleted_at])
|
|
@@index([user_id])
|
|
@@index([team_id])
|
|
@@index([organization_id])
|
|
@@index([key_alias])
|
|
@@index([created_at])
|
|
}
|
|
|
|
model LiteLLM_EndUserTable {
|
|
user_id String @id
|
|
alias String? // admin-facing alias
|
|
spend Float @default(0.0)
|
|
allowed_model_region String? // require all user requests to use models in this specific region
|
|
default_model String? // use along with 'allowed_model_region'. if no available model in region, default to this model.
|
|
budget_id String?
|
|
object_permission_id String?
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
object_permission LiteLLM_ObjectPermissionTable? @relation(fields: [object_permission_id], references: [object_permission_id])
|
|
blocked Boolean @default(false)
|
|
}
|
|
|
|
// Track tags with budgets and spend
|
|
model LiteLLM_TagTable {
|
|
tag_name String @id
|
|
description String?
|
|
models String[]
|
|
model_info Json? // maps model_id to model_name
|
|
spend Float @default(0.0)
|
|
budget_id String?
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
created_at DateTime @default(now()) @map("created_at")
|
|
created_by String?
|
|
updated_at DateTime @default(now()) @updatedAt @map("updated_at")
|
|
}
|
|
|
|
// store proxy config.yaml
|
|
model LiteLLM_Config {
|
|
param_name String @id
|
|
param_value Json?
|
|
}
|
|
|
|
// View spend, model, api_key per request
|
|
model LiteLLM_SpendLogs {
|
|
request_id String @id
|
|
call_type String
|
|
api_key String @default ("") // Hashed API Token. Not the actual Virtual Key. Equivalent to 'token' column in LiteLLM_VerificationToken
|
|
spend Float @default(0.0)
|
|
total_tokens Int @default(0)
|
|
prompt_tokens Int @default(0)
|
|
completion_tokens Int @default(0)
|
|
startTime DateTime // Assuming start_time is a DateTime field
|
|
endTime DateTime // Assuming end_time is a DateTime field
|
|
completionStartTime DateTime? // Assuming completionStartTime is a DateTime field
|
|
model String @default("")
|
|
model_id String? @default("") // the model id stored in proxy model db
|
|
model_group String? @default("") // public model_name / model_group
|
|
custom_llm_provider String? @default("") // litellm used custom_llm_provider
|
|
api_base String? @default("")
|
|
user String? @default("")
|
|
metadata Json? @default("{}") // project_id stored here
|
|
cache_hit String? @default("")
|
|
cache_key String? @default("")
|
|
request_tags Json? @default("[]")
|
|
team_id String?
|
|
organization_id String?
|
|
end_user String?
|
|
requester_ip_address String?
|
|
messages Json? @default("{}")
|
|
response Json? @default("{}")
|
|
session_id String?
|
|
status String?
|
|
mcp_namespaced_tool_name String?
|
|
agent_id String?
|
|
proxy_server_request Json? @default("{}")
|
|
@@index([startTime])
|
|
@@index([end_user])
|
|
@@index([session_id])
|
|
}
|
|
|
|
// View spend, model, api_key per request
|
|
model LiteLLM_ErrorLogs {
|
|
request_id String @id @default(uuid())
|
|
startTime DateTime // Assuming start_time is a DateTime field
|
|
endTime DateTime // Assuming end_time is a DateTime field
|
|
api_base String @default("")
|
|
model_group String @default("") // public model_name / model_group
|
|
litellm_model_name String @default("") // model passed to litellm
|
|
model_id String @default("") // ID of model in ProxyModelTable
|
|
request_kwargs Json @default("{}")
|
|
exception_type String @default("")
|
|
exception_string String @default("")
|
|
status_code String @default("")
|
|
}
|
|
|
|
// Beta - allow team members to request access to a model
|
|
model LiteLLM_UserNotifications {
|
|
request_id String @id
|
|
user_id String
|
|
models String[]
|
|
justification String
|
|
status String // approved, disapproved, pending
|
|
}
|
|
|
|
model LiteLLM_TeamMembership {
|
|
// Use this table to track the Internal User's Spend within a Team + Set Budgets, rpm limits for the user within the team
|
|
user_id String
|
|
team_id String
|
|
spend Float @default(0.0)
|
|
budget_id String?
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
@@id([user_id, team_id])
|
|
}
|
|
|
|
model LiteLLM_OrganizationMembership {
|
|
// Use this table to track Internal User and Organization membership. Helps tracking a users role within an Organization
|
|
user_id String
|
|
organization_id String
|
|
user_role String?
|
|
spend Float? @default(0.0)
|
|
budget_id String?
|
|
created_at DateTime? @default(now()) @map("created_at")
|
|
updated_at DateTime? @default(now()) @updatedAt @map("updated_at")
|
|
|
|
// relations
|
|
user LiteLLM_UserTable @relation(fields: [user_id], references: [user_id])
|
|
organization LiteLLM_OrganizationTable @relation("OrganizationToMembership", fields: [organization_id], references: [organization_id])
|
|
litellm_budget_table LiteLLM_BudgetTable? @relation(fields: [budget_id], references: [budget_id])
|
|
|
|
|
|
|
|
@@id([user_id, organization_id])
|
|
@@unique([user_id, organization_id])
|
|
}
|
|
|
|
model LiteLLM_InvitationLink {
|
|
// use this table to track invite links sent by admin for people to join the proxy
|
|
id String @id @default(uuid())
|
|
user_id String
|
|
is_accepted Boolean @default(false)
|
|
accepted_at DateTime? // when link is claimed (user successfully onboards via link)
|
|
expires_at DateTime // till when is link valid
|
|
created_at DateTime // when did admin create the link
|
|
created_by String // who created the link
|
|
updated_at DateTime // when was invite status updated
|
|
updated_by String // who updated the status (admin/user who accepted invite)
|
|
|
|
// Relations
|
|
liteLLM_user_table_user LiteLLM_UserTable @relation("UserId", fields: [user_id], references: [user_id])
|
|
liteLLM_user_table_created LiteLLM_UserTable @relation("CreatedBy", fields: [created_by], references: [user_id])
|
|
liteLLM_user_table_updated LiteLLM_UserTable @relation("UpdatedBy", fields: [updated_by], references: [user_id])
|
|
}
|
|
|
|
|
|
model LiteLLM_AuditLog {
|
|
id String @id @default(uuid())
|
|
updated_at DateTime @default(now())
|
|
changed_by String @default("") // user or system that performed the action
|
|
changed_by_api_key String @default("") // api key hash that performed the action
|
|
action String // create, update, delete
|
|
table_name String // on of LitellmTableNames.TEAM_TABLE_NAME, LitellmTableNames.USER_TABLE_NAME, LitellmTableNames.PROXY_MODEL_TABLE_NAME,
|
|
object_id String // id of the object being audited. This can be the key id, team id, user id, model id
|
|
before_value Json? // value of the row
|
|
updated_values Json? // value of the row after change
|
|
}
|
|
|
|
// Track daily user spend metrics per model and key
|
|
model LiteLLM_DailyUserSpend {
|
|
id String @id @default(uuid())
|
|
user_id String?
|
|
date String
|
|
api_key String
|
|
model String?
|
|
model_group String?
|
|
custom_llm_provider String?
|
|
mcp_namespaced_tool_name String?
|
|
endpoint String?
|
|
prompt_tokens BigInt @default(0)
|
|
completion_tokens BigInt @default(0)
|
|
cache_read_input_tokens BigInt @default(0)
|
|
cache_creation_input_tokens BigInt @default(0)
|
|
spend Float @default(0.0)
|
|
api_requests BigInt @default(0)
|
|
successful_requests BigInt @default(0)
|
|
failed_requests BigInt @default(0)
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@unique([user_id, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name, endpoint])
|
|
@@index([date])
|
|
@@index([user_id, date])
|
|
@@index([api_key])
|
|
@@index([model])
|
|
@@index([mcp_namespaced_tool_name])
|
|
@@index([endpoint])
|
|
}
|
|
|
|
// Track daily organization spend metrics per model and key
|
|
model LiteLLM_DailyOrganizationSpend {
|
|
id String @id @default(uuid())
|
|
organization_id String?
|
|
date String
|
|
api_key String
|
|
model String?
|
|
model_group String?
|
|
custom_llm_provider String?
|
|
mcp_namespaced_tool_name String?
|
|
endpoint String?
|
|
prompt_tokens BigInt @default(0)
|
|
completion_tokens BigInt @default(0)
|
|
cache_read_input_tokens BigInt @default(0)
|
|
cache_creation_input_tokens BigInt @default(0)
|
|
spend Float @default(0.0)
|
|
api_requests BigInt @default(0)
|
|
successful_requests BigInt @default(0)
|
|
failed_requests BigInt @default(0)
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@unique([organization_id, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name, endpoint])
|
|
@@index([date])
|
|
@@index([organization_id, date])
|
|
@@index([api_key])
|
|
@@index([model])
|
|
@@index([mcp_namespaced_tool_name])
|
|
@@index([endpoint])
|
|
}
|
|
|
|
// Track daily end user (customer) spend metrics per model and key
|
|
model LiteLLM_DailyEndUserSpend {
|
|
id String @id @default(uuid())
|
|
end_user_id String?
|
|
date String
|
|
api_key String
|
|
model String?
|
|
model_group String?
|
|
custom_llm_provider String?
|
|
mcp_namespaced_tool_name String?
|
|
endpoint String?
|
|
prompt_tokens BigInt @default(0)
|
|
completion_tokens BigInt @default(0)
|
|
cache_read_input_tokens BigInt @default(0)
|
|
cache_creation_input_tokens BigInt @default(0)
|
|
spend Float @default(0.0)
|
|
api_requests BigInt @default(0)
|
|
successful_requests BigInt @default(0)
|
|
failed_requests BigInt @default(0)
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
@@unique([end_user_id, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name, endpoint])
|
|
@@index([date])
|
|
@@index([end_user_id, date])
|
|
@@index([api_key])
|
|
@@index([model])
|
|
@@index([mcp_namespaced_tool_name])
|
|
@@index([endpoint])
|
|
}
|
|
|
|
// Track daily agent spend metrics per model and key
|
|
model LiteLLM_DailyAgentSpend {
|
|
id String @id @default(uuid())
|
|
agent_id String?
|
|
date String
|
|
api_key String
|
|
model String?
|
|
model_group String?
|
|
custom_llm_provider String?
|
|
mcp_namespaced_tool_name String?
|
|
endpoint String?
|
|
prompt_tokens BigInt @default(0)
|
|
completion_tokens BigInt @default(0)
|
|
cache_read_input_tokens BigInt @default(0)
|
|
cache_creation_input_tokens BigInt @default(0)
|
|
spend Float @default(0.0)
|
|
api_requests BigInt @default(0)
|
|
successful_requests BigInt @default(0)
|
|
failed_requests BigInt @default(0)
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
@@unique([agent_id, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name, endpoint])
|
|
@@index([date])
|
|
@@index([agent_id, date])
|
|
@@index([api_key])
|
|
@@index([model])
|
|
@@index([mcp_namespaced_tool_name])
|
|
@@index([endpoint])
|
|
}
|
|
|
|
// Track daily team spend metrics per model and key
|
|
model LiteLLM_DailyTeamSpend {
|
|
id String @id @default(uuid())
|
|
team_id String?
|
|
date String
|
|
api_key String
|
|
model String?
|
|
model_group String?
|
|
custom_llm_provider String?
|
|
mcp_namespaced_tool_name String?
|
|
endpoint String?
|
|
prompt_tokens BigInt @default(0)
|
|
completion_tokens BigInt @default(0)
|
|
cache_read_input_tokens BigInt @default(0)
|
|
cache_creation_input_tokens BigInt @default(0)
|
|
spend Float @default(0.0)
|
|
api_requests BigInt @default(0)
|
|
successful_requests BigInt @default(0)
|
|
failed_requests BigInt @default(0)
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@unique([team_id, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name, endpoint])
|
|
@@index([date])
|
|
@@index([team_id, date])
|
|
@@index([api_key])
|
|
@@index([model])
|
|
@@index([mcp_namespaced_tool_name])
|
|
@@index([endpoint])
|
|
}
|
|
|
|
// Track daily team spend metrics per model and key
|
|
model LiteLLM_DailyTagSpend {
|
|
id String @id @default(uuid())
|
|
request_id String?
|
|
tag String?
|
|
date String
|
|
api_key String
|
|
model String?
|
|
model_group String?
|
|
custom_llm_provider String?
|
|
mcp_namespaced_tool_name String?
|
|
endpoint String?
|
|
prompt_tokens BigInt @default(0)
|
|
completion_tokens BigInt @default(0)
|
|
cache_read_input_tokens BigInt @default(0)
|
|
cache_creation_input_tokens BigInt @default(0)
|
|
spend Float @default(0.0)
|
|
api_requests BigInt @default(0)
|
|
successful_requests BigInt @default(0)
|
|
failed_requests BigInt @default(0)
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@unique([tag, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name, endpoint])
|
|
@@index([date])
|
|
@@index([tag, date])
|
|
@@index([api_key])
|
|
@@index([model])
|
|
@@index([mcp_namespaced_tool_name])
|
|
@@index([endpoint])
|
|
}
|
|
|
|
|
|
// Track the status of cron jobs running. Only allow one pod to run the job at a time
|
|
model LiteLLM_CronJob {
|
|
cronjob_id String @id @default(cuid()) // Unique ID for the record
|
|
pod_id String // Unique identifier for the pod acting as the leader
|
|
status JobStatus @default(INACTIVE) // Status of the cron job (active or inactive)
|
|
last_updated DateTime @default(now()) // Timestamp for the last update of the cron job record
|
|
ttl DateTime // Time when the leader's lease expires
|
|
}
|
|
|
|
enum JobStatus {
|
|
ACTIVE
|
|
INACTIVE
|
|
}
|
|
|
|
model LiteLLM_ManagedFileTable {
|
|
id String @id @default(uuid())
|
|
unified_file_id String @unique // The base64 encoded unified file ID
|
|
file_object Json? // Stores the OpenAIFileObject
|
|
model_mappings Json
|
|
flat_model_file_ids String[] @default([]) // Flat list of model file id's - for faster querying of model id -> unified file id
|
|
storage_backend String? // Storage backend name (e.g., "azure_storage", "gcs", "default")
|
|
storage_url String? // The actual storage URL where the file is stored
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @updatedAt
|
|
updated_by String?
|
|
|
|
@@index([unified_file_id])
|
|
}
|
|
|
|
model LiteLLM_ManagedObjectTable { // for batches or finetuning jobs which use the
|
|
id String @id @default(uuid())
|
|
unified_object_id String @unique // The base64 encoded unified file ID
|
|
model_object_id String @unique // the id returned by the backend API provider
|
|
file_object Json // Stores the OpenAIFileObject
|
|
file_purpose String // either 'batch' or 'fine-tune'
|
|
status String? // check if batch cost has been tracked
|
|
batch_processed Boolean @default(false) // set to true by CheckBatchCost after cost is computed
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @updatedAt
|
|
updated_by String?
|
|
|
|
@@index([unified_object_id])
|
|
@@index([model_object_id])
|
|
}
|
|
|
|
model LiteLLM_ManagedVectorStoreTable {
|
|
id String @id @default(uuid())
|
|
unified_resource_id String @unique // The base64 encoded unified vector store ID
|
|
resource_object Json? // Stores the VectorStoreCreateResponse
|
|
model_mappings Json // Maps model_id -> provider_vector_store_id
|
|
flat_model_resource_ids String[] @default([]) // Flat list of provider vector store IDs for faster querying
|
|
storage_backend String? // Storage backend name (if applicable)
|
|
storage_url String? // Storage URL (if applicable)
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @updatedAt
|
|
updated_by String?
|
|
|
|
@@index([unified_resource_id])
|
|
}
|
|
|
|
model LiteLLM_ManagedVectorStoresTable {
|
|
vector_store_id String @id
|
|
custom_llm_provider String
|
|
vector_store_name String?
|
|
vector_store_description String?
|
|
vector_store_metadata Json?
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
litellm_credential_name String?
|
|
litellm_params Json?
|
|
team_id String?
|
|
user_id String?
|
|
|
|
@@index([team_id])
|
|
@@index([user_id])
|
|
}
|
|
|
|
// Guardrails table for storing guardrail configurations
|
|
model LiteLLM_GuardrailsTable {
|
|
guardrail_id String @id @default(uuid())
|
|
guardrail_name String @unique
|
|
litellm_params Json
|
|
guardrail_info Json?
|
|
team_id String?
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
}
|
|
|
|
// Daily guardrail metrics for usage dashboard (one row per guardrail per day)
|
|
model LiteLLM_DailyGuardrailMetrics {
|
|
guardrail_id String // logical id; may not FK if guardrail from config
|
|
date String // YYYY-MM-DD
|
|
requests_evaluated BigInt @default(0)
|
|
passed_count BigInt @default(0)
|
|
blocked_count BigInt @default(0)
|
|
flagged_count BigInt @default(0)
|
|
avg_score Float?
|
|
avg_latency_ms Float?
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@id([guardrail_id, date])
|
|
@@index([date])
|
|
@@index([guardrail_id])
|
|
}
|
|
|
|
// Daily policy metrics for usage dashboard (one row per policy per day)
|
|
model LiteLLM_DailyPolicyMetrics {
|
|
policy_id String
|
|
date String // YYYY-MM-DD
|
|
requests_evaluated BigInt @default(0)
|
|
passed_count BigInt @default(0)
|
|
blocked_count BigInt @default(0)
|
|
flagged_count BigInt @default(0)
|
|
avg_score Float?
|
|
avg_latency_ms Float?
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@id([policy_id, date])
|
|
@@index([date])
|
|
@@index([policy_id])
|
|
}
|
|
|
|
// Index for fast "last N logs for guardrail/policy" from SpendLogs
|
|
model LiteLLM_SpendLogGuardrailIndex {
|
|
request_id String
|
|
guardrail_id String
|
|
policy_id String? // set when run as part of a policy pipeline
|
|
start_time DateTime
|
|
|
|
@@id([request_id, guardrail_id])
|
|
@@index([guardrail_id, start_time])
|
|
@@index([policy_id, start_time])
|
|
}
|
|
|
|
// Prompt table for storing prompt configurations
|
|
model LiteLLM_PromptTable {
|
|
id String @id @default(uuid())
|
|
prompt_id String
|
|
version Int @default(1)
|
|
litellm_params Json
|
|
prompt_info Json?
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@unique([prompt_id, version])
|
|
@@index([prompt_id])
|
|
}
|
|
|
|
model LiteLLM_HealthCheckTable {
|
|
health_check_id String @id @default(uuid())
|
|
model_name String
|
|
model_id String?
|
|
status String
|
|
healthy_count Int @default(0)
|
|
unhealthy_count Int @default(0)
|
|
error_message String?
|
|
response_time_ms Float?
|
|
details Json?
|
|
checked_by String?
|
|
checked_at DateTime @default(now())
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
|
|
@@index([model_name])
|
|
@@index([checked_at])
|
|
@@index([status])
|
|
}
|
|
|
|
// Search Tools table for storing search tool configurations
|
|
model LiteLLM_SearchToolsTable {
|
|
search_tool_id String @id @default(uuid())
|
|
search_tool_name String @unique
|
|
litellm_params Json
|
|
search_tool_info Json?
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
}
|
|
|
|
// SSO configuration table
|
|
model LiteLLM_SSOConfig {
|
|
id String @id @default("sso_config")
|
|
sso_settings Json
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
}
|
|
|
|
model LiteLLM_ManagedVectorStoreIndexTable {
|
|
id String @id @default(uuid())
|
|
index_name String @unique
|
|
litellm_params Json
|
|
index_info Json?
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @updatedAt
|
|
updated_by String?
|
|
}
|
|
|
|
// Cache configuration table
|
|
model LiteLLM_CacheConfig {
|
|
id String @id @default("cache_config")
|
|
cache_settings Json
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
}
|
|
|
|
// UI Settings configuration table
|
|
model LiteLLM_UISettings {
|
|
id String @id @default("ui_settings")
|
|
ui_settings Json
|
|
created_at DateTime @default(now())
|
|
updated_at DateTime @updatedAt
|
|
}
|
|
|
|
// Skills table for storing LiteLLM-managed skills
|
|
model LiteLLM_SkillsTable {
|
|
skill_id String @id @default(uuid())
|
|
display_title String?
|
|
description String?
|
|
instructions String? // The skill instructions/prompt (from SKILL.md)
|
|
source String @default("custom") // "custom" or "anthropic"
|
|
latest_version String?
|
|
file_content Bytes? // Binary content of the skill files (zip)
|
|
file_name String? // Original filename
|
|
file_type String? // MIME type (e.g., "application/zip")
|
|
metadata Json? @default("{}")
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @default(now()) @updatedAt
|
|
updated_by String?
|
|
}
|
|
|
|
// Policy table for storing guardrail policies (versioned)
|
|
model LiteLLM_PolicyTable {
|
|
policy_id String @id @default(uuid())
|
|
policy_name String // No longer @unique; use @@unique([policy_name, version_number])
|
|
version_number Int @default(1)
|
|
version_status String @default("production") // "draft" | "published" | "production"
|
|
parent_version_id String?
|
|
is_latest Boolean @default(true)
|
|
published_at DateTime?
|
|
production_at DateTime?
|
|
inherit String? // Name of parent policy to inherit from
|
|
description String?
|
|
guardrails_add String[] @default([])
|
|
guardrails_remove String[] @default([])
|
|
condition Json? @default("{}") // Policy conditions (e.g., model matching)
|
|
pipeline Json? // Optional guardrail pipeline (mode + steps[])
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @default(now()) @updatedAt
|
|
updated_by String?
|
|
|
|
@@unique([policy_name, version_number])
|
|
@@index([policy_name, version_status])
|
|
}
|
|
|
|
// Policy attachment table for defining where policies apply
|
|
model LiteLLM_PolicyAttachmentTable {
|
|
attachment_id String @id @default(uuid())
|
|
policy_name String // Name of the policy to attach
|
|
scope String? // Use '*' for global scope
|
|
teams String[] @default([]) // Team aliases or patterns
|
|
keys String[] @default([]) // Key aliases or patterns
|
|
models String[] @default([]) // Model names or patterns
|
|
tags String[] @default([]) // Tag patterns (e.g., ["healthcare", "prod-*"])
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @default(now()) @updatedAt
|
|
updated_by String?
|
|
}
|
|
|
|
//Unified Access Groups table for storing unified access groups
|
|
model LiteLLM_AccessGroupTable {
|
|
access_group_id String @id @default(uuid())
|
|
access_group_name String @unique
|
|
description String?
|
|
|
|
// Resource memberships - explicit arrays per type
|
|
access_model_names String[] @default([])
|
|
access_mcp_server_ids String[] @default([])
|
|
access_agent_ids String[] @default([])
|
|
|
|
assigned_team_ids String[] @default([])
|
|
assigned_key_ids String[] @default([])
|
|
|
|
created_at DateTime @default(now())
|
|
created_by String?
|
|
updated_at DateTime @default(now()) @updatedAt
|
|
updated_by String?
|
|
} |