diff --git a/docs/my-website/docs/secret.md b/docs/my-website/docs/secret.md index 6e6e2ad506..21eb639581 100644 --- a/docs/my-website/docs/secret.md +++ b/docs/my-website/docs/secret.md @@ -1,8 +1,4 @@ -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; -import Image from '@theme/IdealImage'; - -# Secret Manager +# Secret Managers Overview :::info @@ -14,359 +10,19 @@ import Image from '@theme/IdealImage'; ::: -LiteLLM supports **reading secrets (eg. `OPENAI_API_KEY`)** and **writing secrets (eg. Virtual Keys)** from Azure Key Vault, Google Secret Manager, Hashicorp Vault, and AWS Secret Manager. +LiteLLM supports **reading secrets (eg. `OPENAI_API_KEY`)** and **writing secrets (eg. Virtual Keys)** from Azure Key Vault, Google Secret Manager, Hashicorp Vault, CyberArk Conjur, and AWS Secret Manager. ## Supported Secret Managers -- AWS Key Management Service -- AWS Secret Manager -- [Azure Key Vault](#azure-key-vault) -- [Google Secret Manager](#google-secret-manager) -- Google Key Management Service -- [Hashicorp Vault](#hashicorp-vault) - -## AWS Secret Manager - -Store your proxy keys in AWS Secret Manager. - - -| Feature | Support | Description | -|---------|----------|-------------| -| Reading Secrets | ✅ | Read secrets e.g `OPENAI_API_KEY` | -| Writing Secrets | ✅ | Store secrets e.g `Virtual Keys` | - -#### Proxy Usage - -1. Save AWS Credentials in your environment -```bash -os.environ["AWS_ACCESS_KEY_ID"] = "" # Access key -os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key -os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2 -``` - -2. Enable AWS Secret Manager in config. - - - - -```yaml -general_settings: - master_key: os.environ/litellm_master_key - key_management_system: "aws_secret_manager" # 👈 KEY CHANGE - key_management_settings: - hosted_keys: ["litellm_master_key"] # 👈 Specify which env keys you stored on AWS - -``` - - - - - -This will only store virtual keys in AWS Secret Manager. No keys will be read from AWS Secret Manager. - -```yaml -general_settings: - key_management_system: "aws_secret_manager" # 👈 KEY CHANGE - key_management_settings: - store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager - prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager - access_mode: "write_only" # Literal["read_only", "write_only", "read_and_write"] - description: "litellm virtual key" # OPTIONAL, if set will set this as the description for all virtual keys - tags: # OPTIONAL, if set will set this as the tags for all virtual keys - Environment: "Prod" - Owner: "AI Platform team" -``` - - - -```yaml -general_settings: - master_key: os.environ/litellm_master_key - key_management_system: "aws_secret_manager" # 👈 KEY CHANGE - key_management_settings: - store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager - prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager - access_mode: "read_and_write" # Literal["read_only", "write_only", "read_and_write"] - hosted_keys: ["litellm_master_key"] # OPTIONAL. Specify which env keys you stored on AWS -``` - - - - -3. Run proxy - -```bash -litellm --config /path/to/config.yaml -``` - - -#### Using K/V pairs in 1 AWS Secret - -You can read multiple keys from a single AWS Secret using the `primary_secret_name` parameter: - -```yaml -general_settings: - key_management_system: "aws_secret_manager" - key_management_settings: - hosted_keys: [ - "OPENAI_API_KEY_MODEL_1", - "OPENAI_API_KEY_MODEL_2", - ] - primary_secret_name: "litellm_secrets" # 👈 Read multiple keys from one JSON secret -``` - -The `primary_secret_name` allows you to read multiple keys from a single AWS Secret as a JSON object. For example, the "litellm_secrets" would contain: - -```json -{ - "OPENAI_API_KEY_MODEL_1": "sk-key1...", - "OPENAI_API_KEY_MODEL_2": "sk-key2..." -} -``` - -This reduces the number of AWS Secrets you need to manage. - - -## Hashicorp Vault - - -| Feature | Support | Description | -|---------|----------|-------------| -| Reading Secrets | ✅ | Read secrets e.g `OPENAI_API_KEY` | -| Writing Secrets | ✅ | Store secrets e.g `Virtual Keys` | - -Read secrets from [Hashicorp Vault](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2) - -**Step 1.** Add Hashicorp Vault details in your environment - -LiteLLM supports two methods of authentication: - -1. TLS cert authentication - `HCP_VAULT_CLIENT_CERT` and `HCP_VAULT_CLIENT_KEY` -2. Token authentication - `HCP_VAULT_TOKEN` - -```bash -HCP_VAULT_ADDR="https://test-cluster-public-vault-0f98180c.e98296b2.z1.hashicorp.cloud:8200" -HCP_VAULT_NAMESPACE="admin" - -# Authentication via TLS cert -HCP_VAULT_CLIENT_CERT="path/to/client.pem" -HCP_VAULT_CLIENT_KEY="path/to/client.key" - -# OR - Authentication via token -HCP_VAULT_TOKEN="hvs.CAESIG52gL6ljBSdmq*****" - - -# OPTIONAL -HCP_VAULT_REFRESH_INTERVAL="86400" # defaults to 86400, frequency of cache refresh for Hashicorp Vault -``` - -**Step 2.** Add to proxy config.yaml - -```yaml -general_settings: - key_management_system: "hashicorp_vault" - - # [OPTIONAL SETTINGS] - key_management_settings: - store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager - prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager - access_mode: "read_and_write" # Literal["read_only", "write_only", "read_and_write"] -``` - -**Step 3.** Start + test proxy - -``` -$ litellm --config /path/to/config.yaml -``` - -[Quick Test Proxy](./proxy/user_keys) - - -#### How it works - -**Reading Secrets** -LiteLLM reads secrets from Hashicorp Vault's KV v2 engine using the following URL format: -``` -{VAULT_ADDR}/v1/{NAMESPACE}/secret/data/{SECRET_NAME} -``` - -For example, if you have: -- `HCP_VAULT_ADDR="https://vault.example.com:8200"` -- `HCP_VAULT_NAMESPACE="admin"` -- Secret name: `AZURE_API_KEY` - - -LiteLLM will look up: -``` -https://vault.example.com:8200/v1/admin/secret/data/AZURE_API_KEY -``` - -#### Expected Secret Format -LiteLLM expects all secrets to be stored as a JSON object with a `key` field containing the secret value. - -For example, for `AZURE_API_KEY`, the secret should be stored as: - -```json -{ - "key": "sk-1234" -} -``` - - - -**Writing Secrets** - -When a Virtual Key is Created / Deleted on LiteLLM, LiteLLM will automatically create / delete the secret in Hashicorp Vault. - -- Create Virtual Key on LiteLLM either through the LiteLLM Admin UI or API - - - - -- Check Hashicorp Vault for secret - -LiteLLM stores secret under the `prefix_for_stored_virtual_keys` path (default: `litellm/`) - - - - -## Azure Key Vault - -#### Usage with LiteLLM Proxy Server - -1. Install Proxy dependencies -```bash -pip install 'litellm[proxy]' 'litellm[extra_proxy]' -``` - -2. Save Azure details in your environment -```bash -export["AZURE_CLIENT_ID"]="your-azure-app-client-id" -export["AZURE_CLIENT_SECRET"]="your-azure-app-client-secret" -export["AZURE_TENANT_ID"]="your-azure-tenant-id" -export["AZURE_KEY_VAULT_URI"]="your-azure-key-vault-uri" -``` - -3. Add to proxy config.yaml -```yaml -model_list: - - model_name: "my-azure-models" # model alias - litellm_params: - model: "azure/" - api_key: "os.environ/AZURE-API-KEY" # reads from key vault - get_secret("AZURE_API_KEY") - api_base: "os.environ/AZURE-API-BASE" # reads from key vault - get_secret("AZURE_API_BASE") - -general_settings: - key_management_system: "azure_key_vault" -``` - -You can now test this by starting your proxy: -```bash -litellm --config /path/to/config.yaml -``` - -[Quick Test Proxy](./proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain-langchain-js) - -## Google Secret Manager - -Support for [Google Secret Manager](https://cloud.google.com/security/products/secret-manager) - - -1. Save Google Secret Manager details in your environment - -```shell -GOOGLE_SECRET_MANAGER_PROJECT_ID="your-project-id-on-gcp" # example: adroit-crow-413218 -``` - -Optional Params - -```shell -export GOOGLE_SECRET_MANAGER_REFRESH_INTERVAL = "" # (int) defaults to 86400 -export GOOGLE_SECRET_MANAGER_ALWAYS_READ_SECRET_MANAGER = "" # (str) set to "true" if you want to always read from google secret manager without using in memory caching. NOT RECOMMENDED in PROD -``` - -2. Add to proxy config.yaml -```yaml -model_list: - - model_name: fake-openai-endpoint - litellm_params: - model: openai/fake - api_base: https://exampleopenaiendpoint-production.up.railway.app/ - api_key: os.environ/OPENAI_API_KEY # this will be read from Google Secret Manager - -general_settings: - key_management_system: "google_secret_manager" -``` - -You can now test this by starting your proxy: -```bash -litellm --config /path/to/config.yaml -``` - -[Quick Test Proxy](./proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain-langchain-js) - - -## Google Key Management Service - -Use encrypted keys from Google KMS on the proxy - -Step 1. Add keys to env -``` -export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json" -export GOOGLE_KMS_RESOURCE_NAME="projects/*/locations/*/keyRings/*/cryptoKeys/*" -export PROXY_DATABASE_URL_ENCRYPTED=b'\n$\x00D\xac\xb4/\x8e\xc...' -``` - -Step 2: Update Config - -```yaml -general_settings: - key_management_system: "google_kms" - database_url: "os.environ/PROXY_DATABASE_URL_ENCRYPTED" - master_key: sk-1234 -``` - -Step 3: Start + test proxy - -``` -$ litellm --config /path/to/config.yaml -``` - -And in another terminal -``` -$ litellm --test -``` - -[Quick Test Proxy](./proxy/user_keys) - - -## AWS Key Management V1 - -:::tip - -[BETA] AWS Key Management v2 is on the enterprise tier. Go [here for docs](./proxy/enterprise.md#beta-aws-key-manager---key-decryption) - -::: - -Use AWS KMS to storing a hashed copy of your Proxy Master Key in the environment. - -```bash -export LITELLM_MASTER_KEY="djZ9xjVaZ..." # 👈 ENCRYPTED KEY -export AWS_REGION_NAME="us-west-2" -``` - -```yaml -general_settings: - key_management_system: "aws_kms" - key_management_settings: - hosted_keys: ["LITELLM_MASTER_KEY"] # 👈 WHICH KEYS ARE STORED ON KMS -``` - -[**See Decryption Code**](https://github.com/BerriAI/litellm/blob/a2da2a8f168d45648b61279d4795d647d94f90c9/litellm/utils.py#L10182) - -## **All Secret Manager Settings** +- [AWS Key Management Service](./secret_managers/aws_kms) +- [AWS Secret Manager](./secret_managers/aws_secret_manager) +- [Azure Key Vault](./secret_managers/azure_key_vault) +- [CyberArk Conjur](./secret_managers/cyberark) +- [Google Secret Manager](./secret_managers/google_secret_manager) +- [Google Key Management Service](./secret_managers/google_kms) +- [Hashicorp Vault](./secret_managers/hashicorp_vault) + +## All Secret Manager Settings All settings related to secret management diff --git a/docs/my-website/docs/secret_managers/aws_kms.md b/docs/my-website/docs/secret_managers/aws_kms.md new file mode 100644 index 0000000000..79dc80897f --- /dev/null +++ b/docs/my-website/docs/secret_managers/aws_kms.md @@ -0,0 +1,34 @@ +# AWS Key Management V1 + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +:::tip + +[BETA] AWS Key Management v2 is on the enterprise tier. Go [here for docs](../proxy/enterprise.md#beta-aws-key-manager---key-decryption) + +::: + +Use AWS KMS to storing a hashed copy of your Proxy Master Key in the environment. + +```bash +export LITELLM_MASTER_KEY="djZ9xjVaZ..." # 👈 ENCRYPTED KEY +export AWS_REGION_NAME="us-west-2" +``` + +```yaml +general_settings: + key_management_system: "aws_kms" + key_management_settings: + hosted_keys: ["LITELLM_MASTER_KEY"] # 👈 WHICH KEYS ARE STORED ON KMS +``` + +[**See Decryption Code**](https://github.com/BerriAI/litellm/blob/a2da2a8f168d45648b61279d4795d647d94f90c9/litellm/utils.py#L10182) + diff --git a/docs/my-website/docs/secret_managers/aws_secret_manager.md b/docs/my-website/docs/secret_managers/aws_secret_manager.md new file mode 100644 index 0000000000..44fa23a4ae --- /dev/null +++ b/docs/my-website/docs/secret_managers/aws_secret_manager.md @@ -0,0 +1,112 @@ +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# AWS Secret Manager + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +Store your proxy keys in AWS Secret Manager. + +| Feature | Support | Description | +|---------|----------|-------------| +| Reading Secrets | ✅ | Read secrets e.g `OPENAI_API_KEY` | +| Writing Secrets | ✅ | Store secrets e.g `Virtual Keys` | + +## Proxy Usage + +1. Save AWS Credentials in your environment +```bash +os.environ["AWS_ACCESS_KEY_ID"] = "" # Access key +os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key +os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2 +``` + +2. Enable AWS Secret Manager in config. + + + + +```yaml +general_settings: + master_key: os.environ/litellm_master_key + key_management_system: "aws_secret_manager" # 👈 KEY CHANGE + key_management_settings: + hosted_keys: ["litellm_master_key"] # 👈 Specify which env keys you stored on AWS + +``` + + + + + +This will only store virtual keys in AWS Secret Manager. No keys will be read from AWS Secret Manager. + +```yaml +general_settings: + key_management_system: "aws_secret_manager" # 👈 KEY CHANGE + key_management_settings: + store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager + prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager + access_mode: "write_only" # Literal["read_only", "write_only", "read_and_write"] + description: "litellm virtual key" # OPTIONAL, if set will set this as the description for all virtual keys + tags: # OPTIONAL, if set will set this as the tags for all virtual keys + Environment: "Prod" + Owner: "AI Platform team" +``` + + + +```yaml +general_settings: + master_key: os.environ/litellm_master_key + key_management_system: "aws_secret_manager" # 👈 KEY CHANGE + key_management_settings: + store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager + prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager + access_mode: "read_and_write" # Literal["read_only", "write_only", "read_and_write"] + hosted_keys: ["litellm_master_key"] # OPTIONAL. Specify which env keys you stored on AWS +``` + + + + +3. Run proxy + +```bash +litellm --config /path/to/config.yaml +``` + +## Using K/V pairs in 1 AWS Secret + +You can read multiple keys from a single AWS Secret using the `primary_secret_name` parameter: + +```yaml +general_settings: + key_management_system: "aws_secret_manager" + key_management_settings: + hosted_keys: [ + "OPENAI_API_KEY_MODEL_1", + "OPENAI_API_KEY_MODEL_2", + ] + primary_secret_name: "litellm_secrets" # 👈 Read multiple keys from one JSON secret +``` + +The `primary_secret_name` allows you to read multiple keys from a single AWS Secret as a JSON object. For example, the "litellm_secrets" would contain: + +```json +{ + "OPENAI_API_KEY_MODEL_1": "sk-key1...", + "OPENAI_API_KEY_MODEL_2": "sk-key2..." +} +``` + +This reduces the number of AWS Secrets you need to manage. + diff --git a/docs/my-website/docs/secret_managers/azure_key_vault.md b/docs/my-website/docs/secret_managers/azure_key_vault.md new file mode 100644 index 0000000000..6ec95b378b --- /dev/null +++ b/docs/my-website/docs/secret_managers/azure_key_vault.md @@ -0,0 +1,47 @@ +# Azure Key Vault + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +## Usage with LiteLLM Proxy Server + +1. Install Proxy dependencies +```bash +pip install 'litellm[proxy]' 'litellm[extra_proxy]' +``` + +2. Save Azure details in your environment +```bash +export["AZURE_CLIENT_ID"]="your-azure-app-client-id" +export["AZURE_CLIENT_SECRET"]="your-azure-app-client-secret" +export["AZURE_TENANT_ID"]="your-azure-tenant-id" +export["AZURE_KEY_VAULT_URI"]="your-azure-key-vault-uri" +``` + +3. Add to proxy config.yaml +```yaml +model_list: + - model_name: "my-azure-models" # model alias + litellm_params: + model: "azure/" + api_key: "os.environ/AZURE-API-KEY" # reads from key vault - get_secret("AZURE_API_KEY") + api_base: "os.environ/AZURE-API-BASE" # reads from key vault - get_secret("AZURE_API_BASE") + +general_settings: + key_management_system: "azure_key_vault" +``` + +You can now test this by starting your proxy: +```bash +litellm --config /path/to/config.yaml +``` + +[Quick Test Proxy](../proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain-langchain-js) + diff --git a/docs/my-website/docs/secret_managers/cyberark.md b/docs/my-website/docs/secret_managers/cyberark.md new file mode 100644 index 0000000000..e226fec814 --- /dev/null +++ b/docs/my-website/docs/secret_managers/cyberark.md @@ -0,0 +1,163 @@ +# CyberArk Conjur + +import Image from '@theme/IdealImage'; + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +| Feature | Support | Description | +|---------|----------|-------------| +| Reading Secrets | ✅ | Read secrets e.g `OPENAI_API_KEY` | +| Writing Secrets | ✅ | Store secrets e.g `Virtual Keys` | +| Deleting Secrets | ❌ | Secrets must be removed via policy updates | + +Read and write secrets from [CyberArk Conjur](https://www.cyberark.com/products/secrets-management/) (self-hosted secrets manager) + +**Step 1.** Add CyberArk Conjur details in your environment + +LiteLLM supports two methods of authentication: + +1. API key authentication - `CYBERARK_API_KEY` (recommended) +2. Certificate authentication - `CYBERARK_CLIENT_CERT` and `CYBERARK_CLIENT_KEY` + +```bash +CYBERARK_API_BASE="http://your-conjur-instance:8080" +CYBERARK_ACCOUNT="default" +CYBERARK_USERNAME="admin" + +# Authentication via API key (recommended) +CYBERARK_API_KEY="your-api-key-here" + +# OR - Authentication via certificate +CYBERARK_CLIENT_CERT="path/to/client.pem" +CYBERARK_CLIENT_KEY="path/to/client.key" + +# OPTIONAL +CYBERARK_REFRESH_INTERVAL="300" # defaults to 300 seconds (5 minutes), frequency of token refresh +``` + +**Step 2.** Add to proxy config.yaml + +```yaml +general_settings: + key_management_system: "cyberark" + + # [OPTIONAL SETTINGS] + key_management_settings: + store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager + prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager + access_mode: "read_and_write" # Literal["read_only", "write_only", "read_and_write"] +``` + +**Step 3.** Start + test proxy + +```bash +$ litellm --config /path/to/config.yaml +``` + +[Quick Test Proxy](../proxy/user_keys) + +## Writing Virtual Keys to CyberArk + +When you create a virtual key in the LiteLLM UI, it automatically gets stored in CyberArk Conjur. + +**Step 1:** Create a virtual key in the LiteLLM Admin UI + +In this example, we create a key named `litellm-cyber-ark-secret-key`: + +Creating virtual key in LiteLLM UI + +**Step 2:** Verify the secret exists in CyberArk + +You can verify the virtual key was stored in CyberArk by querying the secrets API: + +```bash +TOKEN=$(curl -s -X POST http://0.0.0.0:8080/authn/default/admin/authenticate \ + -d "your-api-key" | base64 | tr -d '\n') + +curl -H "Authorization: Token token=\"$TOKEN\"" \ + "http://0.0.0.0:8080/resources/default/variable" | jq . +``` + +The response shows `litellm-cyber-ark-secret-key` exists in CyberArk: + +Virtual key stored in CyberArk API + +The virtual key is stored with the full path: `default:variable:litellm/litellm-cyber-ark-secret-key` + +## How it works + +**Authentication** + +CyberArk Conjur uses a two-step authentication process: + +1. LiteLLM authenticates with your API key to get a session token +2. The session token (base64-encoded) is used for subsequent API requests +3. Tokens expire after ~8 minutes, so LiteLLM caches and refreshes them automatically + +**Reading Secrets** + +LiteLLM reads secrets from CyberArk Conjur using the following URL format: + +``` +{CYBERARK_API_BASE}/secrets/{ACCOUNT}/variable/{SECRET_NAME} +``` + +For example, if you have: +- `CYBERARK_API_BASE="http://conjur.example.com:8080"` +- `CYBERARK_ACCOUNT="default"` +- Secret name: `AZURE_API_KEY` + +LiteLLM will look up: +``` +http://conjur.example.com:8080/secrets/default/variable/AZURE_API_KEY +``` + +**Writing Secrets** + +When a Virtual Key is created on LiteLLM, the following happens automatically: + +1. LiteLLM creates a policy entry to define the variable in Conjur (if it doesn't exist) +2. LiteLLM sets the secret value via the Conjur API + +LiteLLM stores secrets under the `prefix_for_stored_virtual_keys` path (default: `litellm/`) + +For example, a virtual key would be stored as: `litellm/virtual-key-name` + +**Working curl examples** + +Authenticate and get a token: +```bash +TOKEN=$(curl -s -X POST http://conjur.example.com:8080/authn/default/admin/authenticate \ + -d "your-api-key" | base64 | tr -d '\n') +``` + +Read a secret: +```bash +curl -H "Authorization: Token token=\"$TOKEN\"" \ + "http://conjur.example.com:8080/secrets/default/variable/test-secret" +``` + +Write a secret: +```bash +curl -X POST \ + -H "Authorization: Token token=\"$TOKEN\"" \ + --data "my-secret-value" \ + "http://conjur.example.com:8080/secrets/default/variable/test-secret" +``` + +**Important Notes** + +- Variables must be defined in a Conjur policy before setting their values +- LiteLLM automatically creates policy entries when writing new secrets +- Secret names with slashes (e.g., `litellm/key`) are automatically URL-encoded +- CyberArk Conjur does not support direct secret deletion via API (must use policy updates) +- Session tokens are cached for 5 minutes by default to minimize API calls + diff --git a/docs/my-website/docs/secret_managers/google_kms.md b/docs/my-website/docs/secret_managers/google_kms.md new file mode 100644 index 0000000000..0c6f66846f --- /dev/null +++ b/docs/my-website/docs/secret_managers/google_kms.md @@ -0,0 +1,43 @@ +# Google Key Management Service + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +Use encrypted keys from Google KMS on the proxy + +Step 1. Add keys to env +``` +export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json" +export GOOGLE_KMS_RESOURCE_NAME="projects/*/locations/*/keyRings/*/cryptoKeys/*" +export PROXY_DATABASE_URL_ENCRYPTED=b'\n$\x00D\xac\xb4/\x8e\xc...' +``` + +Step 2: Update Config + +```yaml +general_settings: + key_management_system: "google_kms" + database_url: "os.environ/PROXY_DATABASE_URL_ENCRYPTED" + master_key: sk-1234 +``` + +Step 3: Start + test proxy + +``` +$ litellm --config /path/to/config.yaml +``` + +And in another terminal +``` +$ litellm --test +``` + +[Quick Test Proxy](../proxy/user_keys) + diff --git a/docs/my-website/docs/secret_managers/google_secret_manager.md b/docs/my-website/docs/secret_managers/google_secret_manager.md new file mode 100644 index 0000000000..a545e7a85b --- /dev/null +++ b/docs/my-website/docs/secret_managers/google_secret_manager.md @@ -0,0 +1,47 @@ +# Google Secret Manager + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +Support for [Google Secret Manager](https://cloud.google.com/security/products/secret-manager) + +1. Save Google Secret Manager details in your environment + +```shell +GOOGLE_SECRET_MANAGER_PROJECT_ID="your-project-id-on-gcp" # example: adroit-crow-413218 +``` + +Optional Params + +```shell +export GOOGLE_SECRET_MANAGER_REFRESH_INTERVAL = "" # (int) defaults to 86400 +export GOOGLE_SECRET_MANAGER_ALWAYS_READ_SECRET_MANAGER = "" # (str) set to "true" if you want to always read from google secret manager without using in memory caching. NOT RECOMMENDED in PROD +``` + +2. Add to proxy config.yaml +```yaml +model_list: + - model_name: fake-openai-endpoint + litellm_params: + model: openai/fake + api_base: https://exampleopenaiendpoint-production.up.railway.app/ + api_key: os.environ/OPENAI_API_KEY # this will be read from Google Secret Manager + +general_settings: + key_management_system: "google_secret_manager" +``` + +You can now test this by starting your proxy: +```bash +litellm --config /path/to/config.yaml +``` + +[Quick Test Proxy](../proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain-langchain-js) + diff --git a/docs/my-website/docs/secret_managers/hashicorp_vault.md b/docs/my-website/docs/secret_managers/hashicorp_vault.md new file mode 100644 index 0000000000..4d0ef05a32 --- /dev/null +++ b/docs/my-website/docs/secret_managers/hashicorp_vault.md @@ -0,0 +1,115 @@ +import Image from '@theme/IdealImage'; + +# Hashicorp Vault + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +| Feature | Support | Description | +|---------|----------|-------------| +| Reading Secrets | ✅ | Read secrets e.g `OPENAI_API_KEY` | +| Writing Secrets | ✅ | Store secrets e.g `Virtual Keys` | + +Read secrets from [Hashicorp Vault](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2) + +**Step 1.** Add Hashicorp Vault details in your environment + +LiteLLM supports two methods of authentication: + +1. TLS cert authentication - `HCP_VAULT_CLIENT_CERT` and `HCP_VAULT_CLIENT_KEY` +2. Token authentication - `HCP_VAULT_TOKEN` + +```bash +HCP_VAULT_ADDR="https://test-cluster-public-vault-0f98180c.e98296b2.z1.hashicorp.cloud:8200" +HCP_VAULT_NAMESPACE="admin" + +# Authentication via TLS cert +HCP_VAULT_CLIENT_CERT="path/to/client.pem" +HCP_VAULT_CLIENT_KEY="path/to/client.key" + +# OR - Authentication via token +HCP_VAULT_TOKEN="hvs.CAESIG52gL6ljBSdmq*****" + + +# OPTIONAL +HCP_VAULT_REFRESH_INTERVAL="86400" # defaults to 86400, frequency of cache refresh for Hashicorp Vault +``` + +**Step 2.** Add to proxy config.yaml + +```yaml +general_settings: + key_management_system: "hashicorp_vault" + + # [OPTIONAL SETTINGS] + key_management_settings: + store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager + prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager + access_mode: "read_and_write" # Literal["read_only", "write_only", "read_and_write"] +``` + +**Step 3.** Start + test proxy + +``` +$ litellm --config /path/to/config.yaml +``` + +[Quick Test Proxy](../proxy/user_keys) + + +## How it works + +**Reading Secrets** + +LiteLLM reads secrets from Hashicorp Vault's KV v2 engine using the following URL format: +``` +{VAULT_ADDR}/v1/{NAMESPACE}/secret/data/{SECRET_NAME} +``` + +For example, if you have: +- `HCP_VAULT_ADDR="https://vault.example.com:8200"` +- `HCP_VAULT_NAMESPACE="admin"` +- Secret name: `AZURE_API_KEY` + + +LiteLLM will look up: +``` +https://vault.example.com:8200/v1/admin/secret/data/AZURE_API_KEY +``` + +### Expected Secret Format + +LiteLLM expects all secrets to be stored as a JSON object with a `key` field containing the secret value. + +For example, for `AZURE_API_KEY`, the secret should be stored as: + +```json +{ + "key": "sk-1234" +} +``` + + + +**Writing Secrets** + +When a Virtual Key is Created / Deleted on LiteLLM, LiteLLM will automatically create / delete the secret in Hashicorp Vault. + +- Create Virtual Key on LiteLLM either through the LiteLLM Admin UI or API + + + + +- Check Hashicorp Vault for secret + +LiteLLM stores secret under the `prefix_for_stored_virtual_keys` path (default: `litellm/`) + + + diff --git a/docs/my-website/docs/secret_managers/overview.md b/docs/my-website/docs/secret_managers/overview.md new file mode 100644 index 0000000000..fa1e82b1d0 --- /dev/null +++ b/docs/my-website/docs/secret_managers/overview.md @@ -0,0 +1,47 @@ +# Secret Managers Overview + +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +LiteLLM supports **reading secrets (eg. `OPENAI_API_KEY`)** and **writing secrets (eg. Virtual Keys)** from Azure Key Vault, Google Secret Manager, Hashicorp Vault, CyberArk Conjur, and AWS Secret Manager. + +## Supported Secret Managers + +- [AWS Key Management Service](./aws_kms) +- [AWS Secret Manager](./aws_secret_manager) +- [Azure Key Vault](./azure_key_vault) +- [CyberArk Conjur](./cyberark) +- [Google Secret Manager](./google_secret_manager) +- [Google Key Management Service](./google_kms) +- [Hashicorp Vault](./hashicorp_vault) + +## All Secret Manager Settings + +All settings related to secret management + +```yaml +general_settings: + key_management_system: "aws_secret_manager" # REQUIRED + key_management_settings: + + # Storing Virtual Keys Settings + store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager + prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL.I f set, this prefix will be used for stored virtual keys in the secret manager + + # Access Mode Settings + access_mode: "write_only" # OPTIONAL. Literal["read_only", "write_only", "read_and_write"]. Defaults to "read_only" + + # Hosted Keys Settings + hosted_keys: ["litellm_master_key"] # OPTIONAL. Specify which env keys you stored on AWS + + # K/V pairs in 1 AWS Secret Settings + primary_secret_name: "litellm_secrets" # OPTIONAL. Read multiple keys from one JSON secret on AWS Secret Manager +``` + diff --git a/docs/my-website/img/cyberark1.png b/docs/my-website/img/cyberark1.png new file mode 100644 index 0000000000..382b2cda99 Binary files /dev/null and b/docs/my-website/img/cyberark1.png differ diff --git a/docs/my-website/img/cyberark2.png b/docs/my-website/img/cyberark2.png new file mode 100644 index 0000000000..85745a4bcb Binary files /dev/null and b/docs/my-website/img/cyberark2.png differ diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index 41bf9397b0..00a69fcd6d 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -260,7 +260,14 @@ const sidebars = { type: "category", label: "Secret Managers", items: [ - "secret", + "secret_managers/overview", + "secret_managers/aws_secret_manager", + "secret_managers/aws_kms", + "secret_managers/azure_key_vault", + "secret_managers/cyberark", + "secret_managers/google_secret_manager", + "secret_managers/google_kms", + "secret_managers/hashicorp_vault", "oidc" ] }, diff --git a/litellm/proxy/proxy_config.yaml b/litellm/proxy/proxy_config.yaml index 41df2e6c2a..3b7a04a647 100644 --- a/litellm/proxy/proxy_config.yaml +++ b/litellm/proxy/proxy_config.yaml @@ -1,3 +1,10 @@ +general_settings: + key_management_system: "cyberark" + key_management_settings: + store_virtual_keys: true + prefix_for_stored_virtual_keys: "litellm/" + access_mode: "read_and_write" + model_list: - model_name: aws/anthropic/bedrock-claude-3-5-sonnet-v1 litellm_params: diff --git a/litellm/proxy/proxy_server.py b/litellm/proxy/proxy_server.py index 0019365e7e..a437cb705b 100644 --- a/litellm/proxy/proxy_server.py +++ b/litellm/proxy/proxy_server.py @@ -2658,6 +2658,12 @@ class ProxyConfig: ) HashicorpSecretManager() + elif key_management_system == KeyManagementSystem.CYBERARK.value: + from litellm.secret_managers.cyberark_secret_manager import ( + CyberArkSecretManager, + ) + + CyberArkSecretManager() else: raise ValueError("Invalid Key Management System selected") diff --git a/litellm/secret_managers/cyberark_secret_manager.py b/litellm/secret_managers/cyberark_secret_manager.py new file mode 100644 index 0000000000..5aad215453 --- /dev/null +++ b/litellm/secret_managers/cyberark_secret_manager.py @@ -0,0 +1,354 @@ +import base64 +import os +from typing import Any, Dict, Optional, Union +from urllib.parse import quote + +import httpx + +import litellm +from litellm._logging import verbose_logger +from litellm.caching import InMemoryCache +from litellm.llms.custom_httpx.http_handler import ( + _get_httpx_client, + get_async_httpx_client, + httpxSpecialProvider, +) +from litellm.proxy._types import KeyManagementSystem + +from .base_secret_manager import BaseSecretManager + + +class CyberArkSecretManager(BaseSecretManager): + def __init__(self): + from litellm.proxy.proxy_server import CommonProxyErrors, premium_user + + # CyberArk Conjur-specific config + self.conjur_addr = os.getenv("CYBERARK_API_BASE", "http://127.0.0.1:8080") + self.conjur_account = os.getenv("CYBERARK_ACCOUNT", "default") + self.conjur_username = os.getenv("CYBERARK_USERNAME", "admin") + self.conjur_api_key = os.getenv("CYBERARK_API_KEY", "") + + # Optional config for certificate-based auth + self.tls_cert_path = os.getenv("CYBERARK_CLIENT_CERT", "") + self.tls_key_path = os.getenv("CYBERARK_CLIENT_KEY", "") + + # Validate environment + if not self.conjur_api_key and not ( + self.tls_cert_path and self.tls_key_path + ): + raise ValueError( + "Missing CyberArk credentials. Please set CYBERARK_API_KEY or both CYBERARK_CLIENT_CERT and CYBERARK_CLIENT_KEY in your environment." + ) + + litellm.secret_manager_client = self + litellm._key_management_system = KeyManagementSystem.CYBERARK + + # Tokens expire after ~8 minutes, so we cache for 5 minutes to be safe + _refresh_interval = int(os.environ.get("CYBERARK_REFRESH_INTERVAL", "300")) + self.cache = InMemoryCache(default_ttl=_refresh_interval) + + if premium_user is not True: + raise ValueError( + f"CyberArk secret manager is only available for premium users. {CommonProxyErrors.not_premium_user.value}" + ) + + def _authenticate(self) -> str: + """ + Authenticate with CyberArk Conjur and get a session token. + + The token is a JSON object that must be base64-encoded for use in subsequent requests. + + Returns: + str: Base64-encoded session token + """ + # Check if we have a cached token + cached_token = self.cache.get_cache("cyberark_auth_token") + if cached_token is not None: + return cached_token + + verbose_logger.debug("Authenticating with CyberArk Conjur...") + auth_url = f"{self.conjur_addr}/authn/{self.conjur_account}/{self.conjur_username}/authenticate" + + try: + if self.tls_cert_path and self.tls_key_path: + # Certificate-based authentication + client = httpx.Client(cert=(self.tls_cert_path, self.tls_key_path)) + resp = client.post(auth_url, content=self.conjur_api_key) + else: + # API key authentication + client = _get_httpx_client() + resp = client.post(auth_url, content=self.conjur_api_key) + + resp.raise_for_status() + + # The response is a JSON token that needs to be base64-encoded + token_json = resp.text + token_b64 = base64.b64encode(token_json.encode()).decode() + + verbose_logger.debug("Successfully authenticated with CyberArk Conjur.") + + # Cache the token for the refresh interval + self.cache.set_cache(key="cyberark_auth_token", value=token_b64) + + return token_b64 + except Exception as e: + raise RuntimeError(f"Could not authenticate to CyberArk Conjur: {e}") + + def _get_request_headers(self) -> dict: + """ + Get headers for CyberArk API requests including authentication. + + Returns: + dict: Headers with authentication token + """ + token = self._authenticate() + return {"Authorization": f'Token token="{token}"'} + + def _ensure_variable_exists(self, secret_name: str) -> None: + """ + Ensure a variable exists in CyberArk Conjur by creating a policy entry if needed. + + Args: + secret_name: Name of the variable to ensure exists + """ + # In production, we'd check if the variable exists first + # For now, we'll attempt to create it and ignore if it already exists + policy_url = f"{self.conjur_addr}/policies/{self.conjur_account}/policy/root" + policy_yaml = f"- !variable {secret_name}\n" + + try: + client = _get_httpx_client() + resp = client.post( + policy_url, + headers={ + **self._get_request_headers(), + "Content-Type": "application/x-yaml", + }, + content=policy_yaml, + ) + resp.raise_for_status() + verbose_logger.debug("Created policy entry for a variable.") + except httpx.HTTPStatusError as e: + # Variable might already exist, which is fine + if e.response.status_code in [409, 422]: + verbose_logger.debug( + "A variable already exists or policy conflict (expected)" + ) + else: + verbose_logger.warning( + f"Could not ensure variable exists: {e.response.status_code} - {e.response.text}" + ) + except Exception as e: + verbose_logger.warning(f"Error ensuring variable exists: {e}") + + def get_url(self, secret_name: str) -> str: + """ + Build the URL for accessing a secret in CyberArk Conjur. + + Args: + secret_name: Name of the secret (will be URL-encoded) + + Returns: + str: Full URL for the secret + """ + # URL-encode the secret name to handle slashes and special characters + encoded_name = quote(secret_name, safe="") + return ( + f"{self.conjur_addr}/secrets/{self.conjur_account}/variable/{encoded_name}" + ) + + async def async_read_secret( + self, + secret_name: str, + optional_params: Optional[dict] = None, + timeout: Optional[Union[float, httpx.Timeout]] = None, + ) -> Optional[str]: + """ + Reads a secret from CyberArk Conjur using an async HTTPX client. + + Args: + secret_name: Name/path of the secret to read + optional_params: Additional parameters (not used for Conjur) + timeout: Request timeout + + Returns: + Optional[str]: The secret value if found, None otherwise + """ + # Check cache first + if self.cache.get_cache(secret_name) is not None: + return self.cache.get_cache(secret_name) + + async_client = get_async_httpx_client( + llm_provider=httpxSpecialProvider.SecretManager, + ) + + try: + url = self.get_url(secret_name) + response = await async_client.get(url, headers=self._get_request_headers()) + response.raise_for_status() + + # CyberArk Conjur returns the raw secret value as text + secret_value = response.text + self.cache.set_cache(secret_name, secret_value) + return secret_value + + except httpx.HTTPStatusError as e: + if e.response.status_code == 404: + verbose_logger.debug( + f"Secret {secret_name} not found in CyberArk Conjur" + ) + else: + verbose_logger.exception( + f"Error reading secret from CyberArk Conjur: {e}" + ) + return None + except Exception as e: + verbose_logger.exception(f"Error reading secret from CyberArk Conjur: {e}") + return None + + def sync_read_secret( + self, + secret_name: str, + optional_params: Optional[dict] = None, + timeout: Optional[Union[float, httpx.Timeout]] = None, + ) -> Optional[str]: + """ + Reads a secret from CyberArk Conjur using a sync HTTPX client. + + Args: + secret_name: Name/path of the secret to read + optional_params: Additional parameters (not used for Conjur) + timeout: Request timeout + + Returns: + Optional[str]: The secret value if found, None otherwise + """ + # Check cache first + if self.cache.get_cache(secret_name) is not None: + return self.cache.get_cache(secret_name) + + sync_client = _get_httpx_client() + + try: + url = self.get_url(secret_name) + response = sync_client.get(url, headers=self._get_request_headers()) + response.raise_for_status() + + # CyberArk Conjur returns the raw secret value as text + secret_value = response.text + self.cache.set_cache(secret_name, secret_value) + return secret_value + + except httpx.HTTPStatusError as e: + if e.response.status_code == 404: + verbose_logger.debug( + f"Secret {secret_name} not found in CyberArk Conjur" + ) + else: + verbose_logger.exception( + f"Error reading secret from CyberArk Conjur: {e}" + ) + return None + except Exception as e: + verbose_logger.exception(f"Error reading secret from CyberArk Conjur: {e}") + return None + + async def async_write_secret( + self, + secret_name: str, + secret_value: str, + description: Optional[str] = None, + optional_params: Optional[dict] = None, + timeout: Optional[Union[float, httpx.Timeout]] = None, + tags: Optional[Union[dict, list]] = None, + ) -> Dict[str, Any]: + """ + Writes a secret to CyberArk Conjur using an async HTTPX client. + + Args: + secret_name: Name/path of the secret to write + secret_value: Value to store + description: Optional description (not used by Conjur) + optional_params: Additional parameters + timeout: Request timeout + tags: Optional tags (not used by Conjur) + + Returns: + dict: Response containing status and details of the operation + """ + async_client = get_async_httpx_client( + llm_provider=httpxSpecialProvider.SecretManager, + params={"timeout": timeout}, + ) + + try: + # Ensure the variable exists in the policy first + self._ensure_variable_exists(secret_name) + + # Now set the secret value + url = self.get_url(secret_name) + response = await async_client.post( + url=url, headers=self._get_request_headers(), content=secret_value + ) + response.raise_for_status() + + # Update cache + self.cache.set_cache(secret_name, secret_value) + + return { + "status": "success", + "message": f"Secret {secret_name} written successfully", + } + except Exception as e: + verbose_logger.exception(f"Error writing secret to CyberArk Conjur: {e}") + return {"status": "error", "message": str(e)} + + async def async_rotate_secret( + self, + current_secret_name: str, + new_secret_name: str, + new_secret_value: str, + optional_params: Optional[Dict] = None, + timeout: Optional[Union[float, httpx.Timeout]] = None, + ) -> Dict: + """ + CyberArk Conjur does not have built-in secret rotation. + + Raises: + NotImplementedError: Always raised + """ + raise NotImplementedError("CyberArk Conjur does not support secret rotation") + + async def async_delete_secret( + self, + secret_name: str, + recovery_window_in_days: Optional[int] = 7, + optional_params: Optional[dict] = None, + timeout: Optional[Union[float, httpx.Timeout]] = None, + ) -> dict: + """ + CyberArk Conjur does not support direct secret deletion via API. + Secrets can only be removed through policy updates. + + Args: + secret_name: Name of the secret + recovery_window_in_days: Not used + optional_params: Additional parameters + timeout: Request timeout + + Returns: + dict: Response indicating operation not supported + """ + verbose_logger.warning( + "CyberArk Conjur does not support direct secret deletion. " + "Secrets must be removed through policy updates." + ) + + # Clear from cache + self.cache.delete_cache(secret_name) + + return { + "status": "not_supported", + "message": "CyberArk Conjur does not support direct secret deletion. Use policy updates to remove variables.", + } + diff --git a/litellm/secret_managers/main.py b/litellm/secret_managers/main.py index 6ecf5e370f..51a7f96ecb 100644 --- a/litellm/secret_managers/main.py +++ b/litellm/secret_managers/main.py @@ -1,6 +1,4 @@ import ast -import base64 -import binascii import os import traceback from typing import Any, Optional, Union @@ -14,6 +12,7 @@ from litellm.llms.custom_httpx.http_handler import HTTPHandler from litellm.secret_managers.get_azure_ad_token_provider import ( get_azure_ad_token_provider, ) +from litellm.secret_managers.secret_manager_handler import get_secret_from_manager from litellm.types.secret_managers.main import KeyManagementSystem oidc_cache = DualCache() @@ -22,13 +21,6 @@ oidc_cache = DualCache() ######### Secret Manager ############################ # checks if user has passed in a secret manager client # if passed in then checks the secret there -def _is_base64(s): - try: - return base64.b64encode(base64.b64decode(s)).decode() == s - except binascii.Error: - return False - - def str_to_bool(value: Optional[str]) -> Optional[bool]: """ Converts a string to a boolean if it's a recognized boolean string. @@ -222,86 +214,13 @@ def get_secret( # noqa: PLR0915 ): # allow user to specify which keys to check in hosted key manager key_manager = "local" - if ( - key_manager == KeyManagementSystem.AZURE_KEY_VAULT.value - or type(client).__module__ + "." + type(client).__name__ - == "azure.keyvault.secrets._client.SecretClient" - ): # support Azure Secret Client - from azure.keyvault.secrets import SecretClient - secret = client.get_secret(secret_name).value - elif ( - key_manager == KeyManagementSystem.GOOGLE_KMS.value - or client.__class__.__name__ == "KeyManagementServiceClient" - ): - encrypted_secret: Any = os.getenv(secret_name) - if encrypted_secret is None: - raise ValueError("Google KMS requires the encrypted secret to be in the environment!") - b64_flag = _is_base64(encrypted_secret) - if b64_flag is True: # if passed in as encoded b64 string - encrypted_secret = base64.b64decode(encrypted_secret) - ciphertext = encrypted_secret - else: - raise ValueError( - "Google KMS requires the encrypted secret to be encoded in base64" - ) # fix for this vulnerability https://huntr.com/bounties/ae623c2f-b64b-4245-9ed4-f13a0a5824ce - response = client.decrypt( - request={ - "name": litellm._google_kms_resource_name, - "ciphertext": ciphertext, - } - ) - secret = response.plaintext.decode("utf-8") # assumes the original value was encoded with utf-8 - elif key_manager == KeyManagementSystem.AWS_KMS.value: - """ - Only check the tokens which start with 'aws_kms/'. This prevents latency impact caused by checking all keys. - """ - encrypted_value = os.getenv(secret_name, None) - if encrypted_value is None: - raise Exception("AWS KMS - Encrypted Value of Key={} is None".format(secret_name)) - # Decode the base64 encoded ciphertext - ciphertext_blob = base64.b64decode(encrypted_value) - - # Set up the parameters for the decrypt call - params = {"CiphertextBlob": ciphertext_blob} - # Perform the decryption - response = client.decrypt(**params) - - # Extract and decode the plaintext - plaintext = response["Plaintext"] - secret = plaintext.decode("utf-8") - if isinstance(secret, str): - secret = secret.strip() - elif key_manager == KeyManagementSystem.AWS_SECRET_MANAGER.value: - from litellm.secret_managers.aws_secret_manager_v2 import ( - AWSSecretsManagerV2, - ) - - if isinstance(client, AWSSecretsManagerV2): - secret = client.sync_read_secret( - secret_name=secret_name, - primary_secret_name=key_management_settings.primary_secret_name, - ) - print_verbose(f"get_secret_value_response: {secret}") - elif key_manager == KeyManagementSystem.GOOGLE_SECRET_MANAGER.value: - try: - secret = client.get_secret_from_google_secret_manager(secret_name) - print_verbose(f"secret from google secret manager: {secret}") - if secret is None: - raise ValueError(f"No secret found in Google Secret Manager for {secret_name}") - except Exception as e: - print_verbose(f"An error occurred - {str(e)}") - raise e - elif key_manager == KeyManagementSystem.HASHICORP_VAULT.value: - try: - secret = client.sync_read_secret(secret_name=secret_name) - if secret is None: - raise ValueError(f"No secret found in Hashicorp Secret Manager for {secret_name}") - except Exception as e: - print_verbose(f"An error occurred - {str(e)}") - raise e - elif key_manager == "local": - secret = os.getenv(secret_name) - else: # assume the default is infisicial client - secret = client.get_secret(secret_name).secret_value + # Delegate to the secret manager handler + secret = get_secret_from_manager( + client=client, + key_manager=key_manager, + secret_name=secret_name, + key_management_settings=key_management_settings, + ) except Exception as e: # check if it's in os.environ verbose_logger.error( f"Defaulting to os.environ value for key={secret_name}. An exception occurred - {str(e)}.\n\n{traceback.format_exc()}" diff --git a/litellm/secret_managers/secret_manager_handler.py b/litellm/secret_managers/secret_manager_handler.py new file mode 100644 index 0000000000..c150c6ac1d --- /dev/null +++ b/litellm/secret_managers/secret_manager_handler.py @@ -0,0 +1,150 @@ +""" +Secret Manager Handler + +Handles retrieving secrets from different secret management systems. +""" +import base64 +import os +from typing import Any, Optional + +import litellm +from litellm._logging import print_verbose +from litellm.types.secret_managers.main import KeyManagementSystem + + +def _is_base64(s): + """Check if a string is valid base64.""" + import binascii + try: + return base64.b64encode(base64.b64decode(s)).decode() == s + except binascii.Error: + return False + + +def get_secret_from_manager( + client: Any, + key_manager: str, + secret_name: str, + key_management_settings: Optional[Any] = None, +) -> Optional[str]: + """ + Get a secret from the configured secret manager. + + Args: + client: The secret manager client instance + key_manager: The type of key manager (e.g., "azure_key_vault", "google_kms", etc.) + secret_name: The name/path of the secret to retrieve + key_management_settings: Optional settings for the key management system + + Returns: + The secret value as a string, or None if not found + + Raises: + ValueError: If the secret cannot be retrieved or required parameters are missing + Exception: For other errors during secret retrieval + """ + secret = None + + if ( + key_manager == KeyManagementSystem.AZURE_KEY_VAULT.value + or type(client).__module__ + "." + type(client).__name__ + == "azure.keyvault.secrets._client.SecretClient" + ): # support Azure Secret Client - from azure.keyvault.secrets import SecretClient + secret = client.get_secret(secret_name).value + + elif ( + key_manager == KeyManagementSystem.GOOGLE_KMS.value + or client.__class__.__name__ == "KeyManagementServiceClient" + ): + encrypted_secret: Any = os.getenv(secret_name) + if encrypted_secret is None: + raise ValueError("Google KMS requires the encrypted secret to be in the environment!") + b64_flag = _is_base64(encrypted_secret) + if b64_flag is True: # if passed in as encoded b64 string + encrypted_secret = base64.b64decode(encrypted_secret) + ciphertext = encrypted_secret + else: + raise ValueError( + "Google KMS requires the encrypted secret to be encoded in base64" + ) # fix for this vulnerability https://huntr.com/bounties/ae623c2f-b64b-4245-9ed4-f13a0a5824ce + response = client.decrypt( + request={ + "name": litellm._google_kms_resource_name, + "ciphertext": ciphertext, + } + ) + secret = response.plaintext.decode("utf-8") # assumes the original value was encoded with utf-8 + + elif key_manager == KeyManagementSystem.AWS_KMS.value: + """ + Only check the tokens which start with 'aws_kms/'. This prevents latency impact caused by checking all keys. + """ + encrypted_value = os.getenv(secret_name, None) + if encrypted_value is None: + raise Exception("AWS KMS - Encrypted Value of Key={} is None".format(secret_name)) + # Decode the base64 encoded ciphertext + ciphertext_blob = base64.b64decode(encrypted_value) + + # Set up the parameters for the decrypt call + params = {"CiphertextBlob": ciphertext_blob} + # Perform the decryption + response = client.decrypt(**params) + + # Extract and decode the plaintext + plaintext = response["Plaintext"] + secret = plaintext.decode("utf-8") + if isinstance(secret, str): + secret = secret.strip() + + elif key_manager == KeyManagementSystem.AWS_SECRET_MANAGER.value: + from litellm.secret_managers.aws_secret_manager_v2 import ( + AWSSecretsManagerV2, + ) + + if isinstance(client, AWSSecretsManagerV2): + primary_secret_name = None + if key_management_settings is not None: + primary_secret_name = key_management_settings.primary_secret_name + + secret = client.sync_read_secret( + secret_name=secret_name, + primary_secret_name=primary_secret_name, + ) + print_verbose(f"get_secret_value_response: {secret}") + + elif key_manager == KeyManagementSystem.GOOGLE_SECRET_MANAGER.value: + try: + secret = client.get_secret_from_google_secret_manager(secret_name) + print_verbose(f"secret from google secret manager: {secret}") + if secret is None: + raise ValueError(f"No secret found in Google Secret Manager for {secret_name}") + except Exception as e: + print_verbose(f"An error occurred - {str(e)}") + raise e + + elif key_manager == KeyManagementSystem.HASHICORP_VAULT.value: + try: + secret = client.sync_read_secret(secret_name=secret_name) + if secret is None: + raise ValueError(f"No secret found in Hashicorp Secret Manager for {secret_name}") + except Exception as e: + print_verbose(f"An error occurred - {str(e)}") + raise e + + elif key_manager == KeyManagementSystem.CYBERARK.value: + try: + secret = client.sync_read_secret(secret_name=secret_name) + if secret is None: + raise ValueError(f"No secret found in CyberArk Secret Manager for {secret_name}") + except Exception as e: + print_verbose(f"An error occurred - {str(e)}") + raise e + + elif key_manager == "local": + secret = os.getenv(secret_name) + + else: # assume the default is infisicial client + secret = client.get_secret(secret_name).secret_value + + return secret + diff --git a/litellm/types/secret_managers/main.py b/litellm/types/secret_managers/main.py index fd7d8d4f01..dbedb48843 100644 --- a/litellm/types/secret_managers/main.py +++ b/litellm/types/secret_managers/main.py @@ -1,5 +1,5 @@ import enum -from typing import List, Literal, Optional, Dict +from typing import Dict, List, Literal, Optional from litellm.types.llms.base import LiteLLMPydanticObjectBase @@ -10,6 +10,7 @@ class KeyManagementSystem(enum.Enum): AWS_SECRET_MANAGER = "aws_secret_manager" GOOGLE_SECRET_MANAGER = "google_secret_manager" HASHICORP_VAULT = "hashicorp_vault" + CYBERARK = "cyberark" LOCAL = "local" AWS_KMS = "aws_kms" diff --git a/tests/litellm_utils_tests/test_cyberark.py b/tests/litellm_utils_tests/test_cyberark.py new file mode 100644 index 0000000000..d77ec85e0f --- /dev/null +++ b/tests/litellm_utils_tests/test_cyberark.py @@ -0,0 +1,54 @@ +""" +Integration test for CyberArk Conjur Secret Manager. +""" +import os +import sys +import pytest +from dotenv import load_dotenv + +load_dotenv() + +sys.path.insert(0, os.path.abspath("../..")) +from unittest.mock import patch +from litellm._uuid import uuid + +# Set up environment variables for testing +os.environ["CYBERARK_API_KEY"] = "2syke5r262b6je2f4et1x3jptmry3frfx83t65e6417zad632e5qq8a" +os.environ["CYBERARK_API_BASE"] = "http://0.0.0.0:8080" +os.environ["CYBERARK_ACCOUNT"] = "default" +os.environ["CYBERARK_USERNAME"] = "admin" + +from litellm.secret_managers.cyberark_secret_manager import CyberArkSecretManager + + +@pytest.mark.asyncio +async def test_cyberark_write_and_read_secret(): + """ + Integration test: Write a secret to CyberArk Conjur and read it back to validate. + """ + with patch("litellm.proxy.proxy_server.premium_user", True): + # Create CyberArk secret manager instance + cyberark_manager = CyberArkSecretManager() + + # Generate unique secret name and value + secret_name = f"test-secret-{uuid.uuid4()}" + secret_value = f"test-value-{uuid.uuid4()}" + + # Write the secret + write_response = await cyberark_manager.async_write_secret( + secret_name=secret_name, + secret_value=secret_value, + ) + # Avoid logging write_response to prevent leaking secret names + + # Validate write was successful + assert write_response["status"] == "success" + + # Read the secret back + read_value = cyberark_manager.sync_read_secret(secret_name=secret_name) + # Don't log secret value in clear text + + # Validate the secret exists and has the correct value + assert read_value is not None + assert read_value == secret_value +