tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-14 05:06:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
32,598 Commits 2,072 Branches 1,425 Tags
b5aff267d7d57d344bfe02ea2b4eff4e847a768d
Commit Graph

322 Commits

Author SHA1 Message Date
Julio Quinteros Pro 480974e0f9 fix: complete asyncio.iscoroutinefunction deprecation fix across codebase 
Replace all asyncio.iscoroutinefunction() calls with inspect.iscoroutinefunction()
to fix Python 3.16 deprecation warnings throughout the entire codebase.

Files updated:
- litellm/litellm_core_utils/logging_utils.py
- litellm/proxy/common_utils/performance_utils.py
- litellm/proxy/management_endpoints/key_management_endpoints.py (2 occurrences)
- litellm/proxy/management_endpoints/ui_sso.py
- litellm/litellm_core_utils/redact_messages.py
- litellm/integrations/custom_guardrail.py
- tests/proxy_unit_tests/test_response_polling_handler.py

This addresses Greptile's feedback about incomplete deprecation fixes.
All instances now use the standard library inspect.iscoroutinefunction()
which is the recommended approach and won't be deprecated.
 2026-02-17 18:20:44 -03:00
Sameer Kankute 791cef6d99 fix test_chat_completion 2026-02-17 20:26:28 +05:30
Sameer Kankute 288f7b860c fix test_allow_access_by_email 2026-02-17 20:19:39 +05:30
Julio Quinteros Pro 2d41b03f8b fix(test): mock environment variables for callback validation test 
The test test_proxy_config_state_post_init_callback_call was failing with:
```
ValidationError: 2 validation errors for TeamCallbackMetadata
callback_vars.langfuse_public_key
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
```

Root cause: The test uses environment variable references like
"os.environ/LANGFUSE_PUBLIC_KEY" which get resolved at runtime. In
parallel execution with --dist=loadscope, these environment variables
may not be set in all worker processes, causing the resolution to
return None, which fails Pydantic validation expecting strings.

Solution: Use monkeypatch to set the required environment variables
before the test runs. This ensures consistent behavior across all
test execution environments (local, CI, parallel workers).

Fixes test failure exposed by PR #21277.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-15 20:44:17 -03:00
yuneng-jiang 5a78486a15 tests 2026-02-13 22:01:28 -08:00
yuneng-jiang cd148dcb82 added access groups permission checks 2026-02-13 20:01:25 -08:00
Achilleas Athanasiou Fragkoulis cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492) 
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
 2026-02-12 19:39:04 +05:30
Sameer Kankute 9083b06ba7 Fix test_provider_specific_header_in_request 2026-02-11 16:56:26 +05:30
Harshit Jain 51d565f619 fix conflicts with main- (this PR is from upstream/main) 2026-02-07 03:10:53 +05:30
yuneng-jiang 0cb79ace97 Fixing tests 2026-02-05 21:44:00 -08:00
yuneng-jiang 3504f05a5c Adding tests + update pyproject 2026-02-05 21:00:05 -08:00
Sameer Kankute 1f4222e6b2 Add support for 0 cost models 2026-02-02 13:29:37 +05:30
Ishaan Jaffer 2e6659d9cb test_embedding fix 2026-01-31 13:43:53 -08:00
Shin 586b041837 fix(test): update test_chat_completion to handle metadata in body 
The proxy now adds metadata to the request body during processing.
Updated test to compare fields individually and strip metadata from
body comparison.

Fixes litellm_proxy_unit_testing_part2 CI failure.
 2026-01-31 21:25:39 +00:00
Xianzong Xie f9eea06a37 Add tests for native_background_mode feature 
Added 8 new unit tests for the native_background_mode feature:
- test_polling_disabled_when_model_in_native_background_mode
- test_polling_disabled_for_native_background_mode_with_provider_list
- test_polling_enabled_when_model_not_in_native_background_mode
- test_polling_enabled_when_native_background_mode_is_none
- test_polling_enabled_when_native_background_mode_is_empty_list
- test_native_background_mode_exact_match_required
- test_native_background_mode_with_provider_prefix_in_request
- test_native_background_mode_with_router_lookup

Committed-By-Agent: cursor
 2026-01-27 16:48:22 -08:00
yuneng-jiang 45954155d7 Merge pull request #19799 from BerriAI/litellm_sso_email_casing 
[Fix] SSO Email Case Sensitivity
 2026-01-27 09:52:03 -08:00
yuneng-jiang 50612715a5 Merge pull request #19814 from BerriAI/litellm_team_member_add_fix 
[Fix] /team/member_add User Email and ID Verifications
 2026-01-27 09:49:01 -08:00
Sameer Kankute 0214cb04cd Merge branch 'main' into litellm_oss_staging_01_26_2026 2026-01-27 17:00:58 +05:30
yuneng-jiang 7307992cea fixing tests 2026-01-26 20:17:53 -08:00
yuneng-jiang ef7261d0eb Fixing tests 2026-01-26 20:07:30 -08:00
Harshit Jain b920be3ee7 fix: server rooth path (#19790) 2026-01-26 09:48:06 -08:00
Harshit Jain 79603b9c3a fix: optimize logo fetching and resolve mcp import blockers (#19719) 2026-01-25 23:08:16 -08:00
Ishaan Jaffer 0d75e506c5 fix mock_handler 2026-01-20 18:01:22 -08:00
Ishaan Jaffer b9de10bd27 test token ctr 2026-01-20 17:53:53 -08:00
Ishaan Jaff ddebdd47bc [Feat] Add Support for Claude Code Max/OAuth 2 on LiteLLM AI Gateway (#19453) 
* fix count_tokens_with_anthropic_api

* remove outdated file

* fix ANTHROPIC_TOKEN_COUNTING_BETA_VERSION

* refactor: get_token_counter

* init test suite for token counter

* init token counters

* fix: fix pyrightI

* fix Code QA issues

* feat: add OAUTH handling ant

* feat: Oauth handling Ant

* test anthopic common utils

* fix code QA

* docs
 2026-01-20 17:21:17 -08:00
yuneng-jiang 6e8dd06d18 fixing test 2026-01-16 19:06:23 -08:00
yuneng-jiang 809b4cb310 Revert "Revert "[Feature] Deleted Keys and Deleted Teams Table"" 2026-01-16 14:25:23 -08:00
YutaSaito 034e3a6d44 Revert "[Feature] Deleted Keys and Deleted Teams Table" 2026-01-17 06:46:41 +09:00
yuneng-jiang bd4a893daf fixing tests 2026-01-16 12:42:35 -08:00
Yuta Saito 3a55f68ea4 test: handle wildcard routes in route validation test 2026-01-16 12:53:27 +09:00
YutaSaito 960bdbbd8e Revert "[Feat] Add support for 0 cost models" 2026-01-16 09:33:38 +09:00
Sameer Kankute 11cc8dc9d2 Merge pull request #19037 from Jetemple/json-log-fix 
fix: enable JSON logging via configuration and add regression test
 2026-01-14 13:51:36 +05:30
Yuta Saito c5ced033c9 fix: anthropic during call guardrail error 2026-01-14 13:37:01 +09:00
Raghav Jhavar 272a48d880 [bug fix] do not fallback to token counter if disable_token_counter is enabled (#19041) 
* do not fallback to token counter if disable_token_counter is enabled, and return errors instead

* add exceptions and exception utils to map the same as /v1/chat/completions

* use safe_json_loads
 2026-01-13 16:53:38 -08:00
Jack Temple 9e08c2207f fix: enable JSON logging via configuration and add regression test 2026-01-13 09:38:19 -07:00
Sameer Kankute 762a3ef090 Add support for 0 cost models 2026-01-13 16:39:57 +05:30
Ishaan Jaffer f1d516f0ae test fixes 2026-01-10 15:15:41 -08:00
Ishaan Jaffer 35c636ba97 test_health_check_not_called_when_disabled 2026-01-10 13:55:11 -08:00
Ishaan Jaffer bab43ee9dd fix list keys 2026-01-07 14:34:37 +05:30
Alexsander Hamir 5534038e93 Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
yuneng-jiang 81dc70673a Merge remote-tracking branch 'origin' into litellm_ui_unset_values 2025-12-22 11:44:41 -08:00
Ishaan Jaffer 6112160a16 Revert "[Fix] Security - Remove example API keys with high entropy (#18255)" 
This reverts commit 24edbccf5c.
 2025-12-20 20:48:11 +05:30
Ishaan Jaffer eec05c036a test_create_skill_sdk 2025-12-20 15:08:31 +05:30
yuneng-jiang ffcac2eebc Allow deleting key expiry 2025-12-19 18:04:04 -08:00
Sameer Kankute f05e600969 Merge pull request #18236 from BerriAI/litellm_add_background_responses_cost_tracking 
Add cost tracking for responses api in background mode
 2025-12-20 00:14:09 +05:30
Alexsander Hamir 24edbccf5c [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
Ishaan Jaff 09b5ee9428 [Feat] Unified Skills API - works across Anthropic, Vertex, Azure, Bedrock (#18232) 
* init LiteLLM_SkillsTable

* init LiteLLMSkillsTransformationHandler

* init LiteLLMSkillsTransformationHandler

* init skills

* init SkillsInjectionHook

* init litellm skills handler

* _fetch_skill_from_db

* LiteLLMSkillsTransformationHandler

* add _prisma_skill_to_litellm

* use SkillPromptInjectionHandler

* refactor skills handler

* add slack GIF

* test_skill_code_execution_via_deployment_hook

* init readme

* test_get_skill_sdk

* test skills in DB

* fix code qa

* init with containers param

* SkillsInjectionHook

* fix type

* convert_skill_to_anthropic_tool, get_litellm_code_execution_tool_anthropic

* fix messages skills handler

* test_slack_gif_skill_creates_gif

* init SkillsInjectionHook

* clean async_pre_call_hook

* fixes

* working unified skills API! yeet

* fix mypy linting

* LiteLLM_SkillsTable

* llm-sandbox==0.3.31

* fix
 2025-12-19 18:55:59 +05:30
Sameer Kankute 7d0f41f437 Add cost tracking for responses api in background mode 2025-12-19 13:35:48 +05:30
Jorge Yero Salazar 48a3a741e5 Allow base_model for non Azure providers in proxy (#18038) 
* Allow base_model for non Azure providers in proxy

* Add tests
 2025-12-17 02:24:12 +04:00
Alexsander Hamir 32fdb9e60e fix: Add headers to Request scope in JWT tests to fix KeyError (#17927) 
- Add 'headers': [] to all Request(scope={'type': 'http'}) instances in test_jwt.py
- Fixes KeyError: 'headers' when accessing request.headers in user_api_key_auth
- All 7 previously failing tests now pass:
  - test_allow_access_by_email (2 variants)
  - test_allowed_routes_admin (4 variants)
  - test_team_token_output (2 variants)

The Starlette Request object requires 'headers' key in scope dictionary
when accessing request.headers property.
 2025-12-13 10:36:13 -08:00