litellm/tests at 8ebefe99b378f8a0b44b076964c7c79b82b25503 - litellm - Gitea: Git with a cup of tea

tiennm99/litellm

mirror of https://github.com/tiennm99/litellm.git synced 2026-08-02 06:22:48 +00:00

Files

T

History

mateo-berri 8ebefe99b3 test(translation): two-sided google differential gates over the characterization corpus

vertex_gemini / gemini / vertex_anthropic requests, responses, and streams:
snapshot == v1-at-HEAD == v2 (canonical JSON), plus a quirk corpus pinning
the 3-way structured-output fork, gemini-3 defaults and id forwarding,
multi-system parts, top_k passthrough on both routes, and the fail-closed
shapes (large cache markers, AI-Studio https media, xhigh effort, blocked
responses, mid-stream error objects).

2026-06-12 01:24:44 +00:00

..

Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728 )" (#29326 )

2026-05-30 11:26:24 -07:00

fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110 )

2026-05-18 09:15:39 -07:00

basic_proxy_startup_tests

…

test: stabilize batch VCR coverage and stop live upload/network leaks (#29477 )

2026-06-02 16:11:52 -07:00

…

code_coverage_tests

fix(translation): wire v2 tests into CI and clear the code-quality gate

2026-06-11 16:13:40 +00:00

documentation_tests

…

feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687 )

2026-06-06 17:50:29 -07:00

guardrails_tests

Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728 )" (#29326 )

2026-05-30 11:26:24 -07:00

image_gen_tests

feat(fal_ai): add Nano Banana / Gemini 2.5 Flash Image generation support (#29798 )

2026-06-06 11:16:44 -07:00

CI: copy of #25177 (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) (#28223 )

2026-05-23 12:15:41 -07:00

Title: Fix managed batch cancel credential resolution (#29734 )

2026-06-06 12:35:18 -07:00

litellm_core_utils

…

litellm_utils_tests

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

litellm-proxy-extras

…

llm_responses_api_testing

test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview (#29433 )

2026-06-01 09:54:30 -07:00

llm_translation

Litellm oss 090626 (#30021 )

2026-06-10 10:34:07 -07:00

…

feat(cli): per-agent lite claude / codex / opencode commands that wrap coding agents through the proxy (#29850 )

2026-06-10 13:52:26 -07:00

logging_callback_tests

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

[internal copy of #28008 ] Support MCP OAuth passthrough and issuer-scoped JWT auth (#28356 )

2026-06-02 12:22:04 -07:00

multi_instance_e2e_tests

…

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

old_proxy_tests/tests

…

openai_endpoints_tests

chore(ci): modernize model references in tests and configs (#27856 )

2026-05-15 15:44:28 -07:00

feat(prometheus): add user_email and user_alias to user budget metrics (#28155 )

2026-05-18 16:28:14 -07:00

pass_through_tests

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

pass_through_unit_tests

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

proxy_admin_ui_tests

fix(guardrails): persist disable_global_guardrails on keys (#29233 )

2026-05-28 21:19:04 -07:00

fix(team): reserve team budget raises for proxy admins on /team/update (#30030 )

2026-06-09 09:19:15 -07:00

proxy_e2e_anthropic_messages_tests

Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847 )

2026-06-06 14:33:42 -07:00

proxy_migration_tests

test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700 )

2026-06-04 14:56:02 -07:00

proxy_security_tests

test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700 )

2026-06-04 14:56:02 -07:00

proxy_unit_tests

Litellm jwt mapping virtualkeys (#28510 )

2026-06-04 19:00:36 -07:00

router_unit_tests

Litellm oss 090626 (#30021 )

2026-06-10 10:34:07 -07:00

…

fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110 )

2026-05-18 09:15:39 -07:00

spend_tracking_tests

chore(ci): modernize model references in tests and configs (#27856 )

2026-05-15 15:44:28 -07:00

store_model_in_db_tests

…

test(translation): two-sided google differential gates over the characterization corpus

2026-06-12 01:24:44 +00:00

unified_google_tests

test(google): add google-genai SDK proxy integration tests (#29781 )

2026-06-05 21:05:32 +00:00

vector_store_tests

Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728 )" (#29326 )

2026-05-30 11:26:24 -07:00

ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597 )

2026-06-03 11:28:08 -07:00

__init__.py

…

_flush_vcr_cache.py

tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL

2026-05-01 12:32:59 -07:00

_live_test_helpers.py

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

_openai_record_replay_proxy.py

Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847 )

2026-06-06 14:33:42 -07:00

_vcr_conftest_common.py

test(vcr): close out the remaining VCR live-call leaks (#29603 )

2026-06-03 13:46:43 -07:00

_vcr_redis_persister.py

test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784 )

2026-06-05 10:22:41 -07:00

eval_swe_bench.py

…

gettysburg.wav

…

large_text.py

…

openai_batch_completions.jsonl

…

README.MD

…

test_budget_management.py

…

test_callbacks_on_proxy.py

test(callbacks): harden flaky proxy callback-leak detector (#28195 )

2026-05-18 16:39:02 -07:00

test_config.py

…

test_debug_warning.py

…

test_default_encoding_non_root.py

…

test_end_users.py

chore(ci): modernize model references in tests and configs (#27856 )

2026-05-15 15:44:28 -07:00

test_entrypoint.py

…

test_fallbacks.py

…

test_gpt5_azure_temperature_support.py

…

test_health.py

fix(tests): swap dall-e to gpt-image-1 after openai deprecation

2026-05-12 16:55:18 -07:00

test_keys.py

Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847 )

2026-06-06 14:33:42 -07:00

test_litellm_proxy_responses_config.py

chore(ci): modernize model references in tests and configs (#27856 )

2026-05-15 15:44:28 -07:00

test_logging.conf

…

test_models.py

…

test_new_vector_store_endpoints.py

…

test_openai_endpoints.py

Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847 )

2026-06-06 14:33:42 -07:00

test_organizations.py

…

test_otel_thread_leak.py

…

test_passthrough_endpoints.py

…

test_presidio_latency.py

…

test_proxy_server_non_root.py

…

test_ratelimit.py

chore(ci): modernize model references in tests and configs (#27856 )

2026-05-15 15:44:28 -07:00

test_resource_cleanup.py

…

test_service_logger_otel.py

…

test_spend_logs.py

Litellm oss staging 04 21 2026 2 (#26569 )

2026-05-20 21:25:19 -07:00

test_team_logging.py

…

test_team_members.py

Litellm oss staging 04 21 2026 2 (#26569 )

2026-05-20 21:25:19 -07:00

test_team.py

…

test_users.py

Fix: tag budget reset must drop stale management-cache entry (#27568 )

2026-05-10 00:18:55 +00:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.