mirror of https://github.com/tiennm99/litellm.git synced 2026-08-02 08:21:53 +00:00

Files

T

mateo-berri e1d32fe4d1 test(translation): two-sided bedrock differential gates over the characterization corpus

The reference corpus (cases, recorded fixtures, canonical-JSON snapshots for
both bedrock chat routes) is copied verbatim from the translation
characterization branch; each request row asserts snapshot == v1-at-HEAD
(drift guard) == v2 (byte-equal). Responses and streams run full cycles
against v1's transform_response and the real stream decoders inside
CustomStreamWrapper at the parsed-event seam, with the corpus's
fastuuid/time freeze. A quirk corpus pins bedrock-only behaviors against v1
in-process: claude-4.5 cache ttl and parallel tool config, bedrock tool-name
normalization, thinking budget clamp, json_schema-with-effort dropping the
forced choice, tool_choice none gates, and the explained fallbacks
(pdf/file parts, non-Claude models, invoke response_format x reasoning).
Seam tests drive litellm.completion() through respx-intercepted SigV4 calls
for both routes. DIFFERENTIAL_REPORT.md gains the bedrock sections:
126 rows, 0 divergent.

2026-06-11 22:08:20 +00:00

a2a_protocol

Fix : a2a bugs 030626 (#29566 )

2026-06-03 11:14:15 -07:00

anthropic_interface/exceptions

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

caching

fix(caching): restore stored prompt_tokens on embedding cache hits instead of recomputing (#30046 )

2026-06-10 15:49:20 +05:30

completion_extras

fix(responses-bridge): map system-only chat request to system input item (#29817 )

2026-06-06 16:11:54 -07:00

containers

fix(containers): record ownership for service-account keys + fix Prisma Json serialization (#28990 )

2026-05-27 21:00:07 -07:00

enterprise

feat(enterprise): add RESEND_FROM_EMAIL for self-hosted Resend sends (#28830 )

2026-05-30 10:09:05 -07:00

expected_fine_tuning_api

refactor: refactor testing

2026-03-28 18:39:32 -07:00

expected_responses_api_request

[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058 )

2026-02-12 10:00:30 -08:00

experimental_mcp_client

feat(mcp): per-server env vars with global + per-user scopes (#28917 )

2026-06-05 20:15:11 -07:00

google_genai

fix(gemini): normalize response_schema on native generateContent (#27775 )

2026-05-12 23:26:34 -07:00

images

fix(image_edit): read vertex_project/location from litellm_params in Imagen get_complete_url

2026-04-21 15:03:40 +05:30

integrations

fix(callbacks): forward callback_settings to callback initializers and guard consumers against non-dict values (#30161 )

2026-06-10 15:22:00 -07:00

interactions

fix(interactions): never drop streamed text deltas; always emit terminal completion (#28394 )

2026-05-20 16:41:40 -07:00

litellm_core_utils

[internal copy of #30137 ] perf(realtime): eliminate redundant per-frame JSON work on OpenAI realtime relay (#30142 )

2026-06-11 09:56:35 +05:30

llms

feat(bedrock): aws_bedrock_project_id for bedrock-mantle project / workspace association (#30163 )

2026-06-11 10:01:08 +05:30

models

feat(litellm): add models and repository layers (#29686 )

2026-06-06 20:59:33 -07:00

ocr

chore: reject bare str at file-input sinks to prevent local-file read (#27762 )

2026-05-12 16:40:07 -07:00

passthrough

refactor: trim explanatory comments from streaming-flush fix

2026-04-30 02:39:28 +00:00

proxy

feat(bedrock): aws_bedrock_project_id for bedrock-mantle project / workspace association (#30163 )

2026-06-11 10:01:08 +05:30

repositories

feat(litellm): add models and repository layers (#29686 )

2026-06-06 20:59:33 -07:00

responses

Litellm oss staging 080626 (#29932 )

2026-06-08 13:49:52 -07:00

router_strategy

fix(router): enforce deployment budgets for dynamically added models (#29273 )

2026-05-29 19:43:14 -07:00

router_utils

Litellm oss 090626 (#30021 )

2026-06-10 10:34:07 -07:00

secret_managers

Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior.

2026-05-04 18:17:31 +00:00

test_router

fix: use atomic increment-first pattern for model RPM rate limiting

2026-02-24 09:55:07 -03:00

translation

test(translation): two-sided bedrock differential gates over the characterization corpus

2026-06-11 22:08:20 +00:00

types

Litellm oss staging 080626 (#29932 )

2026-06-08 13:49:52 -07:00

vector_stores

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

__init__.py

…

conftest.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

log.txt

…

readme.md

…

test__types.py

Litellm OSS Staging 010626 (#29422 )

2026-06-01 21:42:51 -07:00

test_a2a_registry_lookup.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_acompletion_session_reuse_e2e.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_add_deployment_no_master_key.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_aembedding_session_reuse_e2e.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_anthropic_beta_headers_filtering.py

Litellm oss 090626 (#30021 )

2026-06-10 10:34:07 -07:00

test_anthropic_skills_transformation.py

fix: encode upstream URL path identifiers

2026-04-29 22:02:39 -07:00

test_azure_video_router.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_bedrock_anthropic_1hr_cache_pricing.py

Litellm oss staging 030626 (#29578 )

2026-06-03 11:01:51 -07:00

test_bedrock_usgov_haiku_1hr_cache.py

Litellm OSS Staging 010626 (#29422 )

2026-06-01 21:42:51 -07:00

test_bedrock_usgov_pricing.py

Litellm OSS Staging 010626 (#29422 )

2026-06-01 21:42:51 -07:00

test_chat_ui_responses_session.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_check_licenses.py

fix(check_licenses): read PEP 639 license-expression metadata (#28529 )

2026-05-22 11:22:38 -07:00

test_claude_fable_5_config.py

Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064 )

2026-06-10 08:50:15 +05:30

test_claude_haiku_4_5_config.py

feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238 )

2026-05-28 18:50:33 -07:00

test_claude_opus_4_6_config.py

feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238 )

2026-05-28 18:50:33 -07:00

test_claude_opus_4_8_config.py

fix(anthropic): route Claude Opus 4.8 through adaptive thinking (#29702 )

2026-06-05 16:19:01 +05:30

test_claude_sonnet_4_6_config.py

feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238 )

2026-05-28 18:50:33 -07:00

test_completion_timeout_resolution.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_compression.py

Prompt Compression - add it to the proxy (#25729 )

2026-04-20 15:08:00 -07:00

test_constants.py

Fix test

2026-03-27 21:21:43 +05:30

test_container_router.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_cost_calculation_log_level.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_cost_calculator.py

[internal copy of #27491 ] fix(realtime): Fix Realtime Audio Token Cost Tracking (#29722 )

2026-06-05 18:53:17 +05:30

test_count_tokens_public_api.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_dashscope_image_generation.py

fix(dashscope): fail fast on image generation API errors

2026-04-23 18:41:01 +05:30

test_deepseek_model_metadata.py

Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle"

2026-04-25 17:10:41 -03:00

test_dockerfile_non_root.py

fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268 )

2026-04-22 18:00:04 -07:00

test_eager_tiktoken_load.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_exception_exports.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_exception_header_preservation.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_exception_mapping_request_attribute.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_filter_out_litellm_params.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_get_blog_posts.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_gpt_image_cost_calculator.py

Add gpt-image-2 support (#26644 ) (#26705 )

2026-04-28 20:10:42 -07:00

test_groq_streaming_encoding.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_guardrail_exception_status_codes.py

Litellm oss staging (#28161 )

2026-05-18 16:27:44 -07:00

test_lazy_imports.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_litellm_params_reserved_keys.py

fix(snowflake): transform tool_choice string to object format (#23268 )

2026-03-11 01:41:24 +05:30

test_logging.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_lowest_latency_zero_tokens.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_main_module_header.py

Add main module header comment

2026-05-06 00:26:17 +00:00

test_main.py

Litellm oss staging 040626 (#29671 )

2026-06-04 11:07:20 -07:00

test_model_cost_aliases.py

[Feat] - Ishaan main merge branch (#23596 )

2026-03-14 09:40:00 -07:00

test_model_param_helper.py

style: black format test_model_param_helper.py

2026-04-15 18:18:52 -07:00

test_model_response_normalization.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_nested_drop_params.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_openai_embedding_encoding_format_default.py

fix(embeddings): allow omitting encoding_format via env sentinel none

2026-05-01 23:03:13 +05:30

test_project_alias_tracking.py

feat(proxy): add project_alias tracking through callback metadata pipeline

2026-03-23 10:44:17 -07:00

test_project_tags_pydantic.py

fix: req changes

2026-02-27 13:33:34 +05:30

test_rate_limit_error_unification.py

feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687 )

2026-06-06 17:50:29 -07:00

test_redact_string_in_error_paths.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_redis.py

Litellm oss staging (#28161 )

2026-05-18 16:27:44 -07:00

test_register_model_custom_pricing.py

fix(register_model): preserve built-in cache pricing when registering custom overrides under unmapped keys (#30044 )

2026-06-10 12:11:03 -07:00

test_responses_api_bridge_non_stream.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_responses_id_security.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_retrieve_batch_bedrock_dispatch.py

feat(batch-job): bedrock batch model invocation job retrieval (#26834 )

2026-05-11 13:22:26 -07:00

test_router_google_genai.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_router_model_cost_isolation.py

fix(register_model): preserve built-in cache pricing when registering custom overrides under unmapped keys (#30044 )

2026-06-10 12:11:03 -07:00

test_router_order_fallback.py

fix(router): enable order fallback for wildcard model groups

2026-04-15 17:48:25 +05:30

test_router_per_deployment_num_retries.py

Bugfix/19481 num retries env var type (#19507 )

2026-01-22 19:39:58 -08:00

test_router_redis_init.py

fix: handle deprecated 'redis_db' arg to prevent crash (#19808 )

2026-02-02 18:18:05 +05:30

test_router_retry_non_retryable_errors.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_router_silent_experiment.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_router_weighted_failover.py

feat: add weighted-routing failover (#27980 )

2026-05-15 17:28:54 +00:00

test_router.py

Litellm oss 090626 (#30021 )

2026-06-10 10:34:07 -07:00

test_secret_redaction.py

Litellm oss staging (#29492 )

2026-06-02 08:48:10 -07:00

test_service_logger.py

fix: missing span for guardrail passthrough (#29552 )

2026-06-03 01:25:15 +00:00

test_setup_wizard.py

test: test

2026-03-28 19:17:38 -07:00

test_shared_session_integration.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_ssl_verify_unit.py

Litellm OSS Staging (#29161 )

2026-06-01 21:22:35 -07:00

test_stream_chunk_builder_annotations.py

fix: merge annotations from all streaming chunks in stream_chunk_builder

2026-03-15 14:20:45 +05:30

test_streaming_connection_cleanup.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_system_message_format_bug.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

test_thinking_enabled.py

Litellm oss staging 250526 (#28770 )

2026-05-26 11:57:39 -07:00

test_utils_module_docstring.py

Add utils module docstring

2026-05-06 00:42:49 +00:00

test_utils.py

feat(bedrock): aws_bedrock_project_id for bedrock-mantle project / workspace association (#30163 )

2026-06-11 10:01:08 +05:30

test_uuid_helper.py

…

test_vcr_safe_body_matcher.py

test: stabilize batch VCR coverage and stop live upload/network leaks (#29477 )

2026-06-02 16:11:52 -07:00

test_video_generation.py

fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (#29098 )

2026-05-28 11:45:41 -07:00

test_xai_grok_4_3_model_metadata.py

feat(xai): add grok-4.3 and grok-4.3-latest to model_prices_and_conte… (#27154 )

2026-05-07 09:06:56 -07:00

test_xai_responses_auto_routing.py

style: run black formatter on files from main merge

2026-04-17 13:02:59 -07:00

readme.md

Testing for `litellm/`

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

Increase test coverage of litellm/
Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
test_<filename>.py maps to litellm/<filename>.py

readme.md

Testing for litellm/

File name conventions

Testing for `litellm/`