Kazuki Matsumaru and GitHub
26bdf7b7a8
Remove redundant comments about setting litellm.callbacks ( #18711 )
...
- Removed duplicate comment in test_router_endpoints.py
- Removed duplicate comment in logging.md
- Kept clearer comment: 'Set litellm.callbacks = [proxy_handler_instance] on the proxy'
2026-01-06 23:58:01 +05:30
Ishaan Jaff and GitHub
142567e143
[Fix] Containers API - Allow using LIST, Create Containers using custom-llm-provider ( #17740 )
...
* test_router_acreate_container_without_model
* _init_containers_api_endpoints
* test_init_containers_api_endpoints
2025-12-09 17:00:35 -08:00
4e195d639e
[Feat] New API - Claude Skills API (Anthropic) ( #17042 )
...
* init readme
* init BaseSkillsAPIConfig
* init types for Skills APIs
* add feat: add create, list, retrieve skills
* add base skills config
* add BaseSkillsAPIConfig
* add get_provider_skills_api_config
* init skills
* add ANTHROPIC_SKILLS_API_BETA_VERSION
* init skills APIs
* working list, get skills
* working e2e skills API anthropic API
* add _prepare_skill_multipart_request
* add skills routes to llm api routes
* router _initialize_skills_endpoints
* add fix skills endpoints
* add convert_upload_files_to_file_data
* fix routing skills endpoints
* fix route llm request
* Potential fix for code scanning alert no. 3806: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 3809: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix ruff checks
* test_initialize_skills_endpoints
* fix claude skills mypy linting errors
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-24 15:01:40 -08:00
yuneng-jiang and GitHub
4b25398afe
[Infra] CI/CD Fixes ( #16937 )
...
* Attempt CI/CD Fix
* Adding test for coverage
* Adding max depth to copilot and vertex
* Fixing mypy lint and docker database
* Fixing UI build issues
* Update playwright test
2025-11-21 13:58:19 -08:00
Ishaan Jaffer
7ec2d103f6
test router endpoitns
2025-11-06 16:17:14 -08:00
Sameer Kankute and GitHub
eed3ad0bdb
Fix: Moderations endpoint now respects api_base configuration parameter ( #16087 )
...
* Update moderation to use api base
* Update moderation to use api base
* Fix mypy error
2025-10-30 11:01:26 -07:00
Ishaan Jaff and GitHub
b9132968b2
[Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS ( #13905 )
...
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895 )
* fix: litellm.configured_cold_storage_logger
* fix Session Management - Non-OpenAI Models docs
* ruff fix
* test fix
* create LoggingWorker
* add GLOBAL_LOGGING_WORKER for async task handling
* fix logging tests
* add conftest
* fix conftest
* test fix location of encode bedrock runtime modelid arn
* fix conftest.py
* tuning LoggingWorker
* conftest.py
* fix conftest batches/
* test_async_chat_azure
* event_loop
* test_bedrock_streaming_passthrough_test2
* fix GLOBAL_LOGGING_WORKER
* logging worker
* add flush for global logging worker
* Revert "fix GLOBAL_LOGGING_WORKER"
This reverts commit d254f508f48935652f054777652938ad71976cce.
* fix conftest clear_queue
* fix conftest clear_queue
* setup_and_teardown for llm translation
* docs AWS_REGION
* test_async_chat_azure
* change test DIR
* run ci/cd again
* use 1 job for litellm_router_unit_testing
* fix space
* fix litellm_router_unit_testing
* test_aaarouter_dynamic_cooldown_message_retry_time
* litellm_router_unit_testing
* conftest.py clearing qu
* fixes litellm_router_unit_testing
* fixes clear_queue
* fix router_unit_tests
* remove conftest
* add back conftest for router
* fix event loop test
* test fix
* fixes for LoggingWorker
* ruff fix
2025-08-23 13:13:23 -07:00
Ishaan Jaff
e69a895884
test_apply_default_settings
2025-08-20 08:53:38 -07:00
Ishaan Jaff and GitHub
5802a5bbe3
[Feat] LLM API Endpoint - Expose OpenAI Compatible /vector_stores/{vector_store_id}/search endpoint ( #12749 )
...
* fix _pass_through_endpoint_without_required_model
* add get_litellm_managed_vector_store_from_registry
* undo router change
* fix for using router + vector search methods
* add simple helper for _update_request_data_with_litellm_managed_vector_store_registry
* add vector_stores routes
* test_router_avector_store_search_passes_correct_args
* [Feat] UI - Allow clicking into Vector Stores (#12741 )
* Add View Vector Store
* add /info for vector store
* fix updated_at
* allow easily testing the KB on litellm
* fix
* rename test
* test_init_vector_store_api_endpoints
* test_update_request_data_with_litellm_managed_vector_store_registry
2025-07-18 18:18:53 -07:00
Ishaan Jaff
8ed3557ce7
test_init_responses_api_endpoints
2025-04-24 21:43:59 -07:00
Ishaan Jaff
e0252a9b49
test_initialize_router_endpoints
2025-03-12 19:27:32 -07:00
Ishaan Jaff
ac928ebb16
test_initialize_router_endpoints
2025-03-12 19:24:25 -07:00
Ishaan Jaff
b055aae5ca
test_generic_api_call_with_fallbacks_basic
2025-03-12 19:16:05 -07:00
Ishaan Jaff and GitHub
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec ( #9013 )
...
* anthropic_messages_handler v0
* fix /messages
* working messages with router methods
* test_anthropic_messages_handler_litellm_router_non_streaming
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* AnthropicMessagesConfig
* _handle_anthropic_messages_response_logging
* working with /v1/messages endpoint
* working /v1/messages endpoint
* refactor to use router factory function
* use aanthropic_messages
* use BaseConfig for Anthropic /v1/messages
* track api key, team on /v1/messages endpoint
* fix get_logging_payload
* BaseAnthropicMessagesTest
* align test config
* test_anthropic_messages_with_thinking
* test_anthropic_streaming_with_thinking
* fix - display anthropic url for debugging
* test_bad_request_error_handling
* test_anthropic_messages_router_streaming_with_bad_request
* fix ProxyException
* test_bad_request_error_handling_streaming
* use provider_specific_header
* test_anthropic_messages_with_extra_headers
* test_anthropic_messages_to_wildcard_model
* fix gcs pub sub test
* standard_logging_payload
* fix unit testing for anthopic /v1/messages support
* fix pass through anthropic messages api
* delete dead code
* fix anthropic pass through response
* revert change to spend tracking utils
* fix get_litellm_metadata_from_kwargs
* fix spend logs payload json
* proxy_pass_through_endpoint_tests
* TestAnthropicPassthroughBasic
* fix pass through tests
* test_async_vertex_proxy_route_api_key_auth
* _handle_anthropic_messages_response_logging
* vertex_credentials
* test_set_default_vertex_config
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* test_ageneric_api_call_with_fallbacks_basic
* test__aadapter_completion
2025-03-06 00:43:08 -08:00
Ishaan Jaff and GitHub
5dcb87a88b
(bug fix router.py) - safely handle choices=[] on llm responses ( #8342 )
...
* test fix test_router_with_empty_choices
* fix _should_raise_content_policy_error
2025-02-06 18:22:08 -08:00
Ishaan Jaff and GitHub
a003af6c04
(fix) litellm.amoderation - support using model=openai/omni-moderation-latest, model=omni-moderation-latest, model=None ( #7475 )
...
* test_moderation_endpoint
* fix litellm.amoderation
2024-12-30 09:42:51 -08:00
Ishaan Jaff and GitHub
5e8c64f128
(Bug fix) missing model_group field in logs for aspeech call types ( #7392 )
...
* fix use _update_kwargs_before_fallbacks
* test assert standard_logging_object includes model_group
* test_datadog_non_serializable_messages
* update test
2024-12-27 17:00:39 -08:00
Ishaan Jaff and GitHub
bfb6891eb7
rename llms/OpenAI/ -> llms/openai/ ( #7154 )
...
* rename OpenAI -> openai
* fix file rename
* fix rename changes
* fix organization of openai/transcription
* fix import OA fine tuning API
* fix openai ft handler
* fix handler import
2024-12-10 20:14:07 -08:00
Krish Dholakia and GitHub
21156ff5d0
LiteLLM Minor Fixes & Improvements (11/27/2024) ( #6943 )
...
* fix(http_parsing_utils.py): remove `ast.literal_eval()` from http utils
Security fix - https://huntr.com/bounties/96a32812-213c-4819-ba4e-36143d35e95b?token=bf414bbd77f8b346556e
64ab2dd9301ea44339910877ea50401c76f977e36cdd78272f5fb4ca852a88a7e832828aae1192df98680544ee24aa98f3cf6980d8
bab641a66b7ccbc02c0e7d4ddba2db4dbe7318889dc0098d8db2d639f345f574159814627bb084563bad472e2f990f825bff0878a9
e281e72c88b4bc5884d637d186c0d67c9987c57c3f0caf395aff07b89ad2b7220d1dd7d1b427fd2260b5f01090efce5250f8b56ea2
c0ec19916c24b23825d85ce119911275944c840a1340d69e23ca6a462da610
* fix(converse/transformation.py): support bedrock apac cross region inference
Fixes https://github.com/BerriAI/litellm/issues/6905
* fix(user_api_key_auth.py): add auth check for websocket endpoint
Fixes https://github.com/BerriAI/litellm/issues/6926
* fix(user_api_key_auth.py): use `model` from query param
* fix: fix linting error
* test: run flaky tests first
2024-11-28 00:32:46 +05:30
Krish Dholakia and GitHub
27e18358ab
fix(pattern_match_deployments.py): default to user input if unable to… ( #6632 )
...
* fix(pattern_match_deployments.py): default to user input if unable to map based on wildcards
* test: fix test
* test: reset test name
* test: update conftest to reload proxy server module between tests
* ci(config.yml): move langfuse out of local_testing
reduce ci/cd time
* ci(config.yml): cleanup langfuse ci/cd tests
* fix: update test to not use global proxy_server app module
* ci: move caching to a separate test pipeline
speed up ci pipeline
* test: update conftest to check if proxy_server attr exists before reloading
* build(conftest.py): don't block on inability to reload proxy_server
* ci(config.yml): update caching unit test filter to work on 'cache' keyword as well
* fix(encrypt_decrypt_utils.py): use function to get salt key
* test: mark flaky test
* test: handle anthropic overloaded errors
* refactor: create separate ci/cd pipeline for proxy unit tests
make ci/cd faster
* ci(config.yml): add litellm_proxy_unit_testing to build_and_test jobs
* ci(config.yml): generate prisma binaries for proxy unit tests
* test: readd vertex_key.json
* ci(config.yml): remove `-s` from proxy_unit_test cmd
speed up test
* ci: remove any 'debug' logging flag
speed up ci pipeline
* test: fix test
* test(test_braintrust.py): rerun
* test: add delay for braintrust test
2024-11-08 00:55:57 +05:30
1eb435e50a
test(router_code_coverage.py): check if all router functions are dire… ( #6186 )
...
* test(router_code_coverage.py): check if all router functions are directly tested
prevent regressions
* docs(configs.md): document all environment variables (#6185 )
* docs: make it easier to find anthropic/openai prompt caching doc
* aded codecov yml (#6207 )
* fix codecov.yaml
* run ci/cd again
* (refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208 )
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* fix test_embedding_caching_azure_individual_items_reordered
* (feat) prometheus have well defined latency buckets (#6211 )
* fix prometheus have well defined latency buckets
* use a well define latency bucket
* use types file for prometheus logging
* add test for LATENCY_BUCKETS
* fix prom testing
* fix config.yml
* (refactor caching) use LLMCachingHandler for caching streaming responses (#6210 )
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* refactor async set stream cache
* fix linting
* bump (#6187 )
* update code cov yaml
* fix config.yml
* add caching component to code cov
* fix config.yml ci/cd
* add coverage for proxy auth
* (refactor caching) use common `_retrieve_from_cache` helper (#6212 )
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* refactor async set stream cache
* fix linting
* refactor - use _retrieve_from_cache
* refactor use _convert_cached_result_to_model_response
* fix linting errors
* bump: version 1.49.2 → 1.49.3
* fix code cov components
* test(test_router_helpers.py): add router component unit tests
* test: add additional router tests
* test: add more router testing
* test: add more router testing + more mock functions
* ci(router_code_coverage.py): fix check
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com >
Co-authored-by: yujonglee <yujonglee.dev@gmail.com >
2024-10-14 22:44:00 -07:00
