Commit Graph
18910 Commits
Author SHA1 Message Date
Ishaan Jaff ef8812d150 ci/cd update vertex acct 2025-01-05 13:43:32 -08:00
Low Jian ShengandGitHub 9cfd7c790a Prevent istio injection for db migrations cron job (#7513)
* allow option to prevent istio injection

* make annotations more generic

* make annotations empty

* revert comment deletions
2025-01-04 22:45:19 -08:00
minpeterandGitHub f7931b659b FriendliAI: Documentation Updates (#7517)
* docs(friendliai.md): update FriendliAI documentation and model details

* docs(friendliai.md): remove unused imports for cleaner documentation

* feat: add support for parallel function calling, system messages, and response schema in model configuration
2025-01-04 22:44:24 -08:00
Fredy ÁlvarezandGitHub 2b0738ecc4 add fireworks_ai/accounts/fireworks/models/deepseek-v3 (#7567) 2025-01-04 22:43:09 -08:00
Ishaan JaffandGitHub 2bf20ebfdf latency fix proxy (#7563) 2025-01-04 20:18:32 -08:00
Krrish Dholakia 0d366c58a1 build(ui/): build new ui 2025-01-04 20:03:57 -08:00
Krrish Dholakia 62dd72a705 bump: version 1.56.10 → 1.57.0 2025-01-04 19:45:16 -08:00
Krish DholakiaandGitHub 34a9833b85 Support deleting keys by key_alias (#7552)
* feat(key_management_endpoints.py): allow deleting keys based on key alias

easier for proxy admin to delete known bad key

* fix(key_management_event_hooks.py): fix linting error

* docs(key_management_endpoints.py): document new key_aliases param

* fix(key_management_endpoints.py): return deleted keys to user

fixes return when passing key aliases
2025-01-04 19:41:48 -08:00
Krish DholakiaandGitHub ce97e7e054 fix(groq/chat/transformation.py): fix groq response_format transformation (#7565)
Fixes https://github.com/BerriAI/litellm/issues/4804
2025-01-04 19:39:04 -08:00
Ishaan JaffandGitHub d74fa39454 fix [PROXY] returned data from litellm_pre_call_util (#7558) 2025-01-04 18:47:36 -08:00
Krish DholakiaandGitHub f1540ceeab Create and view organizations + assign org admins on the Proxy UI (#7557)
* feat: initial commit for new 'organizations' tab on ui

* build(ui/): create generic card for rendering complete org data table

can be reused in teams as well

simplifies things

* build(ui/): display created orgs on ui

* build(ui/): support adding orgs via UI

* build(ui/): add org in selection dropdown

* build(organizations.tsx): allow assigning org admins

* build(ui/): show org members on ui

* build(ui/): cleanup + show actual models on org dropdown

* build(ui/): explain user roles within organization
2025-01-04 17:31:24 -08:00
Ishaan JaffandGitHub 46d9d29bff (Feat) Hashicorp Secret Manager - Allow storing virtual keys in secret manager (#7549)
* use a base abstract class

* async_write_secret for hcorp

* fix hcorp

* async_write_secret for hashicopr secret manager

* store virtual keys in hcorp

* add delete secret

* test_hashicorp_secret_manager_write_secret

* test_hashicorp_secret_manager_delete_secret

* docs Supported Secret Managers

* docs storing keys in hcorp

* docs hcorp

* docs secret managers

* test_key_generate_with_secret_manager_call

* fix unused imports
2025-01-04 11:35:59 -08:00
Ishaan JaffandGitHub 7f7222ce30 fix get_custom_logger_compatible_class (#7554) 2025-01-04 11:22:09 -08:00
Ishaan JaffandGitHub e804a1850e add cohere/command-r7b-12-2024 (#7553) v1.56.10 2025-01-04 11:21:04 -08:00
Ishaan JaffandGitHub d1b101b9d7 (Fix) - Slack Alerting , don't send duplicate spend report when used on multi instance settings (#7546)
* fix send_weekly_spend_report

* test_spend_report_cache
2025-01-04 10:54:35 -08:00
Ishaan JaffandGitHub 06eea38c69 (Fix) - Docker build error with pyproject.toml (#7550)
* fix pyproject

* fix homepage
2025-01-04 10:50:24 -08:00
Krrish Dholakia 3230c902c9 bump: version 1.56.9 → 1.56.10 2025-01-03 19:36:14 -08:00
Krish DholakiaandGitHub d43d83f9ef feat(router.py): support request prioritization for text completion c… (#7540)
* feat(router.py): support request prioritization for text completion calls

* fix(internal_user_endpoints.py): fix sql query to return all keys, including null team id keys on `/user/info`

Fixes https://github.com/BerriAI/litellm/issues/7485

* fix: fix linting errors

* fix: fix linting error

* test(test_router_helper_utils.py): add direct test for '_schedule_factory'

Fixes code qa test
2025-01-03 19:35:44 -08:00
Krish DholakiaandGitHub f770dd0c95 Support checking provider-specific /models endpoints for available models based on key (#7538)
* test(test_utils.py): initial test for valid models

Addresses https://github.com/BerriAI/litellm/issues/7525

* fix: test

* feat(fireworks_ai/transformation.py): support retrieving valid models from fireworks ai endpoint

* refactor(fireworks_ai/): support checking model info on `/v1/models` route

* docs(set_keys.md): update docs to clarify check llm provider api usage

* fix(watsonx/common_utils.py): support 'WATSONX_ZENAPIKEY' for iam auth

* fix(watsonx): read in watsonx token from env var

* fix: fix linting errors

* fix(utils.py): fix provider config check

* style: cleanup unused imports
2025-01-03 19:29:59 -08:00
Krish DholakiaandGitHub cac06a32b8 fix(aws_secret_manager_V2.py): Error reading secret from AWS Secrets Manager: (#7541) 2025-01-03 18:22:12 -08:00
Ishaan Jaff 8ad9526625 bump: version 1.56.8 → 1.56.9 v1.56.9 2025-01-03 17:49:23 -08:00
Ishaan JaffandGitHub 716efd5fad (fix proxy perf) use _read_request_body instead of ast.literal_eval to get better performance (#7545)
* fix ast literal eval

* run ci/cd again
2025-01-03 17:48:32 -08:00
Ishaan JaffandGitHub 1bb4941036 [Feature]: - allow print alert log to console (#7534)
* update send_to_webhook

* test_print_alerting_payload_warning

* add alerting_args spec

* test_alerting.py
2025-01-03 17:48:13 -08:00
Krish DholakiaandGitHub 6843f3a2bb Revert "fix: add missing parameters order, limit, before, and after in get_as…" (#7542)
This reverts commit 4b0505dffd.
2025-01-03 16:32:12 -08:00
Ishaan Jaff c8f9e2afab fix _make_common_async_call 2025-01-03 15:15:38 -08:00
Ishaan Jaff 23104d9a14 test_aiohttp_openai 2025-01-03 15:12:56 -08:00
Ishaan JaffandGitHub 02875d4ae8 (fix) aiohttp_openai/ route - get to 1K RPS on single instance (#7539)
* ClientSession

* re use client_session

* _init_client_session

* fix aiohttp
2025-01-03 15:12:17 -08:00
மனோஜ்குமார் பழனிச்சாமிandGitHub 4c6a1e6c9c Add missing prefix for deepseek (#7508)
Solves https://github.com/All-Hands-AI/OpenHands/issues/5978
2025-01-03 14:42:18 -08:00
Jean Carlo de SouzaandGitHub 4b0505dffd fix: add missing parameters order, limit, before, and after in get_assistants method for openai (#7537)
- Ensured that `before` and `after` parameters are only passed when provided to avoid AttributeError.
- Implemented safe access using default values for `before` and `after` to prevent missing attribute issues.
- Added consistent handling of `order` and `limit` to improve flexibility and robustness in API calls.
2025-01-03 14:41:54 -08:00
Krish DholakiaandGitHub 33f301ec86 Litellm dev 01 02 2025 p1 (#7516)
* fix(redact_messages.py): fix redact messages for non-model response input to be dictionary

fixes issue with otel logging when message redaction is enabled

* fix(proxy_server.py): fix langfuse key leak in exception string

* test: fix test

* test: fix test

* test: fix tests
2025-01-03 14:40:57 -08:00
Ishaan JaffandGitHub fb59f20979 (Feat) - Hashicorp secret manager, use TLS cert authentication (#7532)
* fix - don't print hcorp secrets in debug logs

* hcorp - tls auth fixes

* fix tls_ca_cert_path

* test_hashicorp_secret_manager_tls_cert_auth

* hcp secret docs
2025-01-03 14:23:53 -08:00
Ishaan Jaff d3a3e45e5b docs pass through routes 2025-01-03 12:55:23 -08:00
Krish DholakiaandGitHub f6698e871f Fix langfuse prompt management on proxy (#7535)
* fix(types/utils.py): support langfuse + humanloop routes on llm router

* fix(main.py): remove acompletion elif block

just await if coroutine returned
2025-01-03 12:42:37 -08:00
David ManouchehriandGitHub 24589c49af Bump anthropic.claude-3-5-haiku-20241022-v1:0 to new limits. (#7118) 2025-01-03 11:37:38 -08:00
Ishaan JaffandGitHub 6e31bcf5a7 fix - access metadata (#7523) 2025-01-03 10:02:10 -08:00
Ishaan JaffandGitHub 4899ed1b42 fix - don't print hcorp secrets in debug logs (#7529) 2025-01-03 10:01:54 -08:00
Ishaan JaffandGitHub 9fef0a6d16 (fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails (#7519)
* fix async_send_batch for gcs

* fix truncate GCS logger

* test_truncate_standard_logging_payload
2025-01-03 08:09:03 -08:00
Ishaan Jaff b8173c2e9f fix unused imports v1.56.8 2025-01-02 22:28:22 -08:00
Ishaan Jaff a9a2c7afd2 bump: version 1.56.7 → 1.56.8 2025-01-02 22:27:06 -08:00
Ishaan Jaff 087e8c0969 bump: version 1.56.6 → 1.56.7 2025-01-02 22:27:01 -08:00
Ishaan JaffandGitHub 8bd6c977e2 fix - use tp executor (#7509) 2025-01-02 22:25:15 -08:00
Ishaan JaffandGitHub d861aa8ff3 (perf) use aiohttp for custom_openai (#7514)
* use aiohttp handler

* BaseLLMAIOHTTPHandler

* use CustomOpenAIChatConfig

* CustomOpenAIChatConfig

* CustomOpenAIChatConfig

* fix linting

* AiohttpOpenAIChatConfig

* fix order

* aiohttp_openai
2025-01-02 22:15:17 -08:00
Ishaan JaffandGitHub 4d93fe787b Revert "(fix) GCS bucket logger - apply `truncate_standard_logging_payload_co…" (#7515)
This reverts commit 26a37c50c9.
2025-01-02 22:01:02 -08:00
Krish DholakiaandGitHub 25e6f46910 Litellm dev 01 02 2025 p2 (#7512)
* feat(deepgram/transformation.py): support reading in deepgram api base from env var

* fix(litellm_logging.py): make skipping log message a .info

easier to see

* docs(logging.md): add doc on turn off all tracking/logging for a request
2025-01-02 21:57:51 -08:00
Ishaan Jaff b9280528d3 docs enable_pre_call_checks 2025-01-02 08:27:03 -08:00
Krrish Dholakia c292f5805a docs(humanloop.md): add humanloop docs 2025-01-01 22:18:01 -08:00
Krish DholakiaandGitHub 45b93f2721 Litellm dev 01 01 2025 p3 (#7503)
* fix(utils.py): add new validate tool choice helper function

Prevents https://github.com/BerriAI/litellm/issues/7483

* fix(main.py): add tool choice validation on .completion()

prevents user error like - https://github.com/BerriAI/litellm/issues/7483

* fix(utils.py): fix return val of tool choice validation logic
2025-01-01 22:12:15 -08:00
Ishaan Jaff f96f54a0f5 test_aview_spend_per_user 2025-01-01 21:48:35 -08:00
Ishaan Jaff 8c29489c40 test_aadmin_only_routes 2025-01-01 21:48:20 -08:00
Ishaan Jaff 953f8c1ca8 ci/cd run again 2025-01-01 21:45:48 -08:00