2b50b43ae2
Support passing prompt_label to langfuse ( #11018 )
...
* fix: add prompt label support to prompt management hook
* feat: support 'prompt_label' parameter for langfuse prompt management
Closes https://github.com/BerriAI/litellm/discussions/9003#discussioncomment-13221555
* fix(litellm_logging.py): deep copy optional params to avoid mutation while logging
* fix(log-consistent-optional-param-values-across-providers): ensures params can be used for finetuning from providers
* fix: fix linting error
* test: update test
* test: update langfuse tests
* fix(litellm_logging.py): avoid deepcopying optional params
might contain thread object
2025-05-21 22:27:36 -07:00
14321a2708
[Feat] Prometheus - Track route on proxy_* metrics ( #10992 )
...
* fix: trace route on prometheus metrics
* fix: show route on prometheus metrics for total fails
* test: trace route on metrics
* fix: tests for route in prom metrics
* test: fix test metrics
* test: fix test_proxy_failure_metrics
2025-05-20 22:55:55 -07:00
298912bd38
[UI] - Add Guardrail Tracing to LiteLLM SpendLogs ( #10893 )
...
* feat: trace guardrail SLP in spendLogs
* test: trace guardrail SLP in spendLogs
* add guardrail viewer
* checkpoint - working guardrail view on logs
* ui add guardrail view to SpendLogs
* test: fixes guardrails
* trace: fixes guardrails
2025-05-16 12:20:20 -07:00
42e6e664b2
[Refactor] Make Pagerduty a free feature ( #10857 )
...
* refactor: make pagerduty free
* refactor: make pagerduty free
* fix: pagerduty loc
* fix: linting error
2025-05-15 10:12:06 -07:00
2a994d7016
[Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook ( #10840 )
...
* feat: add GuardrailProviderSpecificParams
* feat: add add apply_guardrail helper for presidio
* ui cleanup
* fixes pii config on ui
* fixes for adding presidio pii
* refactor: InitializeGuardrails
* feat: init guardrails from DB
* allow running guardrails from test key pg
* fix: running a request with guardrails on UI
* fix: types/guardrails.py
* test: test_presidio_pre_call_hook_with_different_call_types
* test: test_initialize_presidio_guardrail
* test: fix custom guardrail tests
2025-05-14 17:41:33 -07:00
a421316e56
fix(litellm_logging.py): log custom headers in requester metadata ( #10818 )
...
* fix(litellm_logging.py): log custom headers in requester metadata
allows passing along custom headers from client to logging integration - e.g. `x-correlation-id`
* refactor: move enterprise code out of OSS package
work towards simplified CE version of docker image
* test: update test
* fix: fix linting error
2025-05-13 23:04:37 -07:00
a4fb1da2d9
fix: pass application/json for GenericAPILogger ( #10772 )
...
* fix: pass application/json for GenericAPILogger
* fix: test_generic_api_callback
2025-05-12 14:15:33 -07:00
d13117992c
fix: test_init_custom_logger_compatible_class_as_callback
2025-05-10 17:26:12 -07:00
3731ee436a
[Refactor] Use pip package for enterprise/ folder ( #10709 )
...
* init enterprise pip
* init enterprise pip
* init enterprise pip
* test: enterprise pip
* add litellm-enterprise to pip
* litellm ent check
* litellm ent check
* fix import email router
* fix setup_litellm_enterprise_pip
* fix local testing with enterprise pip
2025-05-09 17:18:48 -07:00
d9f2415f65
[Refactor - Filtering Spend Logs] Add status to root of SpendLogs table ( #10661 )
...
* add status to spendlogs
* add status to spendlogs
* add status to spendlogs
* add status to spendlogs
* fix tests with spendlogs
2025-05-08 11:29:25 -07:00
dd32860d62
[Feat] V2 Emails - Fixes for sending emails when creating keys + Resend API support ( #10602 )
...
* working email integration
* fix get_custom_loggers_for_type
* add SendKeyCreatedEmailEvent type
* bug fix, only send 1 email when creating key for user
* polish for emails for key created
* polish for key created email
* fix test_init_custom_logger_compatible_class_as_callback
* testing resend email integration
* testing fixes for email integration
2025-05-06 22:50:48 -07:00
500e6cddf5
Fix otel - follow genai semantic conventions + support 'instructions' param for tts ( #10608 )
...
* fix(opentelemetry.py): fix passing otel semantic conventions
Fixes SpanAttributes.LLM_PROMPTS to SpanAttributes.LLM_PROMPTS.value
* fix(opentelemetry.py): ensure spanattributes always pass the actual enum value
* fix(main.py): support passing 'instructions' param for gpt-4o-mini-tts
* test: update tests
2025-05-06 21:57:01 -07:00
489f1a6c25
[Feat] v2 Custom Logger API Endpoints ( #10575 )
...
* fixes for generic api logger
* tests for generic api logger
* test_generic_api_callback_multiple_logs
* allow health checking generic api endpoints
* docs generic api endpoint for logging
* allow setting headers for generic api callback
* fix for test_init_custom_logger_compatible_class_as_callback
* fix linting
2025-05-05 16:57:55 -07:00
3f45200802
test_openai_responses_api_web_search_cost_tracking
2025-05-03 20:22:42 -07:00
8faa7dbc39
fixes for web search tests
2025-05-03 18:10:30 -07:00
a37e4cb0b8
fix typo ( #10535 )
2025-05-03 11:39:40 -07:00
988e20aa36
[QA] Bedrock Vector Stores Integration - Allow using with registry + in OpenAI API spec with tools ( #10516 )
...
* refactor KB implementation to use central registry
* allow passing tools when making KB calls
* test fixes
* linting fix
* fix kb tests
* QA for KB stored in DB
* fix, use litellm_credential_name when adding KB on litellm UI
* QA list endpoint vector stores
* allow using UI creds with KBs
2025-05-03 08:30:38 -07:00
28cb7cc0ed
[Feat] KnowledgeBase/Vector Store - Log StandardLoggingVectorStoreRequest for requests made when a vector store is used ( #10509 )
...
* ensure vector store results are logged in SLP
* fix tests
* fix tests with vector_store_request_metadata
* fix linting
2025-05-02 13:43:20 -07:00
96e75628d6
[Fix + Refactor] Trigger Soft Budget Webhooks When Key Crosses Threshold ( #10491 )
...
* fix slack alerting with webhooks
* emit correct event group/entity on webhooks
* refactor to use a common class of alerts with abc methods
* fixes for tests
* refactor to use a common class of alerts with abc methods
* Send a budget alert on slack or webhook
* unit test slack alerting
* fix code qa
2025-05-02 07:06:07 -07:00
be885e4a17
[Feat] Vector Stores/KnowledgeBases - Allow defining Vector Store Configs ( #10448 )
...
* init vector store configs
* working kb init
* add vector store endpoints
* use litellm_credential_name
* working CRUD vector stores litellm
* working creds with vector DB
* ui cleanup
* clean up vector store id
* fix delete button
* refactored vector store component
* working selector for KBs
* ui vector stores
* add vector store tool calls usage on chat ui
* fixes for vector stores litellm
* test fix
* docs Knowledge Bases
* fixes for vector stores litellm
* fix linting
* add managed vectorstores
* fix orjson ci/cd test
* fix linting
* add types.tsx file
2025-04-30 21:49:59 -07:00
f30871ef13
[Feat] Add support for using Bedrock Knowledge Bases with LiteLLM /chat/completions requests ( #10413 )
...
* add make_bedrock_kb_retrieve_request
* working bedrock KB hook
* working bedrock KB hook
* test_openai_with_knowledge_base_mock_openai
* fix linting
* fix BedrockKnowledgeBaseHook
* docs using bedrock kb with litellm
* docs kb with litellm
* fix bedrock kb test
* DynamicPromptManagementParamLiteral
* fix _should_run_prompt_management_hooks_without_prompt_id
* test_init_custom_logger_compatible_class_as_callback
2025-04-29 17:29:02 -07:00
34cdefbb77
[Feat] Add logging callback support for /moderations API ( #10390 )
...
* add OpenAIModerationResponse as a type
* Add typing for litellm.moderation response obj
* Add typing for litellm.moderation response obj
* test_moderations_api_logging
* add assertions for logged slp object
* test_moderations_api_logging
* refactor moderations api usage
* Revert "test_moderations_api_logging"
This reverts commit 5a9001f04c592b063f016d2bb854416255ec6cdf.
* Revert "refactor moderations api usage"
This reverts commit b076d3174ca14251ee3fc28bd4744e25685c1e5f.
* test_moderations_api_logging
2025-04-28 20:24:17 -07:00
331e784db4
[Feat] Responses API - Add session management support for non-openai models ( #10321 )
...
* add session id in spendLogs
* fix log proxy server request as independant field
* use trace id for SpendLogs
* add _ENTERPRISE_ResponsesSessionHandler
* use _ENTERPRISE_ResponsesSessionHandler
* working session_ids
* working session management
* working session_ids
* test_async_gcs_pub_sub_v1
* test_spend_logs_payload_e2e
* working session_ids
* test_get_standard_logging_payload_trace_id
* test_get_standard_logging_payload_trace_id
* test_gcs_pub_sub.py
* fix all linting errors
* test_spend_logs_payload_with_prompts_enabled
* _ENTERPRISE_ResponsesSessionHandler
* _ENTERPRISE_ResponsesSessionHandler
* expose session id on ui
* get spend logs by session
* add sessionSpendLogsCall
* add session handling
* session logs
* ui session details
* fix on rowExpandDetails
* ui working sessions
2025-04-25 23:24:24 -07:00
96e31d205c
feat: Added Missing Attributes For Arize & Phoenix Integration ( #10043 ) ( #10215 )
...
* feat: Added Missing Attributes For Arize & Phoenix Integration
* chore: Added noqa for PLR0915 to suppress warning
* chore: Moved Contributor Test to Correct Location
* chore: Removed Redundant Fallback
Co-authored-by: Ali Saleh <saleh.a@turing.com >
2025-04-22 21:34:51 -07:00
44264ab6d6
fix failing agent ops test
2025-04-22 14:39:50 -07:00
b4f2b3dad1
test: update test to be more robust to usage updates
2025-04-19 09:26:26 -07:00
1ea046cc61
test: update tests to new deployment model ( #10142 )
...
* test: update tests to new deployment model
* test: update model name
* test: skip cohere rbac issue test
* test: update test - replace gpt-4o model
2025-04-18 14:22:12 -07:00
c1a642ce20
[UI] Allow setting prompt cache_control_injection_points ( #10000 )
...
* test_anthropic_cache_control_hook_system_message
* test_anthropic_cache_control_hook.py
* should_run_prompt_management_hooks
* fix should_run_prompt_management_hooks
* test_anthropic_cache_control_hook_specific_index
* fix test
* fix linting errors
* ChatCompletionCachedContent
* initial commit for cache control
* fixes ui design
* fix inserting cache_control_injection_points
* fix entering cache control points
* fixes for using cache control on ui + backend
* update cache control settings on edit model page
* fix init custom logger compatible class
* fix linting errors
* fix linting errors
* fix get_chat_completion_prompt
2025-04-14 21:17:42 -07:00
f9ce754817
[Feat] Add litellm.supports_reasoning() util to track if an llm supports reasoning ( #9923 )
...
* add supports_reasoning for xai models
* add "supports_reasoning": true for o1 series models
* add supports_reasoning util
* add litellm.supports_reasoning
* add supports reasoning for claude 3-7 models
* add deepseek as supports reasoning
* test_supports_reasoning
* add supports reasoning to model group info
* add supports_reasoning
* docs supports reasoning
* fix supports_reasoning test
* "supports_reasoning": false,
* fix test
* supports_reasoning
2025-04-11 17:56:04 -07:00
94a553dbb2
[Feat] Emit Key, Team Budget metrics on a cron job schedule ( #9528 )
...
* _initialize_remaining_budget_metrics
* initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* test_initialize_budget_metrics_cron_job
* LITELLM_PROXY_ADMIN_NAME
* fix code qa checks
* test_initialize_budget_metrics_cron_job
* test_initialize_budget_metrics_cron_job
* pod lock manager allow dynamic cron job ID
* fix pod lock manager
* require cronjobid for PodLockManager
* fix DB_SPEND_UPDATE_JOB_NAME acquire / release lock
* add comment on prometheus logger
* add debug statements for emitting key, team budget metrics
* test_pod_lock_manager.py
* test_initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* initialize_remaining_budget_metrics
* remove outdated test
2025-04-10 16:59:14 -07:00
b168f8b744
test: update test
2025-04-10 14:04:57 -07:00
0c5b4aa96d
feat(realtime/): add token tracking + log usage object in spend logs … ( #9843 )
...
* feat(realtime/): add token tracking + log usage object in spend logs metadata
* test: fix test
* test: update tests
* test: update testing
* test: update test
* test: update test
* test: update test
* test: update test
* test: update tesdt
* test: update test
2025-04-09 22:11:00 -07:00
4a128cfd64
Realtime API Cost tracking ( #9795 )
...
* fix(proxy_server.py): log realtime calls to spendlogs
Fixes https://github.com/BerriAI/litellm/issues/8410
* feat(realtime/): OpenAI Realtime API cost tracking
Closes https://github.com/BerriAI/litellm/issues/8410
* test: add unit testing for coverage
* test: add more unit testing
* fix: handle edge cases
2025-04-07 16:43:12 -07:00
b7cd4cef07
test_get_exception_class_name
2025-04-04 21:32:55 -07:00
df4593d58b
test prom unit tests
2025-04-04 21:30:05 -07:00
a3df0269bb
fix tests
2025-03-29 17:38:24 -07:00
047d767947
fix tests for gcs pub sub
2025-03-29 17:06:36 -07:00
d17ab7da2a
Merge pull request #9508 from BerriAI/litellm_fix_gcs_pub_sub
...
[Fix] Use StandardLoggingPayload for GCS Pub Sub Logging Integration
2025-03-24 18:22:43 -07:00
36d17c23ac
fix flaky test test_openai_responses_api_web_search_cost_tracking
2025-03-24 15:51:33 -07:00
6a0ff944e4
assert_gcs_pubsub_request_matches_expected_standard_logging_payload
2025-03-24 15:41:05 -07:00
43a6c4c635
test_openai_responses_api_web_search_cost_tracking
2025-03-22 17:29:17 -07:00
81033dc8c9
test_openai_responses_api_web_search_cost_tracking
2025-03-22 17:21:41 -07:00
beb68037d0
move web search cost tracking
2025-03-22 17:02:43 -07:00
475dfaa156
fixes for web search cost tracking
2025-03-22 16:56:32 -07:00
cf22d31b2b
search_context_cost_per_query
2025-03-22 14:52:58 -07:00
bddbeff717
test_openai_web_search_logging_cost_tracking
2025-03-22 14:29:02 -07:00
8690873488
test_arize_dynamic_params
2025-03-18 23:22:55 -07:00
8568caf532
test_arize_dynamic_params
2025-03-18 23:18:07 -07:00
19a7bfa6b5
test_arize_callback
2025-03-18 18:49:06 -07:00
00795f2bbf
test arize logging
2025-03-18 18:40:39 -07:00
Krish Dholakia