Ishaan Jaff
b878a67a7c
fic otel load test %
2024-09-14 18:04:28 -07:00
Ishaan Jaff and GitHub
c8d15544c8
[Fix] Router cooldown logic - use % thresholds instead of allowed fails to cooldown deployments ( #5698 )
...
* move cooldown logic to it's own helper
* add new track deployment metrics folder
* increment success, fails for deployment in current minute
* fix cooldown logic
* fix test_aaarouter_dynamic_cooldown_message_retry_time
* fix test_single_deployment_no_cooldowns_test_prod_mock_completion_calls
* clean up get from deployment test
* fix _async_get_healthy_deployments
* add mock InternalServerError
* test deployment failing 25% requests
* add test_high_traffic_cooldowns_one_bad_deployment
* fix vertex load test
* add test for rate limit error models in cool down
* change default cooldown time
* fix cooldown message time
* fix cooldown on 429 error
* fix doc string for _should_cooldown_deployment
* fix sync cooldown logic router
2024-09-14 18:01:19 -07:00
Ishaan Jaff and GitHub
85acdb9193
[Feat] Add max_completion_tokens param ( #5691 )
...
* add max_completion_tokens
* add max_completion_tokens
* add max_completion_tokens support for OpenAI models
* add max_completion_tokens param
* add max_completion_tokens for bedrock converse models
* add test for converse maxTokens
* fix openai o1 param mapping test
* move test optional params
* add max_completion_tokens for anthropic api
* fix conftest
* add max_completion tokens for vertex ai partner models
* add max_completion_tokens for fireworks ai
* add max_completion_tokens for hf rest api
* add test for param mapping
* add param mapping for vertex, gemini + testing
* predibase is the most unstable and unusable llm api in prod, can't handle our ci/cd
* add max_completion_tokens to openai supported params
* fix fireworks ai param mapping
2024-09-14 14:57:01 -07:00
Ishaan Jaff and GitHub
741c8e8a45
[Feat - Perf Improvement] DataDog Logger 91% lower latency ( #5687 )
...
* fix refactor dd to be an instance of custom logger
* migrate dd logger to be async
* clean up dd logging
* add datadog sync and async code
* use batching for datadog logger
* add doc string for dd logging
* add clear doc string
* fix doc string
* allow debugging intake url
* clean up requirements.txt
* allow setting custom batch size on logger
* fix dd logging to use compression
* fix linting
* add dd load test
* fix dd load test
* fix dd url
* add test_datadog_logging_http_request
* fix test_datadog_logging_http_request
2024-09-13 17:39:17 -07:00
Ishaan Jaff and GitHub
cd8d7ca915
[Fix] Performance - use in memory cache when downloading images from a url ( #5657 )
...
* fix use in memory cache when getting images
* fix linting
* fix load testing
* fix load test size
* fix load test size
* trigger ci/cd again
2024-09-13 07:23:42 -07:00
Ishaan Jaff
88706488f9
fix otel load test
2024-09-11 21:27:31 -07:00
Ishaan Jaff
b80f27dce3
fix otel tests
2024-09-11 21:25:27 -07:00
Ishaan Jaff
97ecf86d3d
fix langsmith load tests
2024-09-11 21:19:03 -07:00
Ishaan Jaff
b01a42ef4f
fix langsmith load test
2024-09-11 21:16:16 -07:00
Ishaan Jaff
a1f8fcfeed
fix load test
2024-09-11 21:06:42 -07:00
Ishaan Jaff
850b5dbadc
add otel load test
2024-09-11 20:47:12 -07:00
Ishaan Jaff
e7b047223e
add langsmith logging test
2024-09-11 20:35:11 -07:00
Ishaan Jaff
39a8bb2bc4
add test test_regenerate_key_ui
2024-09-10 09:12:03 -07:00
Ishaan Jaff
aed59abe35
allow passing expiry time to /key/regenerate
2024-09-06 08:36:34 -07:00
Ishaan Jaff
18f019f87d
move prisma test to correct location
2024-09-05 15:50:39 -07:00
Ishaan Jaff
edc51f45ac
add error message on test
2024-09-05 15:46:13 -07:00
Ishaan Jaff
3d9049df6d
move folder key gen prisma is in
2024-09-05 15:24:00 -07:00
Ishaan Jaff
9aff6a4c9d
add test for ui usage endpoints
2024-09-05 15:06:53 -07:00
Ishaan Jaff
94d6e800ee
add test for internal vs admin user
2024-09-05 13:30:51 -07:00
Ishaan Jaff
f9a3e343bb
add ui testing folder
2024-09-05 13:13:58 -07:00
Ishaan Jaff
cdc312d51d
mark test_team_logging as flaky
2024-09-04 20:29:21 -07:00
Krrish Dholakia
25e49a59b2
test: skip flaky test
2024-09-04 08:20:57 -07:00
Ishaan Jaff
3c898e23ea
refactor secret managers
2024-09-03 10:58:02 -07:00
Krish Dholakia and GitHub
9f3fa29624
feat(router.py): Support Loadbalancing batch azure api endpoints ( #5469 )
...
* feat(router.py): initial commit for loadbalancing azure batch api endpoints
Closes https://github.com/BerriAI/litellm/issues/5396
* fix(router.py): working `router.acreate_file()`
* feat(router.py): working router.acreate_batch endpoint
* feat(router.py): expose router.aretrieve_batch function
Make it easy for user to retrieve the batch information
* feat(router.py): support 'router.alist_batches' endpoint
Adds support for getting all batches across all endpoints
* feat(router.py): working loadbalancing on `/v1/files`
* feat(proxy_server.py): working loadbalancing on `/v1/batches`
* feat(proxy_server.py): working loadbalancing on Retrieve + List batch
2024-09-02 21:32:55 -07:00
Ishaan Jaff
e9427205ef
add test for pass through streaming usage tracking
2024-09-02 16:17:49 -07:00
Ishaan Jaff
9e557ed072
fix test
2024-08-31 08:39:52 -07:00
Ishaan Jaff
b35bfb0302
fix cost tracking for vertex ai native
2024-08-31 08:22:27 -07:00
Ishaan Jaff
06857d108d
fix /spend logs call
2024-08-30 17:02:24 -07:00
Ishaan Jaff
2c86a62474
fix vertex ai test
2024-08-30 16:50:23 -07:00
Ishaan Jaff
f43060e8df
mark as async
2024-08-30 16:40:41 -07:00
Ishaan Jaff
414d2dcb52
call spend logs endpoint
2024-08-30 16:35:07 -07:00
Ishaan Jaff
f3f85f6141
add test for vertex basic pass throgh
2024-08-30 16:26:00 -07:00
8d6a0bdc81
- merge - fix TypeError: 'CompletionUsage' object is not subscriptable #5441 ( #5448 )
...
* fix TypeError: 'CompletionUsage' object is not subscriptable (#5441 )
* test(test_team_logging.py): mark flaky test
---------
Co-authored-by: yafei lee <yafei@dao42.com >
2024-08-30 08:54:42 -07:00
Ishaan Jaff and GitHub
e329c4509a
Merge branch 'main' into litellm_add_tag_control_team
2024-08-29 17:34:40 -07:00
Ishaan Jaff
4b7ceade64
mark test_key_info_spend_values_streaming as flaky
2024-08-29 14:39:53 -07:00
Ishaan Jaff
da2cefc45a
fix team based tag routing
2024-08-29 14:37:44 -07:00
Ishaan Jaff
f592aeaa38
add test_chat_completion_with_no_tags
2024-08-29 13:54:11 -07:00
Ishaan Jaff
34f1d32799
add test for tag based routing
2024-08-29 13:45:24 -07:00
Ishaan Jaff
bcf7f3e437
handle flaky pytests
2024-08-27 22:44:49 -07:00
Ishaan Jaff
c27640e6e4
add /rerank test
2024-08-27 17:50:37 -07:00
Ishaan Jaff
918e4fcfe5
feat add test for custom guardrails
2024-08-23 12:01:43 -07:00
Ishaan Jaff
1f0cc72531
test bedrock guardrails
2024-08-22 17:24:42 -07:00
Krish Dholakia and GitHub
409306b266
Merge branch 'main' into litellm_fix_azure_api_version
2024-08-20 11:40:53 -07:00
Ishaan Jaff
aceab2669f
test guardrails with API Key
2024-08-20 08:40:00 -07:00
Ishaan Jaff and Krrish Dholakia
04efdd66c0
fix test guardrails
2024-08-19 21:03:37 -07:00
Ishaan Jaff and Krrish Dholakia
b2d63f752c
add testing for guardrails
2024-08-19 21:03:37 -07:00
Ishaan Jaff and Krrish Dholakia
6fa421ed8e
test_llm_guard_triggered
2024-08-19 21:03:37 -07:00
Ishaan Jaff and Krrish Dholakia
9f927e0e4e
add testing for aporia guardrails
2024-08-19 21:03:37 -07:00
Ishaan Jaff
57bbcba079
fix test guardrails
2024-08-19 18:58:31 -07:00
Ishaan Jaff
5d262a6ee7
add testing for guardrails
2024-08-19 18:57:02 -07:00
Ishaan Jaff