 ishaan-jaff
|
16fb83e007
|
(fix) proxy - remove errant print statement
|
2024-01-01 10:48:12 +05:30 |
|
|
ishaan-jaff
|
6c33fcbf6b
|
(feat) proxy-ui a separate dep
|
2024-01-01 09:23:35 +05:30 |
|
|
Krrish Dholakia
|
d0d08b4dce
|
docs(routing.md): adding latency-based routing to docs
|
2024-01-01 08:36:40 +05:30 |
|
|
ishaan-jaff
|
d71f89aac3
|
bump: version 1.16.7 → 1.16.8
|
2023-12-30 20:25:52 +05:30 |
|
|
ishaan-jaff
|
c269c65371
|
(docs) langfuse log trace id, trace user id
|
2023-12-30 20:25:23 +05:30 |
|
|
ishaan-jaff
|
84fbc903aa
|
(test) langfuse - set custom trace_id
|
2023-12-30 20:19:22 +05:30 |
|
|
ishaan-jaff
|
8ae4554a8a
|
(feat) langfuse - set custom trace_id, trace_user_id
|
2023-12-30 20:19:03 +05:30 |
|
|
ishaan-jaff
|
cc7b964433
|
(docs) add litellm.cache docstring
|
2023-12-30 20:04:08 +05:30 |
|
|
ishaan-jaff
|
70cdc16d6f
|
(feat) cache context manager - update cache
|
2023-12-30 19:50:53 +05:30 |
|
|
ishaan-jaff
|
bf4a9f40e8
|
(docs) cache context manager
|
2023-12-30 19:50:22 +05:30 |
|
|
ishaan-jaff
|
1c93642951
|
(docs) caching use context manager
|
2023-12-30 19:43:26 +05:30 |
|
|
ishaan-jaff
|
e35f17ca3c
|
(test) caching - context managers
|
2023-12-30 19:33:47 +05:30 |
|
|
ishaan-jaff
|
ddddfe6602
|
(feat) add cache context manager
|
2023-12-30 19:32:51 +05:30 |
|
|
ishaan-jaff
|
231148ed73
|
(docs) caching
|
2023-12-30 19:04:36 +05:30 |
|
|
Krrish Dholakia
|
8ff3bbcfee
|
fix(proxy_server.py): router model group alias routing
check model alias group routing before specific deployment routing, to deal with an alias being the same as a deployment name (e.g. gpt-3.5-turbo)
n
|
2023-12-30 17:55:24 +05:30 |
|
|
Krrish Dholakia
|
027218c3f0
|
test(test_lowest_latency_routing.py): add more tests
|
2023-12-30 17:41:42 +05:30 |
|
|
Krrish Dholakia
|
f2d0d5584a
|
fix(router.py): fix latency based routing
|
2023-12-30 17:25:40 +05:30 |
|
|
Krrish Dholakia
|
c41b1418d4
|
test(test_router_init.py): fix test router init
|
2023-12-30 16:51:39 +05:30 |
|
|
Krrish Dholakia
|
3cb7acceaa
|
test(test_least_busy_routing.py): fix test
|
2023-12-30 16:12:52 +05:30 |
|
|
Krrish Dholakia
|
3935f99083
|
test(test_router.py): add retries
|
2023-12-30 15:54:46 +05:30 |
|
|
Krrish Dholakia
|
69935db239
|
fix(router.py): periodically re-initialize azure/openai clients to solve max conn issue
|
2023-12-30 15:48:34 +05:30 |
|
|
Krrish Dholakia
|
d089157925
|
build(model_prices_and_context_window.json): fix text unicorn mapping
|
2023-12-30 13:38:24 +05:30 |
|
|
Krrish Dholakia
|
b66cf0aa43
|
fix(lowest_tpm_rpm_routing.py): broaden scope of get deployment logic
|
2023-12-30 13:27:50 +05:30 |
|
|
Krrish Dholakia
|
a6719caebd
|
fix(aimage_generation): fix response type
|
2023-12-30 12:53:24 +05:30 |
|
|
Krrish Dholakia
|
750432457b
|
fix(openai.py): fix async image gen call
|
2023-12-30 12:44:54 +05:30 |
|
|
Krrish Dholakia
|
2acd086596
|
test(test_least_busy_routing.py): fix test init
|
2023-12-30 12:39:13 +05:30 |
|
|
ishaan-jaff
|
535a547b66
|
(fix) use cloudflare optional params
|
2023-12-30 12:22:31 +05:30 |
|
|
Krrish Dholakia
|
c33c1d85bb
|
fix: support dynamic timeouts for openai and azure
|
2023-12-30 12:14:02 +05:30 |
|
|
Krrish Dholakia
|
77be3e3114
|
fix(main.py): don't set timeout as an optional api param
|
2023-12-30 11:47:07 +05:30 |
|
|
ishaan-jaff
|
aee38d9329
|
(fix) batch_completions - set default timeout
|
2023-12-30 11:35:55 +05:30 |
|
|
Krrish Dholakia
|
38f55249e1
|
fix(router.py): support retry and fallbacks for atext_completion
|
2023-12-30 11:19:32 +05:30 |
|
|
ishaan-jaff
|
7ecd7b3e8d
|
(docs) proxy - timeout per request
|
2023-12-30 11:18:03 +05:30 |
|
|
ishaan-jaff
|
6252987798
|
(docs) proxy - set timeout per request
|
2023-12-30 11:17:31 +05:30 |
|
|
ishaan-jaff
|
5d6954895f
|
(fix) timeout optional param
|
2023-12-30 11:07:52 +05:30 |
|
|
ishaan-jaff
|
523415cb0c
|
(test) dynamic timeout on router
|
2023-12-30 10:56:07 +05:30 |
|
|
ishaan-jaff
|
2f4cd3b569
|
(feat) proxy - support dynamic timeout per request
|
2023-12-30 10:55:42 +05:30 |
|
|
ishaan-jaff
|
459ba5b45e
|
(feat) router, add ModelResponse type hints
|
2023-12-30 10:44:13 +05:30 |
|
|
ishaan-jaff
|
44fc20a0ff
|
(feat) add cost tracking for voyage ai
|
2023-12-30 10:02:48 +05:30 |
|
|
ishaan-jaff
|
0a5f5812df
|
(feat) cloudflare pricing + models
|
2023-12-30 09:50:57 +05:30 |
|
|
Krrish Dholakia
|
a34de56289
|
fix(router.py): handle initial scenario for tpm/rpm routing
|
2023-12-30 07:28:45 +05:30 |
|
|
Krish DholakiaandGitHub
|
05795ead59
|
Merge pull request #1277 from AshGreh/patch-1
Adds support for Vertex AI Unicorn
|
2023-12-30 06:46:49 +05:30 |
|
|
Ash GrehandGitHub
|
f23f227087
|
Update model_prices_and_context_window.json
Corrected the math.
|
2023-12-29 20:53:39 -04:00 |
|
|
Ash GrehandGitHub
|
b0aea659bc
|
Update model_prices_and_context_window.json
Adds simplified name for Unicorn.
|
2023-12-29 20:47:12 -04:00 |
|
|
Ash GrehandGitHub
|
4b8a1800c9
|
Update model_prices_and_context_window.json
Adds support for unicorn
|
2023-12-29 20:45:25 -04:00 |
|
|
Krish DholakiaandGitHub
|
88a38d9c82
|
Merge pull request #1275 from marmikcfc/feature/add-response-format-for-mistral
add support for mistral json mode via anyscale
|
2023-12-29 23:10:52 +05:30 |
|
|
Marmik Pandya
|
1426594d3f
|
add support for mistral json mode via anyscale
|
2023-12-29 22:26:22 +05:30 |
|
|
Krrish Dholakia
|
2fc264ca04
|
fix(router.py): fix int logic
|
2023-12-29 20:41:56 +05:30 |
|
|
Krrish Dholakia
|
cf91e49c87
|
refactor(lowest_tpm_rpm.py): move tpm/rpm based routing to a separate file for better testing
|
2023-12-29 18:33:43 +05:30 |
|
|
Krrish Dholakia
|
54d7bc2cc3
|
test(test_least_busy_router.py): add better testing for least busy routing
|
2023-12-29 17:16:00 +05:30 |
|
|
Krrish Dholakia
|
678bbfa9be
|
fix(least_busy.py): support consistent use of model id instead of deployment name
|
2023-12-29 17:05:26 +05:30 |
|