Ishaan Jaff and GitHub
72a91ea9dd
(fix) Langfuse key based logging ( #6372 )
...
* langfuse use helper for get_langfuse_logging_config
* fix get_langfuse_logger_for_request
* fix import
* fix get_langfuse_logger_for_request
* test_get_langfuse_logger_for_request_with_dynamic_params
* unit testing for test_get_langfuse_logger_for_request_with_no_dynamic_params
* parameterized langfuse testing
* fix langfuse test
* fix langfuse logging
* fix test_aaalangfuse_logging_metadata
* fix langfuse log metadata test
* fix langfuse logger
* use create_langfuse_logger_from_credentials
* fix test_get_langfuse_logger_for_request_with_no_dynamic_params
* fix correct langfuse/ folder structure
* use static methods for langfuse logger
* add commment on langfuse handler
* fix linting error
* add unit testing for langfuse logging
* fix linting
* fix failure handler langfuse
2024-10-23 18:24:22 +05:30
Ishaan Jaff and GitHub
b70147f63b
(refactor) litellm.Router client initialization utils ( #6394 )
...
* refactor InitalizeOpenAISDKClient
* use helper func for _should_create_openai_sdk_client_for_model
* use static methods for set client on litellm router
* reduce LOC in _get_client_initialization_params
* fix _should_create_openai_sdk_client_for_model
* code quality fix
* test test_should_create_openai_sdk_client_for_model
* test test_get_client_initialization_params_openai
* fix mypy linting errors
* fix OpenAISDKClientInitializationParams
* test_get_client_initialization_params_all_env_vars
* test_get_client_initialization_params_azure_ai_studio_mistral
* test_get_client_initialization_params_default_values
* fix _get_client_initialization_params
2024-10-23 17:33:19 +05:30
Ishaan Jaff and GitHub
3991d75511
(refactor) move convert dict to model response to llm_response_utils/ ( #6393 )
...
* refactor move convert dict to model response
* fix imports
* fix import _handle_invalid_parallel_tool_calls
2024-10-23 17:32:14 +05:30
Ishaan Jaff and GitHub
807e9dcea8
(docs + testing) Correctly document the timeout value used by litellm proxy is 6000 seconds + add to best practices for prod ( #6339 )
...
* fix docs use documented timeout
* document request timeout
* add test for litellm.request_timeout
* add test for checking value of timeout
2024-10-23 14:09:35 +05:30
64c3d3210c
build(deps): bump http-proxy-middleware in /docs/my-website ( #6395 )
...
Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware ) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases )
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.7/CHANGELOG.md )
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7 )
---
updated-dependencies:
- dependency-name: http-proxy-middleware
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-23 13:30:12 +05:30
Krrish Dholakia
0a92923f86
bump: version 1.50.2 → 1.50.3
2024-10-22 22:39:39 -07:00
+1
cb2563e3c0
Litellm dev 10 22 2024 ( #6384 )
...
* fix(utils.py): add 'disallowed_special' for token counting on .encode()
Fixes error when '<
endoftext
>' in string
* Revert "(fix) standard logging metadata + add unit testing (#6366 )" (#6381 )
This reverts commit 8359cb6fa9#6378 )
* Add claude 3 5 sonnet 20241022 models for all provides (#6380 )
* Add Claude 3.5 v2 on Amazon Bedrock and Vertex AI.
* added anthropic/claude-3-5-sonnet-20241022
* add new 35 mode lcard
---------
Co-authored-by: Paul Gauthier <paul@paulg.com >
Co-authored-by: lowjiansheng <15527690+lowjiansheng@users.noreply.github.com >
* test(skip-flaky-google-context-caching-test): google is not reliable. their sample code is also not working
* Fix metadata being overwritten in speech() (#6295 )
* fix: adding missing redis cluster kwargs (#6318 )
Co-authored-by: Ali Arian <ali.arian@breadfinancial.com >
* Add support for `max_completion_tokens` in Azure OpenAI (#6376 )
Now that Azure supports `max_completion_tokens`, no need for special handling for this param and let it pass thru. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure#api-support
* build(model_prices_and_context_window.json): add voyage-finance-2 pricing
Closes https://github.com/BerriAI/litellm/issues/6371
* build(model_prices_and_context_window.json): fix llama3.1 pricing model name on map
Closes https://github.com/BerriAI/litellm/issues/6310
* feat(realtime_streaming.py): just log specific events
Closes https://github.com/BerriAI/litellm/issues/6267
* fix(utils.py): more robust checking if unmapped vertex anthropic model belongs to that family of models
Fixes https://github.com/BerriAI/litellm/issues/6383
* Fix Ollama stream handling for tool calls with None content (#6155 )
* test(test_max_completions): update test now that azure supports 'max_completion_tokens'
* fix(handler.py): fix linting error
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com >
Co-authored-by: Low Jian Sheng <15527690+lowjiansheng@users.noreply.github.com >
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda >
Co-authored-by: Paul Gauthier <paul@paulg.com >
Co-authored-by: John HU <hszqqq12@gmail.com >
Co-authored-by: Ali Arian <113945203+ali-arian@users.noreply.github.com >
Co-authored-by: Ali Arian <ali.arian@breadfinancial.com >
Co-authored-by: Anand Taralika <46954145+taralika@users.noreply.github.com >
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com >
2024-10-22 21:18:54 -07:00
Ishaan Jaff and GitHub
b75019c1a5
(feat) Arize - Allow using Arize HTTP endpoint ( #6364 )
...
* arize use helper for get_arize_opentelemetry_config
* use helper to get Arize OTEL config
* arize add helpers for arize
* docs allow using arize http endpoint
* fix importing OTEL for Arize
* use static methods for ArizeLogger
* fix ArizeLogger tests
2024-10-23 09:38:35 +05:30
Krrish Dholakia
f943410e32
test(test_alangfuse.py): handle flaky langfuse test better
2024-10-22 13:33:29 -07:00
Krrish Dholakia
24a0d26eb1
test(skip-flaky-google-context-caching-test): google is not reliable. their sample code is also not working
2024-10-22 12:06:30 -07:00
7939e930ff
Add claude 3 5 sonnet 20241022 models for all provides ( #6380 )
...
* Add Claude 3.5 v2 on Amazon Bedrock and Vertex AI.
* added anthropic/claude-3-5-sonnet-20241022
* add new 35 mode lcard
---------
Co-authored-by: Paul Gauthier <paul@paulg.com >
Co-authored-by: lowjiansheng <15527690+lowjiansheng@users.noreply.github.com >
2024-10-22 11:51:16 -07:00
Low Jian Sheng and GitHub
21ace6de45
add new 35 mode lcard ( #6378 )
2024-10-22 23:39:20 +05:30
Ishaan Jaff and GitHub
400cbff9ba
Revert "(fix) standard logging metadata + add unit testing ( #6366 )" ( #6381 )
...
This reverts commit 8359cb6fa9
2024-10-22 23:20:01 +05:30
Ishaan Jaff and GitHub
8359cb6fa9
(fix) standard logging metadata + add unit testing ( #6366 )
...
* fix setting StandardLoggingMetadata
* add unit testing for standard logging metadata
* fix otel logging test
* fix linting
* fix typing
2024-10-22 19:13:19 +05:30
Ishaan Jaff
7853cb791d
fix docs configs.md
2024-10-22 18:16:47 +05:30
Ishaan Jaff
666741dbad
Merge remote-tracking branch 'origin/main'
2024-10-22 17:37:42 +05:30
Ishaan Jaff and GitHub
7a5f997fc9
(refactor) remove berrispendLogger - unused logging integration ( #6363 )
...
* fix remove berrispendLogger
* remove unused clickhouse logger
2024-10-22 16:53:25 +05:30
Hakan Taşköprü and GitHub
a0c5fee61d
Refactor: apply early return ( #6369 )
2024-10-22 16:41:54 +05:30
Ishaan Jaff
eca09ad89f
langfuse use helper for get_langfuse_logging_config
2024-10-22 16:41:29 +05:30
Krrish Dholakia
a5fe87d6a5
bump: version 1.50.1 → 1.50.2
2024-10-21 21:52:04 -07:00
Krrish Dholakia
e6e518ad58
docs(sidebars.js): add jina ai to left nav
2024-10-21 21:48:06 -07:00
Krrish Dholakia
e30a2743b7
docs(sidebars.js): add jina ai embedding to docs
2024-10-21 21:47:10 -07:00
Krish Dholakia and GitHub
2b9db05e08
feat(proxy_cli.py): add new 'log_config' cli param ( #6352 )
...
* feat(proxy_cli.py): add new 'log_config' cli param
Allows passing logging.conf to uvicorn on startup
* docs(cli.md): add logging conf to uvicorn cli docs
* fix(get_llm_provider_logic.py): fix default api base for litellm_proxy
Fixes https://github.com/BerriAI/litellm/issues/6332
* feat(openai_like/embedding): Add support for jina ai embeddings
Closes https://github.com/BerriAI/litellm/issues/6337
* docs(deploy.md): update entrypoint.sh filepath post-refactor
Fixes outdated docs
* feat(prometheus.py): emit time_to_first_token metric on prometheus
Closes https://github.com/BerriAI/litellm/issues/6334
* fix(prometheus.py): only emit time to first token metric if stream is True
enables more accurate ttft usage
* test: handle vertex api instability
* fix(get_llm_provider_logic.py): fix import
* fix(openai.py): fix deepinfra default api base
* fix(anthropic/transformation.py): remove anthropic beta header (#6361 )
2024-10-21 21:25:58 -07:00
Krish Dholakia and GitHub
7338b24a74
refactor(redis_cache.py): use a default cache value when writing to r… ( #6358 )
...
* refactor(redis_cache.py): use a default cache value when writing to redis
prevent redis from blowing up in high traffic
* refactor(redis_cache.py): refactor all cache writes to use self.get_ttl
ensures default ttl always used when writing to redis
Prevents redis db from blowing up in prod
2024-10-21 16:42:12 -07:00
Krish Dholakia and GitHub
199896f912
fix(proxy_server.py): add 'admin' user to db ( #6223 )
...
* fix(proxy_server.py): add 'admin' user to db
Fixes noisy error https://github.com/BerriAI/litellm/issues/6206
* fix(proxy_server.py): return correct 'userID' for `/login` endpoint
Fixes https://github.com/BerriAI/litellm/issues/6206
2024-10-21 12:19:02 -07:00
Zackeus Bengtsson and GitHub
c7bf693aff
fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values ( #6336 )
2024-10-21 09:38:36 -07:00
Ishaan Jaff and GitHub
274bf3e48d
(fix) get_response_headers for Azure OpenAI ( #6344 )
...
* fix get_response_headers
* unit testing for get headers
* unit testing for anthropic / azure openai headers
* increase test coverage for test_completion_response_ratelimit_headers
* fix test rate limit headers
2024-10-21 20:41:35 +05:30
Ishaan Jaff
fb523b79e9
bump: version 1.50.0 → 1.50.1
2024-10-21 16:02:16 +05:30
Ishaan Jaff and GitHub
d1f457d17a
(testing) add test coverage for init custom logger class ( #6341 )
...
* working test for init custom logger
* add test coverage for custom_logger_compatible_class_as_callback
2024-10-21 15:56:32 +05:30
Ishaan Jaff
bd9e29b8b9
working test for init custom logger
2024-10-21 14:33:52 +05:30
Ishaan Jaff
24a3090ff6
fix init logger tests
2024-10-21 14:25:19 +05:30
Ishaan Jaff
11adc12326
add unit tests for init callbacks
2024-10-21 14:20:37 +05:30
Ishaan Jaff and GitHub
f4630a09bb
fix - unhandled jsonDecodeError in convert_to_model_response_object ( #6338 )
...
* fix unhandled jsonDecodeError
* add unit testing for convert dict to chat completion
2024-10-21 12:59:47 +05:30
Krish Dholakia and GitHub
905ebeb924
feat(custom_logger.py): expose new async_dataset_hook for modifying… ( #6331 )
...
* feat(custom_logger.py): expose new `async_dataset_hook` for modifying/rejecting argilla items before logging
Allows user more control on what gets logged to argilla for annotations
* feat(google_ai_studio_endpoints.py): add new `/azure/*` pass through route
enables pass-through for azure provider
* feat(utils.py): support checking ollama `/api/show` endpoint for retrieving ollama model info
Fixes https://github.com/BerriAI/litellm/issues/6322
* fix(user_api_key_auth.py): add `/key/delete` to an allowed_ui_routes
Fixes https://github.com/BerriAI/litellm/issues/6236
* fix(user_api_key_auth.py): remove type ignore
* fix(user_api_key_auth.py): route ui vs. api token checks differently
Fixes https://github.com/BerriAI/litellm/issues/6238
* feat(internal_user_endpoints.py): support setting models as a default internal user param
Closes https://github.com/BerriAI/litellm/issues/6239
* fix(user_api_key_auth.py): fix exception string
* fix(user_api_key_auth.py): fix error string
* fix: fix test
2024-10-20 09:00:04 -07:00
7cc12bd5c6
LiteLLM Minor Fixes & Improvements (10/18/2024) ( #6320 )
...
* fix(converse_transformation.py): handle cross region model name when getting openai param support
Fixes https://github.com/BerriAI/litellm/issues/6291
* LiteLLM Minor Fixes & Improvements (10/17/2024) (#6293 )
* fix(ui_sso.py): fix faulty admin only check
Fixes https://github.com/BerriAI/litellm/issues/6286
* refactor(sso_helper_utils.py): refactor /sso/callback to use helper utils, covered by unit testing
Prevent future regressions
* feat(prompt_factory): support 'ensure_alternating_roles' param
Closes https://github.com/BerriAI/litellm/issues/6257
* fix(proxy/utils.py): add dailytagspend to expected views
* feat(auth_utils.py): support setting regex for clientside auth credentials
Fixes https://github.com/BerriAI/litellm/issues/6203
* build(cookbook): add tutorial for mlflow + langchain + litellm proxy tracing
* feat(argilla.py): add argilla logging integration
Closes https://github.com/BerriAI/litellm/issues/6201
* fix: fix linting errors
* fix: fix ruff error
* test: fix test
* fix: update vertex ai assumption - parts not always guaranteed (#6296 )
* docs(configs.md): add argila env var to docs
* docs(user_keys.md): add regex doc for clientside auth params
* docs(argilla.md): add doc on argilla logging
* docs(argilla.md): add sampling rate to argilla calls
* bump: version 1.49.6 → 1.49.7
* add gpt-4o-audio models to model cost map (#6306 )
* (code quality) add ruff check PLR0915 for `too-many-statements` (#6309 )
* ruff add PLR0915
* add noqa for PLR0915
* fix noqa
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* doc fix Turn on / off caching per Key. (#6297 )
* (feat) Support `audio`, `modalities` params (#6304 )
* add audio, modalities param
* add test for gpt audio models
* add get_supported_openai_params for GPT audio models
* add supported params for audio
* test_audio_output_from_model
* bump openai to openai==1.52.0
* bump openai on pyproject
* fix audio test
* fix test mock_chat_response
* handle audio for Message
* fix handling audio for OAI compatible API endpoints
* fix linting
* fix mock dbrx test
* (feat) Support audio param in responses streaming (#6312 )
* add audio, modalities param
* add test for gpt audio models
* add get_supported_openai_params for GPT audio models
* add supported params for audio
* test_audio_output_from_model
* bump openai to openai==1.52.0
* bump openai on pyproject
* fix audio test
* fix test mock_chat_response
* handle audio for Message
* fix handling audio for OAI compatible API endpoints
* fix linting
* fix mock dbrx test
* add audio to Delta
* handle model_response.choices.delta.audio
* fix linting
* build(model_prices_and_context_window.json): add gpt-4o-audio audio token cost tracking
* refactor(model_prices_and_context_window.json): refactor 'supports_audio' to be 'supports_audio_input' and 'supports_audio_output'
Allows for flag to be used for openai + gemini models (both support audio input)
* feat(cost_calculation.py): support cost calc for audio model
Closes https://github.com/BerriAI/litellm/issues/6302
* feat(utils.py): expose new `supports_audio_input` and `supports_audio_output` functions
Closes https://github.com/BerriAI/litellm/issues/6303
* feat(handle_jwt.py): support single dict list
* fix(cost_calculator.py): fix linting errors
* fix: fix linting error
* fix(cost_calculator): move to using standard openai usage cached tokens value
* test: fix test
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com >
2024-10-19 22:23:27 -07:00
Krish Dholakia and GitHub
c58d542282
Litellm openai audio streaming ( #6325 )
...
* refactor(main.py): streaming_chunk_builder
use <100 lines of code
refactor each component into a separate function - easier to maintain + test
* fix(utils.py): handle choices being None
openai pydantic schema updated
* fix(main.py): fix linting error
* feat(streaming_chunk_builder_utils.py): update stream chunk builder to support rebuilding audio chunks from openai
* test(test_custom_callback_input.py): test message redaction works for audio output
* fix(streaming_chunk_builder_utils.py): return anthropic token usage info directly
* fix(stream_chunk_builder_utils.py): run validation check before entering chunk processor
* fix(main.py): fix import
2024-10-19 16:16:51 -07:00
Ishaan Jaff and GitHub
979e8ea526
(refactor) get_cache_key to be under 100 LOC function ( #6327 )
...
* refactor - use helpers for name space and hashing
* use openai to get the relevant supported params
* use helpers for getting cache key
* fix test caching
* use get/set helpers for preset cache keys
* make get_cache_key under 100 LOC
* fix _get_model_param_value
* fix _get_caching_group
* fix linting error
* add unit testing for get cache key
* test_generate_streaming_content
2024-10-19 15:21:11 +05:30
Ishaan Jaff and GitHub
4cbdad9fc5
doc - using gpt-4o-audio-preview ( #6326 )
...
* doc on audio models
* doc supports vision
* doc audio input / output
2024-10-19 09:34:56 +05:30
Ishaan Jaff
89c71f1eb3
bump: version 1.49.7 → 1.50.0
2024-10-19 09:03:01 +05:30
Ishaan Jaff and GitHub
19eff1a4b4
(feat) - allow using os.environ/ vars for any value on config.yaml ( #6276 )
...
* add check for os.environ vars when readin config.yaml
* use base class for reading from config.yaml
* fix import
* fix linting
* add unit tests for base config class
* fix order of reading elements from config.yaml
* unit tests for reading configs from files
* fix user_config_file_path
* use simpler implementation
* use helper to get_config
* working unit tests for reading configs
2024-10-19 09:00:27 +05:30
Ishaan Jaff and GitHub
a0d45ba516
(feat) Support audio param in responses streaming ( #6312 )
...
* add audio, modalities param
* add test for gpt audio models
* add get_supported_openai_params for GPT audio models
* add supported params for audio
* test_audio_output_from_model
* bump openai to openai==1.52.0
* bump openai on pyproject
* fix audio test
* fix test mock_chat_response
* handle audio for Message
* fix handling audio for OAI compatible API endpoints
* fix linting
* fix mock dbrx test
* add audio to Delta
* handle model_response.choices.delta.audio
* fix linting
2024-10-18 19:16:14 +05:30
Ishaan Jaff and GitHub
13e0b3f626
(feat) Support audio, modalities params ( #6304 )
...
* add audio, modalities param
* add test for gpt audio models
* add get_supported_openai_params for GPT audio models
* add supported params for audio
* test_audio_output_from_model
* bump openai to openai==1.52.0
* bump openai on pyproject
* fix audio test
* fix test mock_chat_response
* handle audio for Message
* fix handling audio for OAI compatible API endpoints
* fix linting
* fix mock dbrx test
2024-10-18 19:14:25 +05:30
Ishaan Jaff and GitHub
e35fc3203e
doc fix Turn on / off caching per Key. ( #6297 )
2024-10-18 15:43:04 +05:30
Ishaan Jaff and GitHub
610974b4fc
(code quality) add ruff check PLR0915 for too-many-statements ( #6309 )
...
* ruff add PLR0915
* add noqa for PLR0915
* fix noqa
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
2024-10-18 15:36:49 +05:30
Ishaan Jaff and GitHub
7724d5895c
add gpt-4o-audio models to model cost map ( #6306 )
2024-10-18 12:45:42 +05:30
Krrish Dholakia
c29853552b
bump: version 1.49.6 → 1.49.7
2024-10-17 23:01:06 -07:00
Krrish Dholakia
6e7e96211c
docs(argilla.md): add sampling rate to argilla calls
2024-10-17 22:54:12 -07:00
Krrish Dholakia
4f5ff65882
docs(argilla.md): add doc on argilla logging
2024-10-17 22:51:55 -07:00
Krrish Dholakia
87f256bec1
docs(user_keys.md): add regex doc for clientside auth params
2024-10-17 22:42:29 -07:00
Krish Dholakia and GitHub
f252350881
LiteLLM Minor Fixes & Improvements (10/17/2024) ( #6293 )
...
* fix(ui_sso.py): fix faulty admin only check
Fixes https://github.com/BerriAI/litellm/issues/6286
* refactor(sso_helper_utils.py): refactor /sso/callback to use helper utils, covered by unit testing
Prevent future regressions
* feat(prompt_factory): support 'ensure_alternating_roles' param
Closes https://github.com/BerriAI/litellm/issues/6257
* fix(proxy/utils.py): add dailytagspend to expected views
* feat(auth_utils.py): support setting regex for clientside auth credentials
Fixes https://github.com/BerriAI/litellm/issues/6203
* build(cookbook): add tutorial for mlflow + langchain + litellm proxy tracing
* feat(argilla.py): add argilla logging integration
Closes https://github.com/BerriAI/litellm/issues/6201
* fix: fix linting errors
* fix: fix ruff error
* test: fix test
* fix: update vertex ai assumption - parts not always guaranteed (#6296 )
* docs(configs.md): add argila env var to docs
2024-10-17 22:09:11 -07:00
