 Ishaan Jaff
|
f6d7683261
|
[Feat] LiteLLM Overhead metric tracking - Add support for tracking litellm overhead on cache hits (#15045)
* test_litellm_overhead
* vertex track overhead
* fix config.yaml used for testing
* test_litellm_overhead_stream
* add update_response_metadata for caching handler
* add CachingDetails
* fix update_response_metadata import
* add CachingDetails metrics
* add CachingDetails
* test_litellm_overhead_cache_hit
* test_litellm_overhead_cache_hit
* test_litellm_overhead_cache_hit
|
2025-09-29 17:33:27 -07:00 |
|
|
Ishaan Jaffer
|
e0172b86e2
|
test_litellm_overhead_non_streaming
|
2025-09-29 15:48:32 -07:00 |
|
|
Ishaan Jaff
|
619577d4e8
|
[Feat] Add litellm overhead metric for VertexAI (#15040)
* test_litellm_overhead
* vertex track overhead
* fix config.yaml used for testing
* test_litellm_overhead_stream
* add update_response_metadata for caching handler
* Revert "add update_response_metadata for caching handler"
This reverts commit f2a891f2b448b878a5dbf4b5b0a6166c807b3705.
|
2025-09-29 15:15:25 -07:00 |
|
|
Ishaan Jaff
|
b6f2e659b9
|
(Feat) Add x-litellm-overhead-duration-ms and "x-litellm-response-duration-ms" in response from LiteLLM (#7899)
* add track_llm_api_timing
* add track_llm_api_timing
* test_litellm_overhead
* use ResponseMetadata class for setting hidden params and response overhead
* instrument http handler
* fix track_llm_api_timing
* track_llm_api_timing
* emit response overhead on hidden params
* fix resp metadata
* fix make_sync_openai_embedding_request
* test_aaaaatext_completion_endpoint fixes
* _get_value_from_hidden_params
* set_hidden_params
* test_litellm_overhead
* test_litellm_overhead
* test_litellm_overhead
* fix import
* test_litellm_overhead_stream
* add LiteLLMLoggingObject
* use diff folder for testing
* use diff folder for overhead testing
* test litellm overhead
* use typing
* clear typing
* test_litellm_overhead
* fix async_streaming
* update_response_metadata
* move test file
* pply metadata to the response objec
|
2025-01-21 20:27:55 -08:00 |
|