diff --git a/docs/my-website/docs/providers/huggingface.md b/docs/my-website/docs/providers/huggingface.md index 6319041f62..f8ebadfcfa 100644 --- a/docs/my-website/docs/providers/huggingface.md +++ b/docs/my-website/docs/providers/huggingface.md @@ -139,6 +139,72 @@ os.environ["HUGGINGFACE_API_KEY"] = "" os.environ["HUGGINGFACE_API_BASE"] = "" ``` +### Viewing Log probs + +#### Using `decoder_input_details` - OpenAI `echo` +The `echo` param is supported by OpenAI Completions - Use `litellm.text_completion()` for this +```python +from litellm import text_completion +response = text_completion( + model="huggingface/bigcode/starcoder", + prompt="good morning", + max_tokens=10, logprobs=10, + echo=True +) + ``` + +#### Output + ```json +{ + "id":"chatcmpl-3fc71792-c442-4ba1-a611-19dd0ac371ad", + "object":"text_completion", + "created":1698801125.936519, + "model":"bigcode/starcoder", + "choices":[ + { + "text":", I'm going to make you a sand", + "index":0, + "logprobs":{ + "tokens":[ + "good", + " morning", + ",", + " I", + "'m", + " going", + " to", + " make", + " you", + " a", + " s", + "and" + ], + "token_logprobs":[ + "None", + -14.96875, + -2.2285156, + -2.734375, + -2.0957031, + -2.0917969, + -0.09429932, + -3.1132812, + -1.3203125, + -1.2304688, + -1.6201172, + -0.010292053 + ] + }, + "finish_reason":"length" + } + ], + "usage":{ + "completion_tokens":9, + "prompt_tokens":2, + "total_tokens":11 + } +} +``` + ### Models with Prompt Formatting For models with special prompt templates (e.g. Llama2), we format the prompt to fit their template.