mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-14 10:25:37 +00:00
(docs) lm harness tutorial
This commit is contained in:
@@ -1,3 +1,6 @@
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# LM-Evaluation Harness with TGI
|
||||
|
||||
Evaluate LLMs 20x faster with TGI via litellm proxy's `/completions` endpoint.
|
||||
@@ -9,6 +12,13 @@ This tutorial assumes you're using the `big-refactor` branch of [lm-evaluation-h
|
||||
$ litellm --model huggingface/bigcode/starcoder
|
||||
```
|
||||
|
||||
Using a custom api base
|
||||
|
||||
```shell
|
||||
$ export HUGGINGFACE_API_KEY=my-api-key #[OPTIONAL]
|
||||
$ litellm --model huggingface/tinyllama --api_base https://k58ory32yinf1ly0.us-east-1.aws.endpoints.huggingface.cloud
|
||||
```
|
||||
|
||||
OpenAI Compatible Endpoint at http://0.0.0.0:8000
|
||||
|
||||
**Step 2: Set OpenAI API Base**
|
||||
|
||||
Reference in New Issue
Block a user