diff --git a/docs/my-website/docs/tutorials/lm_evaluation_harness.md b/docs/my-website/docs/tutorials/lm_evaluation_harness.md index a66a474fdd..1e4faceefa 100644 --- a/docs/my-website/docs/tutorials/lm_evaluation_harness.md +++ b/docs/my-website/docs/tutorials/lm_evaluation_harness.md @@ -2,7 +2,7 @@ Evaluate LLMs 20x faster with TGI via litellm proxy's `/completions` endpoint. -This tutorial assumes you're using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) +This tutorial assumes you're using the `big-refactor` branch of [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/big-refactor) **Step 1: Start the local proxy** ```shell @@ -19,8 +19,9 @@ $ export OPENAI_API_BASE="http://0.0.0.0:8000" **Step 3: Run LM-Eval-Harness** ```shell -$ python3 main.py \ - --model gpt3 \ - --model_args engine=huggingface/bigcode/starcoder \ - --tasks hellaswag +python3 -m lm_eval \ + --model openai-completions \ + --model_args engine=davinci \ + --task crows_pairs_english_age + ``` \ No newline at end of file