From 53371d37b75103e0d4aec348426ee60d092b96bb Mon Sep 17 00:00:00 2001 From: ishaan-jaff Date: Fri, 3 Nov 2023 18:11:07 -0700 Subject: [PATCH] (docs) lm eval harness --- .../docs/tutorials/lm_evaluation_harness.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/my-website/docs/tutorials/lm_evaluation_harness.md b/docs/my-website/docs/tutorials/lm_evaluation_harness.md index a66a474fdd..1e4faceefa 100644 --- a/docs/my-website/docs/tutorials/lm_evaluation_harness.md +++ b/docs/my-website/docs/tutorials/lm_evaluation_harness.md @@ -2,7 +2,7 @@ Evaluate LLMs 20x faster with TGI via litellm proxy's `/completions` endpoint. -This tutorial assumes you're using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) +This tutorial assumes you're using the `big-refactor` branch of [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/big-refactor) **Step 1: Start the local proxy** ```shell @@ -19,8 +19,9 @@ $ export OPENAI_API_BASE="http://0.0.0.0:8000" **Step 3: Run LM-Eval-Harness** ```shell -$ python3 main.py \ - --model gpt3 \ - --model_args engine=huggingface/bigcode/starcoder \ - --tasks hellaswag +python3 -m lm_eval \ + --model openai-completions \ + --model_args engine=davinci \ + --task crows_pairs_english_age + ``` \ No newline at end of file