File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -129,7 +129,7 @@ cd /tmp ; git clone --depth 1 https://github.com/EleutherAI/lm-evaluation-harnes
129129
130130huggingface-cli login --token ${YOUR_HF_TOKEN}
131131
132- HF_ALLOW_CODE_EVAL=1 lm_eval --model hf \
132+ HF_ALLOW_CODE_EVAL=1 lm_eval --model vllm \
133133 --model_args pretrained=${YOUR_MODEL_PATH} ,enable_thinking=False \
134134 --tasks ifeval \
135135 --batch_size auto \
@@ -140,7 +140,6 @@ HF_ALLOW_CODE_EVAL=1 lm_eval --model hf \
140140 --show_config \
141141 --seed 42 \
142142 --output_path ${YOUR_SAVE_PATH} $ \
143- --device cuda:0 \
144143 --confirm_run_unsafe_code
145144```
146145
Original file line number Diff line number Diff line change @@ -195,7 +195,7 @@ cd /tmp ; git clone --depth 1 https://github.com/EleutherAI/lm-evaluation-harnes
195195
196196huggingface-cli login --token ${YOUR_HF_TOKEN}
197197
198- HF_ALLOW_CODE_EVAL=1 lm_eval --model hf \
198+ HF_ALLOW_CODE_EVAL=1 lm_eval --model vllm \
199199 --model_args pretrained=${YOUR_MODEL_PATH} ,enable_thinking=False \
200200 --tasks ifeval \
201201 --batch_size auto \
@@ -206,7 +206,6 @@ HF_ALLOW_CODE_EVAL=1 lm_eval --model hf \
206206 --show_config \
207207 --seed 42 \
208208 --output_path ${YOUR_SAVE_PATH} $ \
209- --device cuda:0 \
210209 --confirm_run_unsafe_code
211210```
212211
You can’t perform that action at this time.
0 commit comments