I'm trying to reproduce a 73.4% accuracy baseline for Qwen3-4B-Base on MATH500 using your codebase. I'd like to confirm the exact configuration that produced that number.
What I've tried (all on the same checkpoint, single seed, vLLM 0.20.2 across 4 A40 GPUs):
In the paper, you say you use temp=0.6, top-p=0.95, CoT + 4-shot prompt (for base model, same as Qwen3 report) and Pass@1 metric on math benchmark. But in this setting, I get 53.6% only. Can you confirm with the exact parameter for base model on the math benchmark? Thank you!
I'm trying to reproduce a 73.4% accuracy baseline for Qwen3-4B-Base on MATH500 using your codebase. I'd like to confirm the exact configuration that produced that number.
What I've tried (all on the same checkpoint, single seed, vLLM 0.20.2 across 4 A40 GPUs):
In the paper, you say you use temp=0.6, top-p=0.95, CoT + 4-shot prompt (for base model, same as Qwen3 report) and Pass@1 metric on math benchmark. But in this setting, I get 53.6% only. Can you confirm with the exact parameter for base model on the math benchmark? Thank you!