Hi, thanks for your great work and for maintaining this repository!
I noticed that the evaluation of MBPP does not require any post-processing, while for HumanEval, post-processing is typically needed (e.g., trimming extra outputs or formatting the generated code). I’m wondering if you could clarify the reason behind this difference.
Thanks in advance for your help!
Hi, thanks for your great work and for maintaining this repository!
I noticed that the evaluation of MBPP does not require any post-processing, while for HumanEval, post-processing is typically needed (e.g., trimming extra outputs or formatting the generated code). I’m wondering if you could clarify the reason behind this difference.
Thanks in advance for your help!