Thank you for sharing this great work! I am currently testing the model using the provided code, but I found that my inference results are different from those reported in the paper. I want to make sure I am using the correct configuration for inference. Specifically, when running the following command:
torchrun --nproc-per-node=8 eval/eval_videomme.py --data_path ./data/VideoMME --version qwen --model_path ./checkpoints/longvu_qwen
The final accuracy I obtain is only 34.04%, which is much lower than what is presented in the paper.
Could you clarify what specific settings should be used in config for inference? Are there any additional preprocessing or postprocessing steps that might affect the results? I would really appreciate any guidance on how to best reproduce the reported results, thanks for your help!
Thank you for sharing this great work! I am currently testing the model using the provided code, but I found that my inference results are different from those reported in the paper. I want to make sure I am using the correct configuration for inference. Specifically, when running the following command:
torchrun --nproc-per-node=8 eval/eval_videomme.py --data_path ./data/VideoMME --version qwen --model_path ./checkpoints/longvu_qwenThe final accuracy I obtain is only 34.04%, which is much lower than what is presented in the paper.
Could you clarify what specific settings should be used in config for inference? Are there any additional preprocessing or postprocessing steps that might affect the results? I would really appreciate any guidance on how to best reproduce the reported results, thanks for your help!