Hello, I plan to reproduce "Musique (OOD)" in Table 2 of your Q-rag paper.
(1) Should I do "E5 HotpotQA only" training and then "E5 MuSiQue only" evaluation in Section 1 of ReadMe file?
When I do this,
FileNotFoundError: [Errno 2] No such file or directory: '../datasets/musique/hotpot_dev_distractor_v1.json'
(2) Also, the "LLM evaluation" in Section 1 is only for hotpotQA. Should I change "qrag-ft-e5-on-hotpotqa" to "qrag-ft-e5-on-musique" as follows to have LLM evaluation on musique?
python eval_llm_openqa.py \
--file_path your/path/to/qrag-ft-e5-on-musique/eval_seed42.jsonl \
--model_name Qwen/QwQ-32B \
--output_file_path your/path/to/qrag-ft-e5-on-hotpotqa/llm-answering_eval.json
Thanks.
Hello, I plan to reproduce "Musique (OOD)" in Table 2 of your Q-rag paper.
(1) Should I do "E5 HotpotQA only" training and then "E5 MuSiQue only" evaluation in Section 1 of ReadMe file?
When I do this,
FileNotFoundError: [Errno 2] No such file or directory: '../datasets/musique/hotpot_dev_distractor_v1.json'(2) Also, the "LLM evaluation" in Section 1 is only for hotpotQA. Should I change "qrag-ft-e5-on-hotpotqa" to "qrag-ft-e5-on-musique" as follows to have LLM evaluation on musique?
Thanks.