How can I execute the script to obtain a CKPT similar to your Lyra-Mini-3B?

[cheng-haha]

Currently, I have trained Stage 2, but I only have the LoRA weights. How can I execute the script to obtain a CKPT similar to Lyra-Mini-3B? This is because I want to run lyra_chartvqa_speech.sh, but the script don't save the complete CKPT.

[zs-zhong]

Hi, you have two ways to resolve this:

• Replace LORA_PATH="Lyra_Base_9B/speech_lora" in lyra_chartvqa_speech.sh with the path to the LoRA model you trained.

• Use scripts/train/Lyra_Base_9B/Lyra_Base_qwen2vl_9B_SFT_text_image_speech_tmp_model.sh to obtain a new temp model, and modify CKPT="Lyra_Base_9B" to the path of the temp model you trained.

For the second way, you need to create your own data/one_sample.json, containing only one sample, to quickly obtain a temp model.

[cheng-haha]

Thank you! This issue has basically been resolved.

1. Is the evaluation for image-text and image-video based on the lmms-eval framework?
2. Was the evaluation conducted by merging LoRA weights, or by modifying the loading of LoRA within lmms-eval?
3. Are there plans to provide similar scripts?