Request for LongRoPE2 Reproduction Code and Details

[L-z-Chen]

Hello LongRoPE Team,

Thank you for the great work.

Could you please let me know where I can find the code and reproduction details for LongRoPE2? I couldn't find them in this repository.

Thank you!

[J-shang]

hi @L-z-Chen , I push our dev branch to this branch https://github.com/microsoft/LongRoPE/tree/longrope2 , since it has been a long time since the last experiment and lack of documentation, if you have any questions, please feel free to contact us.

[L-z-Chen]

Thanks for sharing the longrope2 branch!

One quick question: would you be able to share the mid-training dataset mixture used for LongRoPE2? If it’s available, a Hugging Face dataset link would be super helpful for reproduction.

Thank you!