Hi Authors,
Thank you for your great piece of work!
Can I check with you how you computed the KL divergence between aligned and unaligned models?
For example, the aligned Llama2-7B-Chat model requires a chat template while the unaligned model does not. Did you consider the special tokens that are required by the aligned chat model?
Hi Authors,
Thank you for your great piece of work!
Can I check with you how you computed the KL divergence between aligned and unaligned models?
For example, the aligned Llama2-7B-Chat model requires a chat template while the unaligned model does not. Did you consider the special tokens that are required by the aligned chat model?