Arc-c performance worse than random chance

[BrownianNotion]

Description

Evals on arc-challenge for both the 1bit/2bit models are 21.5% which is worse than random chance since almost all questions have 4 options (some have 3/5, assume this averages out). Note that some of the baselines in BitDistiller were also around this accuracy, so it isn't totally unexpected.

Dig into the model and understand why this is happening:

1. Understand the data by reading some examples.
2. Are there any common mistakes the model is making. Why could this be?

Checklist before resolving issue

• I have included the issue number in my PR description
• I have rebased/pulled in changes from main and fixed any conflicts
• If I have changed the models/training code, I have run dry_run.sh on a CUDA device successfully