simpE is a simple way to benchmark small LLMs in the following areas:

• Instruction Following
• String Reversing
• Big Integer Addition
• String Rehearsal

Install

git clone https://github.com/Dariton4000/simpE.git

cd simpE

uv sync

uv run simpe

uv run analyze