Search results

[bug] GPQA-Diamond ground truth answers are all D

Status: Open.

#155 In mlfoundations/evalchemy;

· xiefiona opened on Feb 24, 2026

vLLM backend crashes with "please provide at least one prompt" when max_gen_toks exceeds model context window

Status: Open.

#152 In mlfoundations/evalchemy;

· Ali-Elganzory opened on Jan 3, 2026

[bug] LiveCodeBenchv5

Status: Open.

#149 In mlfoundations/evalchemy;

· cyr0930 opened on Dec 23, 2025

[Bug] google/codegemma-7b-it Evaluation Results All Zero

Status: Open.

#147 In mlfoundations/evalchemy;

· juyongjiang opened on Dec 16, 2025

Only return loose result in IFEval?

Status: Open.

#145 In mlfoundations/evalchemy;

· luyy12 opened on Oct 28, 2025

final answer extraction in reasoning model's output

Status: Open.

#144 In mlfoundations/evalchemy;

· maziye-azad opened on Oct 2, 2025

LM Eval Tasks not found

Status: Open.

#143 In mlfoundations/evalchemy;

· AahilA opened on Sep 18, 2025

Difference between self.max_tokens self.max_new_tokens in eval_instruct.py

Status: Open.

#142 In mlfoundations/evalchemy;

· richardbaihe opened on Sep 3, 2025

Add logical reasoning task tests? (e.g., Enigmata data)

Status: Open.

#141 In mlfoundations/evalchemy;

· Siki-cloud opened on Aug 29, 2025

Support for SWEBench

Status: Open.

#140 In mlfoundations/evalchemy;

· sravan500 opened on Aug 12, 2025

Request to add Debug mode

Status: Open.

#139 In mlfoundations/evalchemy;

· richardzhuang0412 opened on Jul 23, 2025

per prompt performance and public data

Status: Open.

#132 In mlfoundations/evalchemy;

· chanansh opened on Jun 27, 2025