Hi, thanks for the great work on NL2RepoBench!
I encountered an issue while running the benchmark. When running tasks, the post-processing phase fails because the base image cannot be pulled from ghcr.io/multimodal-art-projection/nl2repobench/.
Error Log
2025-12-23 12:04:16 - INFO - Start building test image: python-test-aiofiles_bo1
2025-12-23 12:04:19 - ERROR - Failed to build image: The command executed was
`/usr/bin/docker buildx build --file workspaces/aiofiles_bo1/Dockerfile --tag python-test-aiofiles_bo1 ...`.
It returned with code 1
Cause
In openhands/post_processor.py (line 282), the Dockerfile template uses:
FROM --platform=linux/amd64 ghcr.io/multimodal-art-projection/nl2repobench/{project_name}:1.0
However, these images do not appear to exist or are not publicly accessible.
Inconsistency with README
The readme.md only mentions the following required images:
docker.all-hands.dev/all-hands-ai/openhands:0.56
docker.all-hands.dev/all-hands-ai/runtime:0.56-nikolaik
But I couldn't find documentation about the ghcr.io/multimodal-art-projection/nl2repobench/ images needed for post-processing.
Could you please clarify:
- Are these images available somewhere?
- Or is there additional setup required?
Thanks in advance!
Hi, thanks for the great work on NL2RepoBench!
I encountered an issue while running the benchmark. When running tasks, the post-processing phase fails because the base image cannot be pulled from
ghcr.io/multimodal-art-projection/nl2repobench/.Error Log
Cause
In
openhands/post_processor.py(line 282), the Dockerfile template uses:FROM --platform=linux/amd64 ghcr.io/multimodal-art-projection/nl2repobench/{project_name}:1.0However, these images do not appear to exist or are not publicly accessible.
Inconsistency with README
The
readme.mdonly mentions the following required images:docker.all-hands.dev/all-hands-ai/openhands:0.56docker.all-hands.dev/all-hands-ai/runtime:0.56-nikolaikBut I couldn't find documentation about the
ghcr.io/multimodal-art-projection/nl2repobench/images needed for post-processing.Could you please clarify:
Thanks in advance!