Skip to content

docs: restructure SECURITY.md's sandbox section into a claims-vs-roadmap table (supersedes #65) #168

Description

@alphacrack

User story

As a security-conscious adopter, I want SECURITY.md to state exactly which sandbox controls are enforced in code today versus planned for v0.8, so that I can decide whether to run readme2demo against a given repo without discovering a claimed control doesn't exist yet.

What

Restructure the sandbox portion of SECURITY.md (the "threat model" + "Known tradeoffs" sections) into a claims-vs-roadmap table: one row per security claim, each row stating whether it is enforced today (with a file:line pointer to the enforcing code), planned (v0.8) (with a link to #64), or a known limitation (with the tracking issue).

Suggested starting rows — the contributor must re-verify each pointer against current main before writing it down (line numbers drift):

Claim Status Where
All Linux capabilities dropped enforced today src/readme2demo/sandbox.py start()--cap-drop ALL
No privilege escalation enforced today sandbox.py start()--security-opt no-new-privileges
Memory / CPU / PID caps enforced today sandbox.py start() (defaults 4g / 2 / 512 on the Sandbox dataclass)
Non-root user enforced today images/base/Dockerfileuseradd … demo + USER demo
Target repo mounted read-only enforced today agent.py/repo mounted "ro", agent works on a copy in /work
Verify replays in a fresh, credential-free container enforced today verify.pySandbox(...) constructed with no env=
Docker socket only on explicit opt-in enforced today agent.py--allow-docker-socket branch
Egress domain allowlist planned (v0.8) — today it is plain --network bridge sandbox.py start() --network (default "bridge"); #64
Host-side key-injecting proxy (credentials never enter the sandbox) planned (v0.8) #64
Disk quotas on the work volume planned (v0.8) #64
Red-team acceptance harness (hostile-README -m integration tests) planned (v0.8) #64
API key visible on the docker argv (-e KEY=VALUE) known limitation sandbox.py start() env loop; tracked in #51

Also:

Out of scope:

Why

Parent epic: #64 (v0.8 egress proxy & hardening). #64's acceptance explicitly includes rewriting SECURITY.md's tradeoff section; this slice does the honest-docs half now, before the proxy lands, so the doc stops claiming an unshipped control in the meantime.

Real incident: #65 documents that SECURITY.md:39 claims "Network egress is allowlisted" when no allowlisting exists anywhere in sandbox.py — the container gets stock --network bridge. For a project whose entire pitch is "nothing unverified gets published," a security doc asserting an unimplemented control is the most damaging possible docs bug. This issue is the grounding invariant applied to our own documentation: every "enforced" claim in the table must be backed by a file:line the author actually read, exactly like every command in tutorial.md must be backed by a fresh-container replay. No pipeline code changes, so no grounding-enforcement implications — but the doc must not gain any new unverified claim.

Pointers

All verified against current main:

  • SECURITY.md:12-18 — current prose claims: drop all capabilities, --no-new-privileges, non-root user, memory/CPU/PID caps. All true, but unpointered.
  • SECURITY.md:25-33 — tradeoff ci: bump actions/checkout from 4 to 7 #1: "The API key enters the sandbox." True today: agent.py:118 passes env=env into the agent's Sandbox.
  • SECURITY.md:34-38 — tradeoff ci: bump actions/deploy-pages from 4 to 5 #2: --allow-docker-socket opt-in. True: agent.py:106-115 mounts the socket and probes the gid only when cfg.allow_docker_socket is set.
  • SECURITY.md:39-42 — tradeoff ci: bump actions/upload-pages-artifact from 3 to 5 #3, the overstated one: "Network egress is allowlisted, not eliminated." Reality: sandbox.py:120 passes --network self.network, default network: str = "bridge" at sandbox.py:73. No allowlist exists.
  • src/readme2demo/sandbox.py:112-132start() builds the full flag set: --cap-drop ALL (line 115), --security-opt no-new-privileges (line 116), --memory (117, default "4g" at 74), --cpus (118, default "2" at 75), --pids-limit (119, default 512 at 76), --network (120), env injected as -e KEY=VALUE (127-128 — the sandbox: API keys are passed as -e KEY=VALUE on the docker argv — visible in /proc and docker inspect #51 exposure).
  • images/base/Dockerfile:68,72useradd --uid 1000 … demo and USER demo (the non-root claim lives in the image, not in start() — worth saying in the table).
  • src/readme2demo/agent.py:104[(str(run_dir / "repo"), "/repo", "ro")]: repo mounted read-only; agent.py:129 copies it to writable /work.
  • src/readme2demo/verify.py:72-81 — the verify Sandbox is constructed with no env= argument: the fresh replay container never sees credentials.
  • docs/security.md:18-25 — "Known tradeoffs" lists key-in-sandbox and docker-socket only; the egress tradeoff is absent (the two security pages currently disagree).

Acceptance criteria

Notes for contributors

  • Pure text, no Docker, no API keys, no GPU. You never run the pipeline. You do need to read three Python files (sandbox.py, agent.py, verify.py) and one Dockerfile to verify each row — that reading IS the task. If you can't find the line enforcing a claim, the claim goes in the "planned" or "limitation" column, not "enforced."
  • Setup is just pip install -e ".[dev]"; run python -m pytest tests/ -q (~1 second, no Docker or network needed) before opening the PR.
  • Line numbers in this issue were correct at filing time but will drift — cite what you see on your checkout, not what's written here.
  • Keep the tone of the existing doc: this project deliberately names its limitations ("We would rather name them than let you discover them"). The table should read as radical honesty, not marketing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:docsdocs/ site contentdocumentationDocs, README, docs-sitegood first issueSmall, self-contained, newcomer-friendlysecuritySandbox, credentials, isolation, threat model

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions