Skip to content

[Epic]: Set up acceptance testing for the box runtime #17

Description

@michael-duren

Summary

The unit tests in internal/container cover the parts of the lifecycle that can
run without real side effects (state-machine guards, New/Save/Load against
temp dirs). By design they stop at the boundary where the runtime touches the
real system: the InitReexec unix-socket handshake, syscall.Exec, signal
delivery in Kill, and the process kill + cleanup in Delete.

This epic stands up an acceptance test suite that drives the real box
binary end-to-end against real implementations (real namespaces, sockets, and
processes) so those gaps are covered. Acceptance tests answer "does box create → start → state → kill → delete actually work on a Linux host?", complementing the
unit tests (fast, isolated) below them.

Relationship to OCI conformance: this is distinct from the
opencontainers/runtime-tools validation suite epic. That suite measures
spec-conformance using an upstream-defined harness; this epic is our own
project-owned, behavior-level coverage of the lifecycle. They overlap in intent
but are built and run separately.

Goals

  • A single command (e.g. make acceptance) builds box and runs the suite
    end-to-end on a Linux host.
  • The suite exercises the full lifecycle (create, start, state, kill,
    delete) against a real bundle + rootfs, asserting the OCI status transitions
    (creatingcreatedrunningstopped) and on-disk state.
  • The lifecycle paths that unit tests cannot reach are covered: the init/reexec
    socket handshake, the user process actually executing, signal delivery, and
    state/runtime file cleanup on delete.
  • The privilege model is decided and documented (root vs. rootless / user
    namespaces), and the suite skips cleanly when its prerequisites are absent.
  • The suite runs in CI (initially non-blocking, since the runtime is still
    under active development).

Tasks

Non-goals

  • Replacing the unit tests — acceptance tests are slower and run separately, not
    a substitute for the fast internal/container coverage.
  • 100% lifecycle coverage on day one; the harness lands first and cases accrue as
    the runtime matures.
  • OCI runtime-spec conformance reporting (tracked by the separate conformance
    epic).
  • Performance/load testing.

Open questions

  • Privileges: target rootless (user namespaces + newuidmap/subuid) or
    require root/sudo? The sample config.json we test with has no user
    namespace, which implies root today. Rootless may need spec changes first.
  • Rootfs source: vendor a tiny busybox/alpine rootfs tarball, fetch it at
    test time, or build one from docker export? (reproducibility vs. repo size)
  • CI runner: does ubuntu-latest allow the namespace operations we need
    (unprivileged user namespaces, mounting proc/sysfs), or do we need a privileged
    container / self-hosted runner?
  • Go testing vs. shell/bats: drive the binary from Go acceptance tests
    (build tag //go:build acceptance) or a bats-style script suite?

Target milestone

v0.1 — runtime brought up under real acceptance coverage

Metadata

Metadata

Assignees

No one assigned

    Labels

    epicA high-level tracking issue grouping smaller tasksroadmapPlanned, roadmap-level worktesting

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions