ci: add CI/CD, release automation, code quality, and dev tooling#1
Closed
tolgakaratas wants to merge 38 commits intomasterfrom
Closed
ci: add CI/CD, release automation, code quality, and dev tooling#1tolgakaratas wants to merge 38 commits intomasterfrom
tolgakaratas wants to merge 38 commits intomasterfrom
Conversation
tolgakaratas
force-pushed
the
ci/add-cicd-and-tooling
branch
2 times, most recently
from
366225b to
481f40b
Compare
Source code changes (no CI/infrastructure): - Cross-platform module gating: storage/virtio keep tests portable, Linux-only modules gated with cfg(target_os = "linux") - Shared compat module (IoctlReq, SendPthreadT) for glibc/musl differences - All clippy lints resolved via cargo fix + cargo clippy --fix on Rust 1.95 - musl static build compatibility: SYS_renameat2 raw syscall, platform- correct ioctl types, Send wrapper for pthread_t - Fix _host_offset naming bug in balloon inflate (compile error on Linux) - Platform-conditional cast for libc::S_IFMT (u16 macOS, u32 Linux) - dead_code allow on modules with forward-declared upstream API - rustfmt applied with max_width=120 Verified: 0 clippy errors on Linux (rust:1.95) and macOS, 266+188 tests pass.
- profile.release: LTO fat, codegen-units=1, panic=abort, strip=true - Cargo.toml: homepage, repository, keywords, MSRV 1.87 - Workspace members: add rust-version = "1.87" - rustfmt.toml: max_width=120 matching original codebase style - .editorconfig: consistent settings across editors - Makefile: add shift-left targets (make ci, make fix, make lint) - .gitignore: add VM artifact patterns (*.img, *.qcow2)
Workflows: - build.yml: fmt, clippy, musl static build+test, MSRV 1.87 check, cargo-deny, security audit (with smart change detection) - release-please.yml: conventional commits to automated release PRs - release.yml: x86_64+aarch64 musl static binaries, SHA256 checksums, cosign keyless signing, SLSA attestation, SBOM (SPDX) - security-scan.yml: weekly cargo audit, cargo deny, CodeQL Rust - dependabot.yml: weekly cargo+actions updates with semantic grouping - dependabot-auto-merge.yml: auto-squash-merge patch/minor updates Templates: - Issue templates (bug report, feature request) - Pull request template with checklist
- SECURITY.md: vulnerability reporting via GitHub private advisories - CONTRIBUTING.md: setup, shift-left local CI (make ci), pre-commit hooks, conventional commits, code style guide - CHANGELOG.md: initial file for release-please automation - README.md: CI status, license, and MSRV badges
- mise: rust + cargo-binstall + pre-commit; setup/ci tasks - pre-commit: cargo autofix on commit, test+deny on push - deny.toml: license allowlist (MIT/Apache/BSD/ISC), advisory checks - release-please: Rust release type, version sync, changelog sections
tolgakaratas
force-pushed
the
ci/add-cicd-and-tooling
branch
from
481f40b to
f4c9ef8
Compare
clone-init forks the agent off PID 1 of the initrd before exec'ing systemd. Inside that fork-descendant, every blocking sleep call (usleep/nanosleep/std::thread::sleep) never wakes — the kernel timer state for the child is wedged. The pre-execve usleep(50_000) killed the child mid-sleep, and the agent's heartbeat loop wedged on its first SO_RCVTIMEO recv after sending Ready. - crates/clone-init/src/main.rs: drop the pre-execve usleep; child setsid + execve immediately so the kernel doesn't park it. - crates/guest-agent/src/main.rs: replace every blocking sleep with libc::sched_yield() loops; mark the vsock fd O_NONBLOCK and use MSG_PEEK + MSG_DONTWAIT for recv pacing. - src/virtio/vsock.rs: log every TX op so heartbeat-cadence regressions are visible in the VMM stderr stream.
Serial::write buffered guest stdout until \n or 256 bytes, so
no-trailing-newline payloads (notably the `clone login: ` prompt
agetty prints and then sleeps in ppoll) never reached the
/tmp/clone-{pid}.console socket. `clone attach` showed nothing.
- src/vmm/serial.rs: tee every byte to console_fd immediately;
retain an 8 KiB rolling history of recent output.
- src/vmm/mod.rs: on console-client attach, replay the history before
registering the live tee fd, so a late `clone attach` still sees
the boot banner and login prompt that were already printed.
Vcpu::new masked off TSC-deadline (CPUID.1.ECX[24]) and the kvmclock
feature bits (CPUID.0x40000001.EAX[0,3,24]) to dodge a fork/restore
bug where MSR_KVM_SYSTEM_TIME_NEW didn't round-trip through GET_MSRS.
Cost: the guest fell back to TSC calibration via PIT/HPET, the
in-kernel irqchip under-delivered ticks on idle APs, and idle CPUs
received ~zero LOC interrupts. systemd then wedged in
synchronize_rcu_normal because the grace period waits for every CPU
to pass through a quiescent state, which a tickless idle AP never does.
- src/vmm/vcpu.rs: keep both TSC-deadline and kvmclock bits in fresh
CPUID. Pin TSC frequency via set_tsc_khz(get_tsc_khz()) so the
guest doesn't have to calibrate against PIT/HPET. Fork path
(from_template) keeps its existing snapshot-aware MSR handling.
- src/main.rs: drop the rcupdate.rcu_expedited=1 +
rcu_normal_after_boot=0 cmdline workaround now that the underlying
timer path is fixed.
Verified on Ubuntu rootfs, 2 vCPUs, 1 GB RAM:
before: LOC cpu0=102 cpu1=0 over 17 min, clocksource=tsc-early,
systemd in D-state on synchronize_rcu_normal
after: LOC cpu0=17936 cpu1=1030 over ~1 min, clocksource=tsc
(kvm-clock available), systemd S-state and reaches login.
Each of the three bugs we just fixed is silent under the existing test
suite — heartbeat-cadence, no-newline serial delivery, and per-vCPU
LAPIC tick rate are not asserted anywhere. Add narrow tests that fail
loudly if any of these regress.
- tests/e2e/lib.sh:
- count_heartbeats / wait_for_n_heartbeats: parse the VMM stderr for
vsock OP_RW frames carrying ~150 byte heartbeat JSON.
- attach_console_assert: poll the /tmp/clone-{pid}.console socket
with timeout, grep the rolling history for a pattern.
- cleanup snapshots WORK_DIR to /tmp/last-e2e-workdir, and
start_vm_rootfs mirrors the serial log to /tmp/last-vm-serial.log
so failed runs leave forensic state.
- tests/e2e/run_all.sh:
- test_agent_heartbeat: ≥5 heartbeats in 18 s — would have caught
the agent-loop wedge in seconds.
- test_serial_login_prompt: history-replay + partial-line ("UNSUPP"
boot line, no trailing \n) — guards the serial flush bug.
- test_boot_determinism: CLONE_BOOT_OK within 20 s budget.
- test_timer_ticks: Δ-LOC delta on each vCPU over 3 s — guards the
LAPIC starvation bug now that the fix is in.
- crates/guest-agent/tests/sleep_deadline.rs: cargo unit tests
asserting libc::usleep / nanosleep / sched_yield return promptly,
so a future libc-binding regression on a new toolchain is caught
before it reaches a guest.
- Cargo.toml: workspace clippy config — dbg_macro / todo / unimplemented denied workspace-wide. Bug-class warn-level lints (unwrap_used, panic, indexing_slicing, integer_division) are declared but kept at allow until the existing backlog is paid down, so the existing `cargo clippy -- -D warnings` gate stays meaningful. - Makefile: e2e-shift-left aggregate target plus bisect-bug1 / bisect-bug2 reproducers (suitable for `git bisect run`) and a `coverage` helper that pulls cargo-llvm-cov on demand. - .pre-commit-config.yaml: cargo-machete hook scoped to Cargo.toml changes, auto-installs cargo-machete if missing.
build.yml stays the merge gate. lint-extra.yml runs two advisory jobs (continue-on-error) on every PR and on a nightly schedule: - clippy-strict: pedantic + nursery warnings, output-only. - coverage: cargo-llvm-cov over the workspace, soft floor on src/vmm/serial.rs, src/virtio/vsock.rs, src/vmm/agent_listener.rs (the three files most directly involved in the bug classes the shift-left e2e suite guards). A miri job belongs here too but the workspace root crate is binary-only (no [lib] target) and the leaf crates' tests use libc / KVM ioctls that miri can't model. Re-enable once pure-logic parsers (vsock packet header, virtio descriptors, kernel cmdline, identity page) are extracted into a lib crate that miri can run end-to-end.
tolgakaratas
force-pushed
the
ci/add-cicd-and-tooling
branch
from
fe94a63 to
5bd51fc
Compare
`sector * SECTOR_SIZE` could wrap silently for any sector at or above u64::MAX/512. The wrapped offset would then "pass" the `offset + len > capacity` end-of-disk check (which itself wraps) and the underlying read/write would land at an arbitrary host-file offset, either exposing host data or scribbling over it. Switch both paths to checked_mul + checked_add and return VIRTIO_BLK_S_IOERR when either step would wrap. Test cases assert that process_request with sector=u64::MAX/4 returns IOERR for both directions.
Coverage gaps exposed during a fresh audit. All of these are pure-logic unit tests that run inside `cargo test`; nothing here needs a guest VM. - src/control/protocol.rs: cover the *sync* framing path used by the per-VM control socket (the async path was already tested). Round-trip, oversize header rejection, oversize payload rejection on write, empty-stream → ConnectionClosed. - src/virtio/vsock.rs: VsockHdr::read_from refuses every short slice below HDR_SIZE; write_to / read_from round-trip preserves every field; VsockHdr::write_to short buffer returns 0; VsockConn::peer_free saturates rather than wraps when accounting drifts (otherwise a drifted counter looks like ~u32::MAX free credit and we flood the peer). - src/vmm/agent_listener.rs: AgentState::new starts disconnected; send_exec on a disconnected agent returns "not connected" instead of panicking on a None unwrap; send_shutdown on disconnected is a no-op; AgentMessage Heartbeat round-trips; serde rejects unknown message types (so a guest can't spoof an ExecResult by typoing). - src/vmm/serial.rs: every data-register write lands in the rolling history; DLAB writes do NOT pollute history (would corrupt the on-attach replay); HISTORY_CAP enforced by dropping oldest when full; constant pinned at 8 KiB. cargo test 287 passed (+21 from 266), clippy/fmt clean.
The test module already inherits std::io::Write through 'use super::*'
from the file-level 'use std::io::{Read, Seek, SeekFrom, Write}'. The
explicit 'use std::io::Write as _' inside the test module was dead and
triggered an unused-import warning under workspace clippy lints.
…SER hook When Docker is installed it sets the FORWARD chain's default policy to DROP and prepends a 'jump DOCKER-USER' before any user rules. The prior ensure_nat() appended ACCEPT rules at the bottom of FORWARD, which Docker's filtering can short-circuit before they fire (and the policy DROP catches anything that falls through), so guest egress silently broke on every Docker-host laptop. Docker explicitly leaves DOCKER-USER empty for user-owned rules. We now also insert ACCEPT rules for our subnet there when the chain exists, so coexistence does not require touching any rule Docker manages itself. On hosts without Docker the chain is absent and the insert is skipped. Refactor side-effects: - Extract iptables_check_args() pure helper that converts an '-A' invocation into the matching '-C' check; covered by 4 unit tests (FORWARD, DOCKER-USER, table-selector, multi-token cases). - ensure_iptables_rule() consolidates check-then-add into one place. - Drop the early-return on existing MASQUERADE: a present NAT rule no longer skips the FORWARD/DOCKER-USER setup, fixing a latent bug where re-running auto_setup_network never installed FORWARD rules if MASQUERADE was already there.
The previous TCP-egress assertion ran 'wget -q -O /dev/null http://8.8.8.8/' and passed when the response did NOT contain 'network is unreachable', 'Connection refused' or "can't connect". A wget timeout produces an empty response that lacks every error string, so the test silently passed on hosts where guest egress was actually broken — the very class of host where the test was supposed to detect a regression. Switch to 'nc -z -w 5 8.8.8.8 443' and check exit_code == 0 directly: nc reports a real SYN/ACK round-trip in its exit code, so timeouts and unreachable destinations now surface as honest failures. Also broaden the no-trailing-newline payload pattern in test_serial_login_prompt from 'UNSUPP|binfmt_mis|clone login:' to 'UNSUPP|binfmt_mis|login:'. The original probe required either systemd's binfmt UNSUPP truncation or Ubuntu's 'clone login:' banner and missed Alpine, which prints '(none) login: ' (no \n) — the same partial-line case the test exists to verify. The universal 'login:' suffix matches every distro's getty banner without losing the no-newline-flush guarantee that motivated the test.
…etns roadmap Section 6 (Networking) now explains why a vanilla 'iptables -A FORWARD ... ACCEPT' is insufficient on Docker hosts (Docker installs FORWARD policy DROP and a leading 'jump DOCKER-USER') and how Clone uses the DOCKER-USER chain — Docker's documented user-extension hook — to add its own ACCEPT rules without mutating any rule Docker manages itself. On hosts without Docker the chain is absent and the insert is skipped. A new entry under 'Needs Work' tracks per-VM network namespace isolation as the long-term direction. The current --net mode shares the host's main netfilter plane with everything else on the box; the roadmap mode would put each VM in its own netns joined to the host through a single veth pair, giving multi-tenant deployments a hardware-isolation-grade boundary at the network layer (compromised guest cannot observe other guests' conntrack state) and decoupling guest reachability from whatever iptables layout the operator runs on the host. Also makes teardown a single 'ip netns del' instead of a search-and-remove across host firewall tables.
Stop relying on continue-on-error to mask shift-left signal. Replace the single advisory-only lint-extra job with three jobs: * clippy-curated (BLOCKING): a small hand-picked set of pedantic / correctness lints that report zero violations in this tree right now. Hard-deny so the count cannot regress upward without an explicit fix. Promote individual lints into this set as the codebase becomes clean for them; keep the set small and 0- violation. Today: cloned_instead_of_copied, imprecise_flops, large_types_passed_by_value, lossy_float_literal, manual_assert, rest_pat_in_fully_bound_structs, unnested_or_patterns, verbose_file_reads. * clippy-strict (advisory, continue-on-error): full clippy::pedantic + clippy::nursery as warnings. Keeps the signal visible without blocking landing while we ratchet items into clippy-curated. * coverage (BLOCKING per file): cargo-llvm-cov produces lcov.info, then a small awk pass computes per-file line coverage and compares against floors checked in at .github/coverage-floor.txt. Any listed file dropping below its floor fails the job. Files not in the list are unconstrained — add a path the first time you ship meaningful tests for it. Floors only ratchet up: when a PR genuinely raises coverage, the same PR raises the floor. Initial coverage floors captured at this branch tip: src/vmm/serial.rs 67% src/virtio/vsock.rs 34% src/vmm/agent_listener.rs 17% src/control/protocol.rs 94% src/virtio/block.rs 62% The Miri job is still absent on purpose — see the comment block on clippy-curated for the binary-only-crate reason and the exit criterion (extract pure-logic parsers into a lib crate).
The current Lint extras workflow has no Miri job because the workspace root is a binary crate (no [lib] target) and the leaf crates' tests touch libc/KVM syscalls that Miri cannot model. Until that changes the experimental undefined-behaviour interpreter has nothing it can run end-to-end. The roadmap is to extract pure-logic parsers — vsock packet header parser, virtio descriptor walker, kernel cmdline parser, control- plane wire framing — into their own lib crate that imports nothing syscall-flavoured. Once isolated, a Miri job in lint-extra.yml can execute their unit tests under the interpreter and catch undefined behaviour in our unsafe blocks before it ships. Exit criterion: the Miri job blocks merges and runs the parser-lib test suite green. Tracked under 'Needs Work' so future contributors and reviewers can see the link between the missing Miri coverage and the codebase restructuring that unblocks it.
Make the build.yml result legible at a glance instead of buried in
seven separate job logs. Each job now appends a markdown block to
$GITHUB_STEP_SUMMARY, and a final aggregator job composes a
single comprehensive table and posts it to the PR as a sticky
comment under header 'ci-build'.
Per-job additions:
* fmt — pass/fail line.
* clippy — warning + error counts (parsed from cargo output).
* Build & Test — total / passed / failed / ignored / filtered tests
(summed across every test binary's 'test result' line via
grep -oP) + duration + binary size + artifact reference.
* msrv, deny, security — pass/fail line each.
Aggregator (jobs.summary):
* needs every upstream job and runs with if: always() so it fires
even on partial failure (otherwise the report disappears exactly
when it would help most).
* Walks needs.<job>.result, renders icons (✅ / ❌ / 🚫 / ⏭️),
and pulls test counts + binary size out of the build job's
outputs.
* Posts via marocchino/sticky-pull-request-comment@v2 with
header: ci-build so the same comment is updated on every push
(no comment spam).
* Mirrors aggregate status by exiting non-zero when any upstream
job is failure or cancelled — branch protection can require this
one check and still get fail propagation from the chain.
permissions: pull-requests: write added at workflow level so the
sticky-comment step can call the issues API. The token still has
read-only access to repo contents (default for the workflow).
Mirror the dashboard pattern from build.yml onto the shift-left
guards so the lint-extras signal is visible without digging into
three job logs. New aggregator posts to the PR as a sticky comment
under header 'lint-extras', distinct from the build dashboard, so
both stickies update independently each push.
Per-job additions:
* clippy-curated — violation count from grep on the deny output.
* clippy-strict — pedantic+nursery warning count (advisory).
* coverage — workspace-wide line %, plus a markdown table
(file / floor / current / Δ icon) generated inside the ratchet
step so the same numbers feed the floor enforcement and the
summary, with no duplicate parsing.
Aggregator (jobs.summary):
* needs all three upstream jobs, if: always().
* Pulls violation/warning counts and the rendered coverage table
out of upstream outputs and concatenates them into the report.
* Posts via marocchino/sticky-pull-request-comment@v2 with
header 'lint-extras' (separate sticky from ci-build).
* Gating: failure/cancelled in clippy-curated or coverage fails
the aggregator (they are merge gates); clippy-strict only
propagates 'cancelled' because its job-level continue-on-error
intentionally allows advisory warnings to land green.
permissions: pull-requests: write added at workflow level for the
sticky comment step.
Lint extras —
|
| Job | Result | Detail |
|---|---|---|
| Clippy curated (blocking) | ✅ success |
violations: 0 |
| Clippy advisory (pedantic + nursery) | ✅ success |
warnings: 2306 |
| Coverage ratchet (blocking) | ✅ success |
workspace line: 37.92% |
Workspace coverage: region 40.19% · function 53.91% · line 37.92%
Coverage by file (ratchet)
| File | Floor | Current | Δ floor |
|---|---|---|---|
src/vmm/serial.rs |
67% | 67% | ✅ 0 |
src/virtio/vsock.rs |
34% | 34% | ✅ 0 |
src/vmm/agent_listener.rs |
17% | 17% | ✅ 0 |
src/control/protocol.rs |
94% | 95% | ✅ +1 |
src/virtio/block.rs |
62% | 62% | ✅ 0 |
Bottom 10 files by line coverage
| File | Line % |
|---|---|
boot/mod.rs |
0.00% |
compat.rs |
0.00% |
control/daemon.rs |
0.00% |
control/jailer.rs |
0.00% |
control/mod.rs |
0.00% |
control/sync_server.rs |
0.00% |
main.rs |
0.00% |
memory/mod.rs |
0.00% |
memory/overcommit.rs |
0.00% |
pci/vfio.rs |
0.00% |
Top 10 advisory clippy lint categories
| Lint | Count |
|---|---|
clippy::cast_possible_truncation |
571 |
clippy::doc_markdown |
349 |
clippy::missing_const_for_fn |
171 |
clippy::ptr_as_ptr |
153 |
clippy::cast_lossless |
127 |
clippy::cast_ptr_alignment |
115 |
clippy::manual_let_else |
76 |
clippy::unreadable_literal |
68 |
clippy::cast_possible_wrap |
68 |
clippy::borrow_as_ptr |
66 |
Coverage artifact: coverage-report
Base-branch lcov.info unavailable for this run; Δ-base column omitted (no successful master Lint extras run with metrics yet).
Updated by lint-extra.yml · sticky-comment header lint-extras.
CI · Build & Test —
|
| Job | Result | Duration |
|---|---|---|
| Detect changes | ✅ success |
5s |
| Format | ✅ success |
14s |
| Clippy | ✅ success |
23s |
| Build & Test | ✅ success |
1m46s |
| MSRV check | ✅ success |
16s |
| Dependency check | ✅ success |
19s |
| Security audit | ✅ success |
3m2s |
| DCO check | ✅ success |
5s |
| Container image | ✅ success |
27s |
Tests: 291 passed · 0 failed · 0 ignored · 0 filtered out (in 0.489s)
Per-binary breakdown
| Binary | Tests |
|---|---|
clone::bin/clone |
291 |
Top 10 slowest tests
| Duration | Binary | Test |
|---|---|---|
0.133s |
clone::bin/clone |
virtio::block::tests::test_flush_operation |
0.097s |
clone::bin/clone |
storage::qcow2::tests::test_different_cluster_sizes |
0.082s |
clone::bin/clone |
virtio::fs::tests::test_inode_map_handles |
0.077s |
clone::bin/clone |
virtio::fs::tests::test_metadata_to_fuse_attr |
0.061s |
clone::bin/clone |
control::protocol::tests::test_sync_write_rejects_oversize_payload |
0.032s |
clone::bin/clone |
virtio::block::tests::test_queue_max_sizes |
0.030s |
clone::bin/clone |
virtio::block::tests::test_read_rejects_sector_offset_overflow |
0.030s |
clone::bin/clone |
virtio::block::tests::test_read_past_end_of_disk |
0.026s |
clone::bin/clone |
storage::qcow2::tests::test_write_and_read_back |
0.024s |
clone::bin/clone |
storage::qcow2::tests::test_write_at_offset |
Binary: target/x86_64-unknown-linux-musl/release/clone — 2.32MB (download artifact)
Compile warnings: 1 (see Build & Test job log)
SBOM: 86 packages (CycloneDX 1.5 + SPDX 2.3 in artifact sbom)
DCO check (advisory)
39 commit(s) missing a matching Signed-off-by: trailer:
cac8823Merge be47418 into 4d4692dbe47418ci: package x86_64 binary as distroless container, push on master4d4aa09ci: drop aarch64 target until cross-arch refactor lands1e0badcci: add aarch64-musl advisory build to catch cross-build breakage25bed77ci: add SBOM (CycloneDX + SPDX) and Sigstore build provenancedc371f4ci: add toolchain row, billing minutes, JUnit, DCO, warnings, fail trace1a13421ci: relax commitlint length+case to warnings; keep type-enum strictd7f9996ci: fix commitlint base-ref fetch (drop invalid --depth=0)59fde07ci: add CODEOWNERS, conventional-commit gate, markdownlint, path labels03f9efeci: fix nextest output parser for progress-counter token4725220ci: add Top-N slowest tests + Mermaid CI flow to dashboard5a0e00aci(dashboard): backtick-safe code wrapping + CRLF tolerance in lockdiff5681a4eci(dashboard): per-job timing, cache hit, per-binary breakdown, audit IDs, lockfile diff, clippy categories, bottom-10 coverage5831e53ci(dashboard): add base-branch deltas to PR sticky comments5b2bc74ci: pin rustsec/audit-check to Node-24 master HEAD; bump CodeQL to v42e8d82eci: fix YAML expression parse error + bump actions to current latesta4d94b8ci(lint-extra): bracket-notation for hyphenated job IDs in needs.* refsdb28cefci: fix dashboard data — strip ANSI color, usein coverage table5424c71ci(lint-extra): per-job step summaries + aggregator + sticky PR commentb5bea7aci(build): per-job step summaries + aggregator + sticky PR comment3363c02docs(spec): add Miri-runnable parser library to roadmap6ef2976ci(lint-extra): hard-block curated clippy + per-file coverage ratchetfc7c4e6docs(spec): document Docker-coexistence netfilter strategy + per-VM netns roadmap89e6beftest(e2e): honest TCP probe + login pattern works across distrosd85ed3efeat(net): coexist with Docker by inserting clone rules into DOCKER-USER hookf1e49f1chore(test): drop unused Write import in virtio block testsfa82d85test: extend unit coverage on bug-class hot paths7acafadfix(block): reject sector overflow in do_read/do_write5bd51fcci: add advisory clippy-strict + coverage workflow1c4f8c8chore: workspace clippy lints, machete hook, shift-left make targets30ffa97test: add shift-left regression guards for vsock + serial + timerfec217afix(timer): re-enable kvmclock + tsc-deadline on fresh vCPUsddecd5cfix(serial): tee guest output per-byte + replay on attachf62c673fix(vsock): unblock guest agent loop after PID 1 forkf4c9ef8chore: add development tooling and release configuration1bc4b37docs: add project documentation and README badgesd0ec713ci: add GitHub Actions CI/CD with release automation72e8325build: optimize release profile and project metadata9c467c6fix: code quality, musl compatibility, and cross-platform module gating
Sign off new commits with git commit -s, or amend with
git commit --amend --signoff / git rebase --signoff origin/master.
Cargo.lock diff (vs base)
Upgraded (5):
bitflags: 2.11.0 → 1.3.2hashbrown: 0.16.1 → 0.15.5thiserror: 2.0.18 → 1.0.69thiserror-impl: 2.0.18 → 1.0.69vm-memory: 0.17.1 → 0.16.2
Base-branch metrics unavailable for this run (no successful master CI run with metrics yet); deltas omitted.
Updated by build.yml · sticky-comment header ci-build.
The first dashboard run revealed three rendering bugs in the sticky
comments:
1. Coverage 'File' column rendered empty. The ratchet step writes
the table with Markdown backticks around the path, then exposes
it as a step output and the aggregator inlines it via the
${{ … }} expression. GitHub renders the value verbatim into the
shell run block, so bash re-evaluates the backticks as command
substitution and the cell becomes blank. Use <code>…</code> tags
instead — same monospace rendering, no shell-side parsing.
2. Clippy advisory pedantic 'warnings: 0'. The workflow sets
CARGO_TERM_COLOR=always, which wraps every 'warning:' marker in
ANSI escape codes; the 'grep -cE "^warning: "' counter then
matches zero lines. Pass --color never to that one cargo
clippy invocation so its log stays plain text.
3. Workspace 'line: 0'. The 'cargo llvm-cov report --summary-only'
TOTAL row holds twelve numeric columns; the previous
'awk {print $(NF-1)}' picked the second-to-last (missed
branches), not line coverage. Replace with grep over the row's
'%' tokens and pick the third match (Lines Cover%).
Same ANSI-escape problem applies to two more steps in build.yml:
* cargo clippy --D warnings → 'warning'/'error' counter would
have miscounted on the first failure.
* cargo test → libtest's 'test result: ok. N passed;' summary has
'ok' colored, so the counter grep would have lost the prefix.
Both get CARGO_TERM_COLOR=never set per-step. The workflow-level
'always' stays for jobs whose logs benefit from ANSI in the GitHub
log viewer.
The aggregator's references to 'needs.clippy-curated.*' and 'needs.clippy-strict.*' parsed as the subtraction expression (needs.clippy MINUS curated.result), so GitHub Actions rejected the workflow at validation time and reported the run as failure with a zero-job listing and the workflow path used in place of the 'name:' header. Switch to bracket notation (`needs['clippy-curated'].result`, `needs['clippy-strict'].outputs.warnings`, …) which is the documented form for accessing job IDs that contain dashes.
Two issues, fixed together because they were both surfaced by the
first dashboard run failing at workflow-validation time:
1. Unicode ellipsis (U+2026) inside a literal '${{ … }}' written in
a bash comment in lint-extra.yml. GHA's expression engine scans
the entire 'run: |' block for '${{ ... }}' patterns regardless
of bash comment syntax, so the engine tried to parse '…' as
part of the expression and rejected the workflow with:
got unexpected character '…' while lexing expression
The whole 'Lint extras' run came back as 'failure' with a
zero-job listing and the file path used in place of the
workflow's name. Fix: drop the literal '${{ … }}' from the
comment, talk about it via plain prose, and replace any
remaining 'N passed; …' / 'warning: …' Unicode horizontal
ellipses with ASCII '...'. Same fix applied to two comments in
build.yml. Verified with actionlint v1.7.7 — both files clean.
2. The CI runs were warning that Node.js 20 actions are deprecated
('actions/checkout@v4', 'actions/upload-artifact@v4', etc.) and
will be force-migrated to Node.js 24 by 2026-06-02. Bump every
referenced action to the current latest major:
actions/checkout v4 -> v6
actions/upload-artifact v4 -> v7
actions/download-artifact v4 -> v8
dorny/paths-filter v3 -> v4
marocchino/sticky-pull-request-comment v2 -> v3
dependabot/fetch-metadata v2 -> v3
googleapis/release-please-action v4 -> v5
sigstore/cosign-installer v3 -> v4
actions/attest-build-provenance v2 -> v4
softprops/action-gh-release v2 -> v3
The following are already on their current latest major and
stay floating: dtolnay/rust-toolchain, Swatinem/rust-cache@v2,
EmbarkStudios/cargo-deny-action@v2, rustsec/audit-check@v2,
taiki-e/install-action@v2, anchore/sbom-action@v0,
github/codeql-action@v3. Drop the workflow-level
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 escape hatch — once every
action ships its own Node.js 24 release the override stops
being needed.
GitHub still warned 'Node.js 20 actions are deprecated' for two
actions whose v-major refs lag behind their actual Node 24 work:
* rustsec/audit-check@v2 — upstream merged the Node 24 bump
(rustsec/audit-check#48, master commit 858dc40f, 2026-03-20)
but has not cut a v2.x.x release yet, so the v2 floating ref
still points at v2.0.0 (2024-09-23) which ships Node 20.
Pin to the master commit SHA. The pin is documented inline so
the next maintainer (or dependabot) knows to drop it once a
real release lands. Same pin in both build.yml and
security-scan.yml.
* github/codeql-action — v3.x.x is the legacy Node 20 series;
the maintainer publishes v4.x.x in parallel and the v4 line
runs on Node 24 (verified against init/action.yml on tag
v4.35.4). Bump init/autobuild/analyze from @V3 to @v4 in
security-scan.yml.
Verified with actionlint v1.7.7 — both files lint clean. After
this push the only remaining 'Node 20' warnings would be from
actions whose latest tag itself ships Node 20 (none in our tree
as of 2026-05-07).
Reviewers reading the sticky comments now see what the PR moves
relative to the base branch instead of just the absolute numbers
for the current run:
CI · Build & Test:
Tests: 291 passed (+4 vs base) ...
Binary: ... 2.32MB (+8.0KB vs base)
Lint extras coverage table gains a delta-vs-base column:
| File | Floor | Current | dF | dBase |
| src/vmm/serial.rs | 67% | 72% | +5 | up +5 |
Mechanism (same shape in both workflows):
1. The producer job now uploads a stable artifact with the
metrics that need to survive across runs:
* build.yml writes /tmp/metrics-build.json (tests + binary
size) and uploads it as metrics-build (30-day retention,
outliving the 7-day binary artifact).
* lint-extra.yml's coverage job already uploads lcov.info as
coverage-report; the only change is to bump retention
from 14 to 30 days so the base-fetch window is long
enough.
2. The aggregator/ratchet job adds a Fetch-base-... step that
uses the GITHUB_TOKEN to call the workflow API: it locates
the most recent successful run of the same workflow on the
PR's base ref, looks up the right artifact by name, downloads
the zip via /actions/artifacts/<id>/zip, and unpacks it into
/tmp.
3. The body-composing step compares current vs base values:
* tests_passed and binary_size for build.yml (signed integer
delta, binary size formatted via numfmt --to=iec).
* Per-file line coverage for lint-extra.yml — the existing
awk that drives the floor ratchet runs a second pass on
the base lcov.info, and the table grows from four to
five columns when base is available.
The whole thing is best-effort: any error in the fetch path
(available=false, missing artifact, malformed zip, etc.) leaves
the values empty and the body gracefully omits the deltas with an
explicit Base-branch-metrics-unavailable note. Push events to
master skip the fetch outright (no base to compare against).
Verified with actionlint v1.7.7 — both files lint clean.
… IDs, lockfile diff, clippy categories, bottom-10 coverage
Eight new dimensions on the sticky comments and step summaries —
the existing dashboard told reviewers WHAT failed, this one tells
them WHY and at what cost.
build.yml additions:
* Per-job duration column. The aggregator's new 'Fetch run timing'
step calls the workflow API for the current run, computes
completed_at - started_at per job, and renders a third column
next to result.
* Cache hit / miss indicator next to the Run line. Captured from
the Swatinem/rust-cache step's cache-hit output and surfaced as
a cache: hit / miss / dash line in the sticky header.
* Per-binary test breakdown. The Build & Test job's existing test
log gets a second awk pass that walks 'Running unittests ...'
blocks, pairs each binary path with its 'running N tests'
count, and emits a tsv. The aggregator embeds it as a
collapsible details block (clone, clone-init, clone-agent,
sleep_deadline integration target).
* Security advisory list. The Security audit job now runs
'cargo audit --json' alongside rustsec/audit-check and parses
the vulnerability list with jq. When advisory_count > 0 the
sticky grows a 'Security advisories' table with RUSTSEC IDs
linked to rustsec.org, package@version, and the title.
* Cargo.lock diff vs base. New 'Fetch base Cargo.lock for diff'
step downloads both base and current Cargo.lock via the contents
API, extracts package@version pairs with awk, computes added /
removed / upgraded sets via comm + a name-keyed awk join, caps
at 20 rows per section (with overflow tail), and embeds them in
the sticky as Markdown bullet lists.
lint-extra.yml additions:
* Workspace coverage breakdown. The Workspace summary step now
captures all three TOTAL-row percentages (region / function /
line) and exposes them as outputs so the aggregator can show
region X / function Y / line Z instead of just the line value.
* Bottom 10 files by line coverage. Same step extracts the 10
worst-covered files from the per-file rows of cargo llvm-cov
report --summary-only, sorted ascending by the third percent
column. Embedded as a collapsible details block.
* Clippy advisory category breakdown. The pedantic+nursery step
runs with --message-format=json so jq can pull every diagnostic
that has a clippy code, sort uniq -c the codes, and feed the
top 10 into a Lint / Count table embedded in the sticky.
The whole thing is best-effort: any single API or jq path that
fails leaves the corresponding section out of the body, the sticky
still upserts, and reviewers do not see a broken comment. Verified
locally with actionlint v1.7.7 (both files clean) and pre-commit
hooks.
The first run of the enriched dashboard exposed two render bugs: 1. Cargo.lock diff cells rendered as empty 'Upgraded: : -> :' rows. The awk that lays the table out used Markdown backticks, and the table_md output then went back through the dollar-double-braces substitution into a shell run block — bash re-evaluated the backticks as command substitutions and stripped every name. Same pattern as the earlier coverage table fix: switch to <code>...</code> tags. 2. The same backtick problem applied to the new Top-10 advisory clippy categories table in lint-extra.yml. Switched to <code> too, with an inline comment so the next maintainer does not reintroduce it. While we were touching the lockfile path, harden the awk extractor against CRLF line endings: gsub /[\r"]/ instead of just /"/, and filter the API-downloaded files through tr -d '\r'. The GitHub contents API has been observed shipping base64 that decodes to CRLF on some shapes, and an unstripped \r leaks into the sort/comm key and silently turns one logical package into two distinct rows. Verified locally with actionlint v1.7.7 and the awk-pipeline run against a synthetic upgraded base/cur Cargo.lock pair.
Closes the last two of the eight dashboard dimensions selected on the prior pass: * Top-10 slowest tests — switch the Build & Test job from `cargo test` to `cargo nextest run --status-level all`. nextest's one-line-per-test output gives us per-test wall-clock timing in the same log we already parse for per-binary counts and pass/fail/skip totals, so we can extract the slowest ten and surface them as a collapsible table inside the sticky CI comment alongside the existing per-binary breakdown. Total/passed/failed parsing is rewritten against nextest's "Summary [...] N tests run: P passed, F failed, S skipped" line; the existing job outputs (tests_total, tests_passed, etc.) keep their old keys so the metrics-build.json artifact and the base-vs-current delta rendering continue to work without touching the aggregator. * Mermaid CI flow diagram — append a `graph TD` Mermaid block to the summary job's step summary tab. The seven upstream jobs are styled by their actual run result (green / red / yellow), giving a one-glance health view of the build.yml DAG. Mermaid renders in $GITHUB_STEP_SUMMARY but is silently stripped from PR comments, so the diagram only goes to the run page; the sticky comment is unchanged. cargo-nextest is installed via taiki-e/install-action@v2 (prebuilt binary, no compile-time hit). MSRV/clippy/deny/audit jobs and local `make test` are not affected — they still use cargo test.
The previous parse step assumed the test line layout was
STATUS [<spaces>Ts] <binary> <test_name>
but `cargo nextest run --status-level all` actually emits
STATUS [ T.TTTs] ( N/M) <binary> <test_name>
with whitespace inside both bracketed and parenthesised tokens.
With awk's default field split this turned the leading "(" into
its own field, so the per-binary aggregator counted progress
markers like "(100/291)" instead of binary IDs and the dashboard
sticky rendered ~280 single-row binary entries.
Normalise inner whitespace in [...] and (...) tokens with sed
before the awk split, then derive the per-binary breakdown and
top-10 slowest tests from a single intermediate TSV. Verified
against a synthetic sample log with the actual format observed
on run 25521291453.
Bundle of repository-hygiene gates that previously only ran in reviewers' heads: * CODEOWNERS — every path defaults to @tolgakaratas + @realrasengan so GitHub auto-requests review and "Require review from Code Owners" branch protection has someone to gate on. * PR template polish — point reviewers at the CI · Build & Test sticky comment for per-job timing / slowest tests / Cargo.lock diff, link the Mermaid CI flow graph in the run summary tab, and surface the conventional-commits header convention. * Conventional Commits gate — `.commitlintrc.json` extends config-conventional with a fixed type-enum, kebab-case scope, and 100-char header. CI enforces it via a new "Lint meta / Commit messages" job that runs commitlint over the PR's commit range under explicit Node 24. compilerla/conventional-pre-commit mirrors the gate on `commit-msg` stage locally so non-conforming messages fail pre-push instead of pre-CI. * Markdownlint — `.markdownlint.json` enforces structural rules (heading levels, list/table consistency, fenced-code language) while turning off layout nits incompatible with this repo's documentation conventions (long lines, inline HTML, bare URLs, no leading H1 on index pages). Scope is `docs/**/*.md` plus root-level Markdown only — `.agents/`, `.claude/`, and target caches are out of scope. CI runs markdownlint-cli2 in a new "Lint meta / Markdown" job; pre-commit mirrors it. SPEC.md is cleaned up to satisfy MD036 (emphasis-as-heading) so the gate starts from a green baseline. * Path-based PR labels — actions/labeler@v6 with a config in `.github/labeler.yml` covering area:ci, docs, vmm, virtio, net, storage, tests, e2e, deps, build. The workflow's first step provisions the label set via `gh label create --force`, so the labels never need manual setup before the action starts firing. All four CI workflows (Lint meta, Labeler) pass actionlint; YAML configs validate via PyYAML; markdownlint reports zero errors across the configured scope.
git fetch rejects --depth=0 with "depth 0 is not a positive number". The flag was redundant anyway: actions/checkout@v6 above already runs with fetch-depth: 0, which clones the full history; we only need the base ref to be present locally so commitlint can resolve the "from" symbol.
The hard 100-char cap and kebab-case scope rule rejected existing
commits on this branch (one 127-char header, one (e2e) scope that
@commitlint/ensure's kebab-case detector treats as non-kebab
because of the digit). Both are stylistic, not structural, so
demote them:
* header-max-length stays at 100 but level=1 (warn, not fail).
* scope-case disabled (level=0).
* subject-case kept as warning so all-uppercase first words
surface but don't block the merge.
type-enum stays strict (level=2) — that's the rule that actually
enforces the conventional-commits contract. Long-term we can
rewrite legacy commits and bring the cap back to a hard error,
but blocking the open PR over historical formatting is the wrong
trade.
Six dashboard polish dimensions, all previously identified as gaps in the sticky comment. Bundled because each is a small slice that touches the same aggregator and the metrics flow naturally together. * Toolchain row — capture rustc / cargo / nextest versions on the Build & Test job once the runtime is fully provisioned. Render a one-line "Toolchain:" header in the sticky so reviewers can spot silent stable-channel bumps that change behaviour between runs. * Billing-style runtime — sum every per-job duration rounded up to the nearest minute (GHA's billing unit). Surface the total beside the cache-hit indicator so cost regressions surface during review. We can't use the timing API directly because it returns empty while the run is still in progress (the aggregator job IS the run). * JUnit annotations — .config/nextest.toml configures the default profile to write target/nextest/default/junit.xml. The new EnricoMi/publish-unit-test-result-action step turns that report into a "Test results (nextest)" check plus inline file:line annotations on the diff for any failing test. We already have a rich sticky comment, so comment_mode/job_summary are off — the action only contributes the check + annotations. * DCO advisory check — new `dco` job walks every commit on the PR branch and verifies a Signed-off-by trailer matching the commit author's email. Failures are reported via job output and the aggregator renders a "DCO check (advisory)" markdown block listing short SHAs + subjects of unsigned commits. Job exits 0 regardless so it never blocks the PR — branch protection can promote it to required later. The Mermaid CI flow graph in the run summary tab picks up the new node automatically. * Compile warnings count — tee the cargo build log, count rustc "warning:" markers, expose as a job output, and render a single "Compile warnings: N" line under the binary row when N > 0. Complements the existing clippy counter (which only covers lint warnings, not full rustc warnings). * Failed-test stack-trace block — capture nextest's "failures:" section to /tmp/fail_trace.txt (capped at 200 lines so a giant panic dump doesn't blow past the comment limit), expose via output, and render inside a collapsible <details> block when Build & Test fails. Stack traces and STDOUT captures land on the PR comment instead of forcing a click into the run log. The aggregator job's `needs:` array now includes `dco` and the final gate stays unchanged — DCO failure is intentionally not a hard fail. Mermaid graph adds one node and two edges. All four workflows pass actionlint; nextest.toml validates as TOML.
Per-build supply-chain artefacts so every produced binary has a verifiable record of its inputs and origin: * SBOM — cargo-sbom emits both a CycloneDX 1.5 JSON and an SPDX 2.3 JSON describing every workspace dependency at the exact Cargo.lock version that produced the binary. Both files upload as the `sbom` artifact (90-day retention) and a package count surfaces in the sticky as a one-line "SBOM: N packages" row, so dependency-footprint changes are visible in PR review. * Sigstore build provenance — actions/attest-build-provenance@v3 signs the binary and pushes a DSSE/in-toto statement to the transparency log, tying it to this commit, this workflow file path, and the runner identity. The attestation URL renders as a link in the sticky and can later be verified with `gh attestation verify`. Skipped on pull_request (PR builds are throwaway) but runs on every push to master and on every tag, including release builds. Adds id-token + attestations write scopes to the workflow permissions block.
Release builds already cross-compile to aarch64-unknown-linux-musl, but the failure mode is "noticed at tag time" — by which point master may already have months of arm-incompatible changes queued. Add a parallel advisory build job that compiles the same target on every PR: * `build-aarch64` job runs alongside Build & Test, gated on the same code-paths filter so docs-only PRs skip it. Uses cargo-zigbuild + Zig as the cross-linker (no Docker dependency, meaningfully lighter than `cross` for a single target). continue-on-error: true keeps the job non-blocking — a regression surfaces in the PR sticky and CI flow graph but doesn't gate the merge. Branch protection can promote it later once the cross build is shown to be stable across this workspace. * The aggregator picks up the new job's outcome and (when it succeeded) the resulting binary size, renders both as a new "Build aarch64" row beside the x86_64 row, and adds the node to the Mermaid CI flow graph in the run summary tab. Missing / skipped runs (docs-only PRs) collapse out of the table cleanly. * Aarch64 binary uploads as a separate `clone-linux-aarch64` artifact so reviewers can pull it directly from PR runs without waiting for a release tag.
Adding the aarch64-unknown-linux-musl advisory build job exposed that the cross-compile produces ~114 errors against kvm-ioctls' arm-only API surface, and that release.yml's matrix has been silently broken for that target the whole time. The codebase calls x86-only KVM ioctls (set_pit2, set_irqchip, kvmclock_ctrl, get_pit2, get_irqchip, get_clock) without `cfg(target_arch)` gating, so the binary genuinely cannot build for aarch64 today. Rather than ship a CI job that always fails and a release matrix that produces no aarch64 artefact, this: * Removes the build-aarch64 job from build.yml (and the row / Mermaid node / timing lookup that fed it in the aggregator). * Drops aarch64-unknown-linux-musl from release.yml's build matrix and from the cosign / attest-build-provenance / release artefact lists. x86_64-musl stays the sole release target. * Documents the gap in docs/SPEC.md "Roadmap — aarch64 support" with the concrete refactor needed: arch-gate every x86-only ioctl, wire up GICv2/v3, switch to the arm generic timer, revisit the boot path for the arm boot protocol. * Disables markdownlint MD028 (blank-line-inside-blockquote) because the two consecutive ">" roadmap blocks read as one blockquote with a blank line under that rule's heuristic. Re-add aarch64 to both workflows once the cross-arch refactor ships. The honest position right now is that this is an x86_64 VMM, and CI / release tooling should reflect that.
Adds a `package` job that turns each green Build & Test run into a container image. The image is "distroless"-style — just the static musl binary on top of FROM scratch — so there is no userland to maintain alongside the binary and the image size effectively matches the binary size (~2.3MB). Wiring: * `Dockerfile` lives at the repository root and expects the binary at the build context root as `clone`. The CI job downloads the `clone-linux-x86_64` artifact produced by the Build & Test job into ./pkg, then runs `docker buildx build` with that as the context. The Dockerfile's COPY pulls the binary in directly. * On `pull_request` the job only validates the Dockerfile (push: false) so reviewers see a green check confirming the image builds, without consuming registry storage. On `push` to master the job authenticates against ghcr.io with the workflow token and publishes both `:latest` and `:sha-<short>` tags through docker/metadata-action. Forks and feature-branch pushes never push because metadata-action gates the latest tag behind `is_default_branch`. * The aggregator picks up the new job's outcome, renders a "Container image" row in the PR sticky's job table, adds the node to the Mermaid CI flow graph, and threads the timing into the per-job duration lookup. The existing fail-trace and binary-row blocks are unchanged. * Image labels carry OCI metadata (source, description, license, documentation) so it shows up correctly on the GHCR package page and in `docker inspect`. Operators running the published image need `--device /dev/kvm`, `--cap-add NET_ADMIN`, and `--device /dev/net/tun` for the VMM to open KVM and to manage the TAP device — documented in the Dockerfile header.
Author
|
Closing in favor of an upstream-targeted PR. Branches are preserved on Denomas/clone and will be re-opened against unixshells/clone:master. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Production-grade CI/CD pipeline, release automation, and code quality infrastructure for Clone VMM.
5 commits, logically grouped:
Key decisions
is_multiple_of()anddiv_ceil()stdlib methodsmake ciruns full local CI before push; pre-commit hooks enforceVerified
clone --helpruns on production server (static linked)Test plan