Skip to content

refactor: harden observed capability evidence and enforcement - #47

Merged
rcyw merged 8 commits into
mainfrom
refactor/capability-governance
Aug 10, 2026
Merged

refactor: harden observed capability evidence and enforcement#47
rcyw merged 8 commits into
mainfrom
refactor/capability-governance

Conversation

@rcyw

@rcyw rcyw commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the post-PR refactor: implement observed capability providers #45 capability contract with evidence whose identity and freshness are orthogonal.
    • Model config, boot, mount, runtime, and derived identity explicitly.
    • Persist observation proofs, selected admission proofs, dependency proofs, and bounded diagnostics.
  • Make capability publication provider-owned and time-correct.
    • Schedule fast health probes independently from serialized runtime conformance.
    • Expire stale successes to UNKNOWN and require debounced recovery.
  • Enforce allocation capabilities at placement, create, and runtime.
    • Derive requirements independently in controld and axnoded.
    • Verify actual cgroup, project-quota, OverlayFS, runsc envelope, process, and mount enforcement.
    • Serialize durable fail-stop cleanup without coupling conditions to lifecycle status.
  • Normalize controld persistence and operator diagnostics.
    • Index allocation dependencies by node and capability key.
    • Store conditions and reconcile generations in dedicated tables.
    • Add typed CLI views for snapshots, transitions, backlog, and allocation evidence.

Architecture

  • Observation, policy, and enforcement remain separate layers.
    • Atomic snapshot publication does not imply simultaneous sampling.
    • Internal facts cannot be supplied as workload requirements.
    • Node-level loss triggers allocation-specific verification instead of bulk deletion.
  • The replacement contract is intentionally incompatible.
    • No legacy proto fields, JSON blobs, capability strings, boot markers, or fallback aliases remain.
    • The database schema is merged into 000001_initial.sql and expects a rebuild.
  • Rootfs enforcement is representation-aware without making image formats public workload types.
    • Effective OverlayFS lower chains carry mount identity.
    • EROFS requirements are derived from actual backing facts after the image lease is acquired.

Validation

  • Contract and generated surfaces:
    • make proto-generated-check
    • make -C sdk/proto lint
    • Go, Python, and TypeScript SDK verification
  • Targeted correctness:
    • race-enabled shared catalog, provider manager, rootfs, allocation, placement, and capability tests
    • real PostgreSQL capability schema, admission, transition, queue, and condition-report tests
    • controld and axnoded test, vet, and architecture checks
  • Linux and end-to-end:
    • axnoded Docker verification for runc, runsc, cgroup, XFS, EROFS, OCI, Nydus, inventory, retention, locality, and warm pool
    • make axern-cn-verify
    • make axern-cn-verify-all with all 33 serial repository checks passing

Tracking

rcyw added 8 commits August 10, 2026 04:39
@rcyw
refactor: redesign observed capability contracts
8a5e30f 
- Separate capability identity from freshness.
  - Model config, boot, mount, runtime, and derived evidence explicitly.
  - Persist observation and dependency proofs with bounded validation.
- Centralize capability policy and requirement derivation.
  - Validate provider ownership, audience boundaries, and dependency graphs.
  - Share transition and eligibility semantics across node and control plane.
- Replace public and private protobuf surfaces without compatibility fields.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
refactor: schedule capability providers independently
b4266d6 
- Publish atomic observation batches on provider-specific schedules.
  - Preserve actual sample times and expire stale successes to unknown.
  - Isolate slow runtime conformance from fast network and mount health probes.
- Consolidate host facts and evidence identities.
  - Bind config, boot, mount, and runtime facts to explicit identities.
  - Cache runtime digests by file identity and separate network from extension configuration.
- Recompute node readiness from current provider generations with recovery debounce.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
refactor: enforce allocation capabilities on the node
706f7a4 
- Derive and revalidate allocation requirements at request and rootfs gates.
  - Reject mismatched or internal dependencies before side effects.
  - Rebind current proofs before runtime activation and persist admitted evidence.
- Verify real runtime enforcement.
  - Check cgroup membership and memory.max for runc and runsc host processes.
  - Verify project quota, overlay envelopes, process identity, and filestore mounts.
- Serialize durable capability-loss reconciliation.
  - Keep lifecycle conditions separate from allocation state.
  - Coalesce generations and run one detached fail-stop cleanup workflow per allocation.
- Replace speculative execution envelopes with prepared container activation and durable exit state.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
refactor: persist capability admission atomically
54497d5 
- Re-evaluate typed capability eligibility while candidate nodes are locked.
  - Persist placement proofs with allocation and resource reservations.
  - Fence stale summaries and continue to the next eligible candidate.
- Normalize capability dependencies, conditions, and reconcile work.
  - Index dependencies by node and capability key instead of scanning allocation JSON.
  - Separate condition revisions from lifecycle state and queue pending keys by generation.
- Make node reports transactional.
  - Commit summary, transitions, and affected allocation work together.
  - Publish in-memory registry changes only after PostgreSQL commit.
- Merge the replacement schema into the initial migration for database rebuilds.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
refactor: expose capability governance diagnostics
095813f 
- Add typed admin commands for snapshots, transitions, backlog, and allocation evidence.
  - Render provider identity, freshness, proof graphs, conditions, and reconcile state.
  - Keep list output bounded while preserving complete detail views.
- Align verification and operator documentation with the replacement architecture.
  - Document observation, policy, enforcement, and lifecycle ownership boundaries.
  - Remove obsolete execution-envelope prewarm paths and stale compatibility guidance.
- Exercise rootfs lower chains and capability observations in runtime verification helpers.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
fix: allow staged Helm replica overrides
29ead28 
Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
fix: keep runtime recovery fail closed
2976ba0 
Require every configured runtime handler to load before persistent inventory reconciliation can start.

Retry transient initialization failures under the service context, reject malformed factory results, and preserve race-safe recovery tests and documentation.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
refactor: compose Helm upgrade overrides
f3c390e 
Keep staged and caller-provided Helm upgrade arguments on one continuation line so empty optional values do not leave misleading shell continuations.

Signed-off-by: wayne <rcywjo@gmail.com>
@rcyw
rcyw merged commit 4f3b574 into main Aug 10, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

proposal: harden observed capability evidence freshness and allocation enforcement

1 participant

@rcyw