ACT: Automated Configuration Testing

A validation gate for Pulumi infrastructure-as-code. ACT intercepts resource declarations before they hit the cloud, checks them against security rules, exercises them under fuzz/property variations, and verifies they deploy reproducibly across a fleet of CPU and accelerator targets.

ACT is designed to run as a CI/CD gate: every commit is validated; bad programs exit non-zero and block deployment.

Quick start

# 1. Clone + install
git clone https://github.com/HIRO-MicroDataCenters-BV/act.git
cd act
git submodule update --init --recursive
uv sync

# 2. Validate a sample program
uv run python -m act.run \
    --program tests/fixtures/cape/path_a_valid.py \
    --schema tests/fixtures/cape/schema.json

# Expected:
# PASS: 2 resources captured, 0 violations
# (exit 0)

Try the negative case:

uv run python -m act.run \
    --program tests/fixtures/cape/path_a_invalid.py \
    --schema tests/fixtures/cape/schema.json

# Expected:
# FAIL: 1 resource captured, 1 violation:
#   my-instance (cape:compute:Instance): ssh_keys set without security_group_ref
# (exit 1)

That's the gate. Wire the same invocation into CI and bad commits stop at the gate instead of at the cloud.

What ACT does

For every Pulumi program you run through it, ACT does up to four things:

Captures the plan without provisioning. It hooks the Pulumi SDK and records what resources would be created and what inputs they carry. Never calls a real cloud API.
Checks structural rules. Security and policy violations (missing security groups, exposed credentials, wrong arch labels, weak random passwords, etc.) surface as Violation objects with severity and a recommendation.
Fuzzes parameterised programs. When the program takes inputs, ACT mutates them (atheris fuzz + hypothesis property tests) to find configurations that pass the type checker but break the policy.
Verifies reproducibility. Optional: re-runs the program against an ephemeral cluster (k3s in Docker) twice and confirms the deployed state hashes identically. Covers amd64, arm64, riscv64, GPU, FPGA, and CXL targets.

Architecture and flow

flowchart TB
    PROG["Pulumi program<br/>(my_program.py)"]
    SCHEMA["Provider schema<br/>(schema.json)"]

    subgraph PIPE[ACT pipeline]
        MOCK["MockGenerator<br/>capture plan without<br/>calling the cloud"]
        ORACLE["Correctness Oracle<br/>structural rule checks"]
        FUZZ["Fuzz + Property<br/>(parameterised programs only)"]
        ACV["Cognitive Validator<br/>(optional, LLM-based)"]
    end

    subgraph REPRO[Reproducibility checks]
        PLAN["Plan determinism<br/>always on"]
        ARCH["Image arch readiness<br/>--check-deployment-arch"]
        RUN["Runtime twice-and-hash<br/>--check-deployment-runtime"]
    end

    GATE["CI Gate<br/>exit 0 / 1 / 2"]
    REPORT["PASS/FAIL report<br/>(stdout)"]
    JSON["Structured artefact<br/>(--output JSON)"]

    PROG --> MOCK
    SCHEMA --> MOCK
    MOCK --> ORACLE
    ORACLE --> FUZZ
    FUZZ --> ACV
    ACV --> GATE

    PROG -.-> PLAN
    PROG -.-> ARCH
    PROG -.-> RUN
    PLAN --> GATE
    ARCH --> GATE
    RUN --> GATE

    GATE --> REPORT
    GATE --> JSON

    classDef input fill:#fff4d6,stroke:#a35200
    classDef core fill:#e6f2ff,stroke:#1a5fb4
    classDef repro fill:#e6ffe6,stroke:#1f6e1f
    classDef out fill:#f4e6ff,stroke:#6f1a99
    class PROG,SCHEMA input
    class MOCK,ORACLE,FUZZ,ACV core
    class PLAN,ARCH,RUN repro
    class GATE,REPORT,JSON out

Solid arrows are the always-on validation pipeline; dotted arrows are the opt-in reproducibility checks. The CI gate maps the combined result to an exit code that any CI system can act on.

Exit codes:

Code	Meaning
0	All checks passed
1	One or more violations found
2	Pipeline error (bad schema, program crash, missing tooling)

CLI reference

The entry point is python -m act.run.

Flag	Required	Default	Description
`--program PATH`	yes	none	Path to the Pulumi program file (or project directory)
`--schema PATH [PATH ...]`	yes	none	One or more provider schema JSON files. Repeat for multi-provider programs
`--output DIR`	no	none	Write a structured run artefact (JSON) to this directory
`--log-level LEVEL`	no	`WARNING`	One of `DEBUG`, `INFO`, `WARNING`, `ERROR`
`--rules ENGINE [ENGINE ...]`	no	none	Load an additional rule engine. Currently: `checkov` (193+ Kubernetes checks)
`--check-deployment-arch ARCH`	no	off	Smoke-boot every container image referenced by the program under `linux/<ARCH>` via QEMU. Example: `--check-deployment-arch riscv64`
`--check-deployment-runtime`	no	off	Provision an ephemeral k3s cluster matching the program's target, run `pulumi up` twice, and verify the deployed state hashes identically. Requires `docker`, `kubectl`, and `pulumi` CLI

Show the help text:

uv run python -m act.run --help

Examples

Validate a CAPE program

uv run python -m act.run \
    --program tests/fixtures/cape/path_a_valid.py \
    --schema tests/fixtures/cape/schema.json

Validate a Kubernetes program

uv run python -m act.run \
    --program tests/fixtures/kubernetes/nginx_deployment.py \
    --schema examples/kubernetes/schema.json

Validate a multi-provider program (CAPE + random)

uv run python -m act.run \
    --program my_program.py \
    --schema tests/fixtures/cape/schema.json tests/fixtures/random/schema.json

Add Checkov rules on top of the built-in oracle

uv run python -m act.run \
    --program tests/fixtures/kubernetes/nginx_deployment.py \
    --schema examples/kubernetes/schema.json \
    --rules checkov

Check that every image in the program can boot under riscv64

uv run python -m act.run \
    --program tests/fixtures/kubernetes/nginx_deployment.py \
    --schema examples/kubernetes/schema.json \
    --check-deployment-arch riscv64

Full reproducibility check on a real ephemeral cluster

uv run python -m act.run \
    --program tests/fixtures/kubernetes/configmap.py \
    --schema examples/kubernetes/schema.json \
    --check-deployment-runtime \
    --output ./act_runs

The --output directory will contain act_run_<TIMESTAMP>.json with plan-check, arch-check, and runtime-check results, plus the package versions used. Useful for audit.

Get JSON logs for a log aggregator

uv run python -m act.run \
    --program my_program.py --schema schema.json \
    --log-level INFO 2>&1 1>/dev/null | jq .

JSON goes to stderr; the human-readable PASS/FAIL report goes to stdout. The two streams are independent, so you can pipe or redirect them separately.

CI/CD integration

ACT exits 0/1/2. Wire it into any pipeline that respects exit codes.

GitHub Actions

# .github/workflows/act.yml
name: ACT validation
on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: recursive

      - uses: astral-sh/setup-uv@v3
      - run: uv sync --frozen

      # Optional: register QEMU binfmt for --check-deployment-arch
      - uses: docker/setup-qemu-action@v3

      - name: Validate IaC
        run: |
          uv run python -m act.run \
            --program infra/main.py \
            --schema schemas/cape.json schemas/kubernetes.json \
            --check-deployment-arch riscv64 \
            --output ./act_runs

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: act-runs
          path: ./act_runs/

GitLab CI

# .gitlab-ci.yml
act:
  image: ghcr.io/astral-sh/uv:python3.11
  variables:
    GIT_SUBMODULE_STRATEGY: recursive
  script:
    - uv sync --frozen
    - uv run python -m act.run --program infra/main.py --schema schemas/cape.json
  artifacts:
    when: always
    paths: [act_runs/]

Jenkins

pipeline {
  agent any
  stages {
    stage('ACT') {
      steps {
        sh '''
          uv sync --frozen
          uv run python -m act.run --program infra/main.py --schema schemas/cape.json
        '''
      }
    }
  }
}

Docker

A pre-built image is published. The image expects the program + schema on a mounted volume.

docker run --rm \
  -v "$PWD/infra:/work" \
  ghcr.io/hiro-microdatacenters-bv/act:latest \
  --program /work/main.py --schema /work/schemas/cape.json

Kubernetes

A Helm chart ships in charts/act/. It runs ACT as a one-shot Job:

helm install act ./charts/act \
  --set program=/work/main.py \
  --set schema[0]=/work/schemas/cape.json \
  --set-file files.program=./infra/main.py

The in-cluster Job runs the plan-time checks only: mock generation, the correctness oracle, and (when reachable) the cognitive validator. The reproducibility flags (--check-deployment-arch, --check-deployment-runtime) are intentionally not exposed by the chart. They shell out to docker, kubectl, and the pulumi CLI on the host, and the runtime check provisions an ephemeral k3s container next to ACT, which requires a docker socket and privileged access. That trust profile is appropriate for a developer workstation or a dedicated CI runner, not for a pod scheduled inside the cluster being validated. Run these checks from CI (see the GitHub Actions, GitLab CI, and Jenkins snippets above) or locally with uv run python -m act.run.

Writing programs ACT can validate

ACT accepts any standard Pulumi Python program. No special imports, no decorators.

Two flavours

LLM-generated programs: hard-coded inputs, no parameters. ACT runs the mock generator + oracle + (optionally) the cognitive validator.

# valid_instance.py
import pulumi_cape as cape

cape.Instance("web",
    zone="zone-1",
    sku_ref=cape.SkuReference(resource="skus/medium"),
    security_group_ref=cape.SecurityGroupReference(resource="security-groups/web"),
    ssh_keys=["ssh-keys/admin"],
)

Developer-written programs: parameterised, accept config. ACT additionally fuzzes the inputs and runs hypothesis property tests to explore corners that pass typing but fail policy.

# parameterised.py
import pulumi
import pulumi_cape as cape

config = pulumi.Config()
zone = config.require("zone")
sku  = config.require("sku")

cape.Instance("web", zone=zone, sku_ref=cape.SkuReference(resource=sku))

Rules

Use provider SDK classes normally (from pulumi_cape.compute import Instance).
Use pulumi.export(...) to expose outputs; ACT captures them too.
Do NOT call external APIs or read live cloud state inside the program. ACT runs the program in a sandboxed mock environment; live calls will be intercepted or fail.

Supported providers

ACT is provider-agnostic and works with any Pulumi provider's schema. The repo ships ready-to-run schemas + fixtures for:

Provider	Schema location
CAPE	`tests/fixtures/cape/schema.json`
Kubernetes	`examples/kubernetes/schema.json`
pulumi-random	`tests/fixtures/random/schema.json`

For any other Pulumi provider:

pulumi package get-schema <provider-name> > schemas/<provider>.json
uv run python -m act.run --program my_program.py --schema schemas/<provider>.json

Extending ACT

Two extension points in act/plugins/base.py.

Custom oracle rules

To add rules to the built-in oracle without replacing it, drop a file in act/rules/<provider>.py with a register(oracle) function:

# act/rules/myprovider.py
def rule_no_public_endpoint(resource_type, inputs):
    if resource_type == "myprovider:net/v1:Service" and inputs.get("public"):
        return [Violation(field="public", message="Public endpoints are forbidden", severity="HIGH")]
    return []

def register(oracle):
    oracle.add_rule(rule_no_public_endpoint, scope="myprovider:net/v1:Service")

Rules auto-load by file name on startup.

To replace the oracle entirely, subclass OraclePlugin:

from act.plugins.base import OraclePlugin
from act.core.violations import Violation

class MyOracle(OraclePlugin):
    def check(self, resource_type: str, inputs: dict) -> list[Violation]:
        return []

Custom test generators

Subclass TestGeneratorPlugin to inject a mutation strategy beyond the built-in fuzz and property runners:

from act.plugins.base import TestGeneratorPlugin
from act.core.violations import Violation

class MyGenerator(TestGeneratorPlugin):
    def run(self, program_path: str) -> list[Violation]:
        # mutate program inputs, run the pipeline, collect violations
        return []

Custom generators only run on parameterised programs.

Architecture

act/
  core/
    mock_generator.py     # Pulumi SDK interception; produces a captured plan
    oracle.py             # Pluggable rule engine
    violations.py         # Violation dataclass
    fuzz_runner.py        # atheris-based fuzz runner (parameterised programs)
    property_runner.py    # hypothesis-based property tests (parameterised programs)
    pipeline.py           # Wires everything together
  rules/
    cape.py               # CAPE security rules (security_group_ref, ssh_keys, ...)
  integrations/
    checkov_adapter.py    # Optional Checkov rule engine
  acv/                    # Cognitive validator (LangGraph + LLM tools)
  gate/
    ci_gate.py            # Exit-code mapping, structured report
  plugins/
    base.py               # OraclePlugin + TestGeneratorPlugin ABCs
  reproducibility/
    plan_check.py         # Same-program-twice plan hashing
    deployment_arch.py    # Per-image arch smoke-boot
    runtime_check.py      # Substrate-driven twice-and-hash on a real cluster
    substrates/           # DockerSubstrate (CPU) + AcceleratorSubstrate (GPU/FPGA/CXL)
    artefact.py           # JSON-per-run artefact writer
  run.py                  # CLI entry point

Design decisions worth knowing:

Provider-agnostic by construction. No provider is hardcoded; the mock generator reads any Pulumi schema and works.
Oracle is structural. Missing fields, wrong types, policy violations. Content analysis (shell-command-in-user_data, embedded secrets) is the cognitive validator's job.
The cognitive validator is additive. If the LLM endpoint is unreachable, ACT still passes/fails on the deterministic oracle. It never blocks on AI unavailability.
Logging is two-stream. Structured JSON on stderr (for log aggregators); human report on stdout (for terminals and CI logs). The two streams are independent; redirect them separately.
Plugins are stable. OraclePlugin and TestGeneratorPlugin are the two extension points; everything else is internal.

Reproducibility checks

ACT verifies your IaC is deterministic at three levels. Each is opt-in (except plan check, which always runs).

1. Plan determinism (always on)

Runs your program twice on the host with the mock generator, hashes the canonical-JSON output, compares. Catches programs that produce different plans on repeated invocations (random suffixes, time-dependent generators, env-dependent values).

2. Image arch readiness (`--check-deployment-arch <arch>`)

For every container image referenced by your program, runs docker run --platform linux/<arch> --entrypoint /bin/true <image>. Catches images that lack an arch variant or fail to start under QEMU.

3. Runtime reproducibility (`--check-deployment-runtime`)

Provisions an ephemeral k3s cluster in Docker, runs pulumi up against it twice, hashes the deployed state across runs. Catches non-deterministic deployments that survive the host-side plan check (random pod-name suffixes, env-leak into ConfigMap data, time-of-day branches in IaC).

The substrate registry covers six target classes:

Target	How it works
amd64	`rancher/k3s` upstream image
arm64	`rancher/k3s` upstream image (native on Apple Silicon, otherwise via QEMU)
riscv64	A pinned k3s build with bundled bridge CNI, runs under QEMU `binfmt_misc`
GPU	k3s + declares `nvidia.com/gpu` as a schedulable Extended Resource on the node, so GPU-aware programs schedule correctly. Validates the IaC layer; real CUDA execution needs a GPU on the host
FPGA	k3s + `cape.eu/fpga` Extended Resource + iverilog workload image. Boot-flow simulator's `$display` output is captured and hashed for byte-equal comparison across runs
CXL	k3s + `cape.eu/cxl` Extended Resource + QEMU-in-Pod workload image. Boots a Linux 6.8 guest with a `cxl-type3` memory device; the `cxl list -v` topology JSON is captured and hashed

The substrate is selected automatically from the program's target architecture and declared resource needs (nvidia.com/gpu, cape.eu/fpga, cape.eu/cxl). To force a specific substrate from the CLI, pass --check-deployment-runtime and ACT picks the matching row.

Troubleshooting

Symptom	Likely cause + fix
`ModuleNotFoundError: No module named 'pulumi_cape'`	The `cape-sdks/` submodule isn't initialised. Run `git submodule update --init --recursive`
`FileNotFoundError: schema.json`	Either the path is wrong, or you haven't fetched the schema. Run `pulumi package get-schema <provider> > schemas/<provider>.json`
`--check-deployment-arch riscv64` exits with `docker_missing`	Docker isn't on PATH. Install Docker Desktop or `docker.io`
Image arch check fails with `no_arch_variant`	The image's manifest list doesn't include the target arch. Either rebuild the image multi-arch (`docker buildx build --platform linux/amd64,linux/arm64,linux/riscv64`), or remove the target arch from your validation matrix
`--check-deployment-runtime` exits with `substrate_unavailable`	One of `docker`, `kubectl`, or `pulumi` CLI isn't on PATH. Install all three
`--check-deployment-runtime` for the FPGA/CXL targets is skipped	The workload image (`act-fpga:iverilog`, `act-cxl:qemu`) isn't built locally. Run `bash tests/integration/fpga/build.sh` or `bash tests/integration/cxl/build.sh` to build it
Pulumi `pip list` fails inside the runtime check	The Python venv ACT is running in needs `pip` installed. Run `uv pip install pip` once
Exit code 2 with `Pipeline failed: ...`	Programmatic error in the Pulumi program itself, or a schema parse failure. Re-run with `--log-level DEBUG` to see the full traceback

Logging

# Silent (default): only the PASS/FAIL report on stdout
uv run python -m act.run ...

# INFO: one JSON line per pipeline stage with duration_ms
uv run python -m act.run ... --log-level INFO

# DEBUG: every oracle call, every fuzz/property entry/exit
uv run python -m act.run ... --log-level DEBUG

Example INFO output (stderr):

{"level": "INFO", "logger": "act.core.pipeline", "msg": "pipeline.start", "program": "main.py", "parameterized": false}
{"level": "INFO", "logger": "act.core.pipeline", "msg": "pipeline.mock_done", "resources": ["web"], "duration_ms": 28}
{"level": "INFO", "logger": "act.core.pipeline", "msg": "pipeline.oracle_done", "violations": 2, "duration_ms": 0}
{"level": "INFO", "logger": "act.gate.ci_gate", "msg": "ci_gate.result", "passed": false, "violations": 2, "exit_code": 1}

Pipe the JSON to jq (stderr) and keep the report on stdout:

uv run python -m act.run ... --log-level INFO 2> >(jq .) > report.txt

License

Released under an open-source license.

Name		Name	Last commit message	Last commit date
Latest commit History 88 Commits
.github/workflows		.github/workflows
act		act
cape-sdks @ 055fe49		cape-sdks @ 055fe49
charts/act		charts/act
examples/kubernetes		examples/kubernetes
tests		tests
tools		tools
.dockerignore		.dockerignore
.flake8		.flake8
.gitignore		.gitignore
.gitmodules		.gitmodules
.pre-commit-config.yaml		.pre-commit-config.yaml
.python-version		.python-version
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
LICENSE		LICENSE
NOTICE		NOTICE
README.md		README.md
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

ACT: Automated Configuration Testing

Table of contents

Quick start

What ACT does

Architecture and flow

CLI reference

Examples

Validate a CAPE program

Validate a Kubernetes program

Validate a multi-provider program (CAPE + random)

Add Checkov rules on top of the built-in oracle

Check that every image in the program can boot under riscv64

Full reproducibility check on a real ephemeral cluster

Get JSON logs for a log aggregator

CI/CD integration

GitHub Actions

GitLab CI

Jenkins

Docker

Kubernetes

Writing programs ACT can validate

Two flavours

Rules

Supported providers

Extending ACT

Custom oracle rules

Custom test generators

Architecture

Reproducibility checks

1. Plan determinism (always on)

2. Image arch readiness (--check-deployment-arch <arch>)

3. Runtime reproducibility (--check-deployment-runtime)

Troubleshooting

Logging

License

About

Resources

License

Contributing

Uh oh!

Stars

Watchers

Forks

Releases 3

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

2. Image arch readiness (`--check-deployment-arch <arch>`)

3. Runtime reproducibility (`--check-deployment-runtime`)

Packages