diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..d11faf1 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,44 @@ +# Changelog + +All notable changes to this project are documented here. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/) applied +to the contract (see [`docs/versioning.md`](docs/versioning.md)). "The contract" +is the public API surface — the `fetcher.yaml` / category / manifest / envelope +schemas and the `paramify` CLI — not the internal code. + +## [Unreleased] + +## [0.2.0] - 2026-07-09 + +First public release. + +### Added + +- Fetcher framework: the `paramify` CLI (list · catalog · describe · manifests · + validate · run · runs · evidence · upload · manifest builder) plus the + `paramify tui` front-end, both talking only to `framework.api`. +- 108 fetchers across 8 categories (aws 79, okta 8, sentinelone 5, knowbe4 4, + gitlab 3, k8s 3, rippling 3, checkov 2). The AWS category collects where + deployed via the ambient credential chain, with optional per-target + profile/region fanout. +- Evidence envelope (`{schema_version, metadata, payload}`, `schema_version` + `1.0`) wrapped around every fetcher output by the runner. +- Paramify evidence uploader (`uploaders/paramify_evidence/`). +- KSI metadata: an optional `ksis` array on `fetcher.yaml`, mappings populated + for 89 fetchers, and `paramify ksi` — a FedRAMP 20x KSI coverage view over + `api.ksi_coverage()`. +- Optional `validators` metadata on `fetcher.yaml` (regex checks over the + evidence payload). +- Versioning & contract policy ([`docs/versioning.md`](docs/versioning.md)), this + changelog, and the manual release runbook + ([`docs/releasing.md`](docs/releasing.md)). + +### Changed + +- Licensed under GPL-3.0-only. +- Documentation rewritten for public consumption. + +[Unreleased]: https://github.com/paramify/paramify-fetchers/compare/v0.2.0...HEAD +[0.2.0]: https://github.com/paramify/paramify-fetchers/releases/tag/v0.2.0 diff --git a/README.md b/README.md index 414cda0..438760b 100644 --- a/README.md +++ b/README.md @@ -364,6 +364,8 @@ To add evidence collection for a new control or a new tool, see [`docs/authoring | [`docs/run_manifest_reference.md`](docs/run_manifest_reference.md) | Manifest format reference | | [`docs/config_injection_design.md`](docs/config_injection_design.md) | Platform/config/auth model | | [`docs/design.md`](docs/design.md) | Why the framework is shaped this way + current state of the work | +| [`docs/versioning.md`](docs/versioning.md) | How we version, the contract, and what 1.0 means | +| [`docs/releasing.md`](docs/releasing.md) | How a release is cut | ## License diff --git a/docs/releasing.md b/docs/releasing.md new file mode 100644 index 0000000..24a05ae --- /dev/null +++ b/docs/releasing.md @@ -0,0 +1,108 @@ +# Releasing + +How we cut a release today: **manual and tag-driven**. Releases are curated — +we do *not* release on every merge, because customers run this in production. +The version and changelog are maintained by hand against the +[versioning policy](versioning.md); automation +([release-please](https://github.com/googleapis/release-please)) is a future +upgrade when cadence justifies it (see [below](#future-automation)). + +## Prerequisites + +- The [`gh`](https://cli.github.com/) CLI, authenticated (`gh auth status`). +- On an up-to-date `main` with a clean working tree. +- CI green on `main`. +- A version chosen per the [bump policy](versioning.md#bump-policy). Pre-1.0, a + breaking change bumps the **minor**, not the major. + +## Steps + +Using `X.Y.Z` as the version (no `v` in files; the **tag** is `vX.Y.Z`). + +1. **Update `CHANGELOG.md`.** Rename the `## [Unreleased]` heading to + `## [X.Y.Z] - YYYY-MM-DD` (today's date), add a fresh empty `## [Unreleased]` + above it, and update the link references at the bottom: + + ```markdown + [Unreleased]: https://github.com/paramify/paramify-fetchers/compare/vX.Y.Z...HEAD + [X.Y.Z]: https://github.com/paramify/paramify-fetchers/compare/vPREV...vX.Y.Z + ``` + +2. **Bump the version** in `pyproject.toml` (`version = "X.Y.Z"`). + +3. **Commit** on a branch and open a PR (so it passes CI / review like anything + else): + + ```bash + git checkout -b release/vX.Y.Z + git commit -am "chore(release): vX.Y.Z" + gh pr create --fill + ``` + +4. **Merge** the PR into `main`. + +5. **Tag the merge commit** and push the tag: + + ```bash + git checkout main && git pull + git tag -a vX.Y.Z -m "vX.Y.Z" + git push origin vX.Y.Z + ``` + +6. **Create the GitHub Release** from the tag, with the changelog section as the + notes: + + ```bash + gh release create vX.Y.Z --title "vX.Y.Z" --notes-file <(sed -n '/## \[X.Y.Z\]/,/## \[/p' CHANGELOG.md | sed '$d') + ``` + + GitHub automatically attaches the **source code** as `.tar.gz` and `.zip` to + every release — that is our release artifact for now (see + [Artifacts](#artifacts)). Add `--generate-notes` if you also want GitHub's + auto-generated commit/PR list appended. + +7. **Verify:** `gh release view vX.Y.Z` — confirm the notes, the tag, and that + the source archives are attached. + +## Pre-releases + +For dogfood / early-customer builds, tag a pre-release and flag it so it doesn't +show as "Latest": + +```bash +git tag -a vX.Y.Z-rc.1 -m "vX.Y.Z-rc.1" +git push origin vX.Y.Z-rc.1 +gh release create vX.Y.Z-rc.1 --title "vX.Y.Z-rc.1" --prerelease --generate-notes +``` + +Use `-rc.N` (release candidate) or `-beta.N`. These sort before the final +`vX.Y.Z` under SemVer. + +## Artifacts + +- **Now:** the GitHub-attached **source tarball/zip**. The supported install is + from source (`pip install -e .`), so the source archive is the artifact. +- **Deferred:** wheel/sdist + PyPI publishing. Blocked on the editable-only / + cwd-repo-root packaging issue (schemas resolve via the discovered repo root, + and fetchers execute as subprocesses out of `fetchers/`, which a plain wheel + doesn't ship). Revisit when that's fixed; a build+publish step then slots into + step 6. + +## What a release does *not* touch + +Only the **tool version** (tag + `pyproject.toml`) moves here. The envelope +`schema_version` and each fetcher's `version` are edited in their own files as +part of the change that alters them — see +[the three version axes](versioning.md#the-three-version-axes). Call them out in +the changelog entry so consumers notice. + +## Future automation + +When release cadence justifies removing the manual toil, adopt `release-please` +(`release-type: python`, `bump-minor-pre-major: true` to keep the pre-1.0 rule). +It maintains the changelog and version bump in a rolling release PR; merging it +tags and cuts the Release. Two things to wire at that point: enable *Settings → +Actions → "Allow GitHub Actions to create and approve pull requests"*, and build +the source tarball **inside** the release-please workflow gated on +`release_created` (a tag made by the default `GITHUB_TOKEN` won't trigger a +separate on-tag workflow). diff --git a/docs/versioning.md b/docs/versioning.md new file mode 100644 index 0000000..e4c641a --- /dev/null +++ b/docs/versioning.md @@ -0,0 +1,77 @@ +# Versioning & contract policy + +How this project versions itself, what we promise to keep stable, and what a +version bump means. Releases follow [Semantic Versioning](https://semver.org/) +against **the contract**, defined below — not the code, and not any single +fetcher. + +## The contract = the public API surface + +The thing we version and promise stability for is the **public API surface**, +not the internal implementation. It is: + +- **`framework/schemas/fetcher_schema.json`** — the `fetcher.yaml` contract +- **`framework/schemas/category_schema.json`** — the `_categories/.yaml` contract +- **`framework/schemas/run_manifest_schema.json`** — the run manifest format +- **`framework/schemas/envelope_schema.json`** — the evidence envelope, carried + by its own `schema_version` (see below) +- **the `paramify` CLI** — its commands, subcommands, flags, and `--json` output + shapes + +Everything else is *not* contract and can change without a major bump: the +framework's internal Python modules, the runner/uploader internals, the TUI's +layout, a fetcher's implementation details, and the exact prose of human CLI +output. If you build automation against this project, build it against the +surface above. + +## The three version axes + +These move **independently**. A release bumps only the first; the other two are +edited by hand in their own files and are called out in the changelog. + +| Axis | Where it lives | Scheme | Today | +|---|---|---|---| +| **Tool version** | git tag / GitHub Release (and `pyproject.toml`) | SemVer, applied to the contract | `0.2.0` | +| **Envelope `schema_version`** | `framework/envelope.py` → every evidence file's `schema_version` | SemVer, additive-only | `1.0` | +| **Per-fetcher `version`** | each `fetcher.yaml` | SemVer per fetcher | `0.x` (v0.x ports) | + +- **Tool version** is the release train — what a tag `vX.Y.Z` and a GitHub + Release name. This is what customers pin. +- **Envelope `schema_version`** stays **additive**: new optional fields only, so + the runner and any downstream consumer keep reading older-versioned evidence. + A non-additive envelope change is a contract break (see the bump table). +- **Per-fetcher `version`** tracks a single fetcher's own evolution and is + independent of the tool version. Ported fetchers are `0.x` until they conform + to the contract-native pattern. + +## Bump policy + +Apply this to a release by looking at everything merged since the last tag. + +| Bump | Trigger | +|---|---| +| **major** | manifest format break · a CLI command/flag removed or renamed · envelope shape change (non-additive) · a new **required** `fetcher.yaml` field · a fetcher removed or its output shape changed | +| **minor** | new fetchers or categories · new **optional** config/target fields · new CLI commands/flags · additive envelope fields | +| **patch** | fetcher bug fixes · doc fixes · dependency bumps with no behavior change | + +Take the highest bump that any single change triggers. + +## Pre-1.0 and the meaning of 1.0 + +We are pre-1.0 (`0.x`). While under 1.0, **the contract is not yet frozen**: a +breaking change bumps the **minor** (e.g. `0.2 → 0.3`), not the major. This is +standard SemVer 0.x behavior and lets the contract still move as we learn from +early use. + +**1.0 means the contract is frozen and supported.** After 1.0, any break in the +surface above requires a major bump and follows the deprecation policy +(announce → warn → remove across ≥1 minor cycle; see the deprecation issue). +Cutting 1.0 is a deliberate decision, not an automatic consequence of a break — +we ship it when the schemas, manifest, envelope, and CLI are ones we're willing +to stand behind for the long haul. + +## See also + +- [`docs/releasing.md`](releasing.md) — how a release is actually cut +- [`CHANGELOG.md`](../CHANGELOG.md) — the release history +- [`docs/envelope_design.md`](envelope_design.md) — the evidence envelope diff --git a/pyproject.toml b/pyproject.toml index cc6bc80..45e388a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "paramify-fetchers" -version = "0.1.0" +version = "0.2.0" description = "Paramify evidence fetcher framework — discover, build, run, and inspect evidence fetchers." readme = "README.md" requires-python = ">=3.10"