diff --git a/README.md b/README.md index f139ebf..bf7c2f6 100644 --- a/README.md +++ b/README.md @@ -66,9 +66,10 @@ generated runtime, source, and docs. Each runtime manifest records schema, skill, source identity, source path, local source-repository root, validation profile, and installer version. Skill consistency review compares only parsed `INSTALLER_VERSION` values before the selected installed-skill check. -The `global-skills-consistency-review` automation enumerates every direct -manifest-backed skill under `$CODEX_HOME/skills` and invokes the single-skill -action once per skill without repository deduplication. +The `global-skills-consistency-review` automation uses the lifecycle runtime +inventory helper to enumerate every direct manifest-backed skill under +`$CODEX_HOME/skills`, then invokes the single-skill action once per valid entry +without repository deduplication. ## Scripts @@ -79,7 +80,7 @@ action once per skill without repository deduplication. | `skills/ceratops-skill-lifecycle/scripts/runtime/install-managed-skills.py` | Installed source-scoped runtime installer; full installs run full source validation and same-source stale cleanup, while targeted installs validate only selected skills and remove no stale folders. | | `skills/ceratops-skill-lifecycle/scripts/runtime/resolve-lifecycle-bundle.py` | Installed-first resolver with target-checkout fallback only for the initial Ceratops installation. | | `skills/ceratops-skill-lifecycle/scripts/runtime/synchronize-installers.py` | Copies the authoritative installer into an approved task worktree only when its parsed version is missing or lower, then runs full validation. | -| `skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py` | Discovers one repository's installed managed runtime, validates identity and installer versions, and compares every managed file with canonical builder output. | +| `skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py` | Inventories direct managed runtime manifests or validates one selected skill after deriving its source repository, including identity, installer version, and complete managed-file comparison. | | `skills/ceratops-skill-lifecycle/scripts/runtime/managed_runtime_builder.py` | Canonical managed-runtime builder used for installation and expected-tree generation. | | `skills/ceratops-gh-repo-lifecycle/scripts/github_contract_engine/` | Package CLI for contract schemas, consistency, source documents, org/repo validation, shared severity levels, and non-deterministic evidence. | | `skills/ceratops-gh-repo-lifecycle/scripts/github_pr_workflow/` | Package CLI for prepared-branch PR publication, PR readiness, Codex review wait/resolution, merge orchestration, live merge verification, and post-merge local sync. | diff --git a/skills/ceratops-credit-savings-analysis/SKILL.md b/skills/ceratops-credit-savings-analysis/SKILL.md index ebeb57c..52082a4 100644 --- a/skills/ceratops-credit-savings-analysis/SKILL.md +++ b/skills/ceratops-credit-savings-analysis/SKILL.md @@ -39,6 +39,9 @@ cannot be identified. - Treat preventable rework, duplicate investigation, broad reads, noisy output, oversized validation, stale checks, waits, reversions, and user corrections as candidate credit waste. +- Treat avoidable model requests as candidate credit waste even when they do not + cause rework, including requests made unnecessary by applicable rules or + already-sufficient evidence. - Use credit-waste signals as prompts for analysis, not mandatory checks; inspect only categories visible in the selected evidence window. - Prefer the smallest durable control: wording, deterministic helper, preflight, @@ -53,10 +56,10 @@ cannot be identified. instructions, skills, automations, or helpers, provide the exact proposed change before mutating; otherwise name the target artifact and target behavior. -- Before reporting recommendations, classify candidate controls against - inspected evidence as implemented or still unimplemented; omit implemented - controls unless needed to justify that no still-unimplemented proposal - remains. +- Before reporting findings, classify each recommended control against + inspected evidence as implemented or still unimplemented. Report every + confirmed finding; for implemented controls, state the status without + re-recommending the control. - Merge recommendations that share the same producer and control. - When prompt-level savings cases exist, rank the top five evidence-backed cases, or all available cases when fewer exist, using only information @@ -104,13 +107,14 @@ cannot be identified. ### Output Contract -Start with one of: +Start with `Blocked: .` when missing +evidence prevents analysis. Otherwise start with both: -- `No avoidable credit spend found in the inspected runs.` -- `No still-unimplemented credit-savings proposals found in the inspected runs.` -- `Found still-unimplemented credit-savings proposals.` -- `Blocked: .` +- `Avoidable credit spend: found.` or + `Avoidable credit spend: none found.` +- `Still-unimplemented controls: found.` or + `Still-unimplemented controls: none found.` -Then report only findings with still-unimplemented recommendations, any required -ranked prompt-level table, excluded ordinary failures, and important evidence -limits. +Then report every confirmed finding, any required ranked prompt-level table, +excluded ordinary failures, and important evidence limits. For each finding, +state whether its control is implemented or still unimplemented. diff --git a/skills/ceratops-gh-repo-lifecycle/references/create-or-publish.md b/skills/ceratops-gh-repo-lifecycle/references/create-or-publish.md index e676ba6..5f1a82a 100644 --- a/skills/ceratops-gh-repo-lifecycle/references/create-or-publish.md +++ b/skills/ceratops-gh-repo-lifecycle/references/create-or-publish.md @@ -20,9 +20,10 @@ release semantics. ### Inputs To Capture -- GitHub organization (not a personal account), repo name, default branch, - visibility, fork status, and branch naming. Default organization: - `Ceratops-Code`. +- Capture the repo name, default branch, visibility, fork status, and branch + naming. +- Always use `Ceratops-Code` as the GitHub repository owner; do not ask which + owner to use. - Maintainer merge policy, expected artifact type, release expectations, and missing inputs required by the GitHub repo, code repo, or artifact contracts. diff --git a/skills/ceratops-skill-lifecycle/references/contracts/skill-deterministic-contract.json b/skills/ceratops-skill-lifecycle/references/contracts/skill-deterministic-contract.json index 239f546..9aea8f2 100644 --- a/skills/ceratops-skill-lifecycle/references/contracts/skill-deterministic-contract.json +++ b/skills/ceratops-skill-lifecycle/references/contracts/skill-deterministic-contract.json @@ -88,8 +88,9 @@ "skill_validation_command": "python skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py --mode skill --skill ", "full_validation_command": "python skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py --mode full", "section_validation_command": "python skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py --mode sections", - "runtime_validation_command": "python skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py --repo-root --skill ", - "result_contract": "Exit 0 with an ok line when selected deterministic checks pass; nonzero with line-specific errors when they fail." + "runtime_validation_command": "python skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py --skill ", + "runtime_inventory_command": "python skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py --inventory", + "result_contract": "Validation exits zero with compact JSON when selected deterministic checks pass and nonzero with line-specific errors when they fail; inventory exits zero with compact valid entries and malformed direct-manifest blockers." }, "checks": [ { @@ -211,7 +212,7 @@ "installed source_repository_root and installer_version", "selected install scope" ], - "pass_condition": "Full installation runs full source-repository validation before building; explicit skill installation validates only selected skills. Runtime manifests record schema, skill, runtime_source_id, source_path, source_repository_root, validation_profile, and installer_version. The builder preflights all selected targets, refuses unmanaged folders, links, invalid ownership, and same-name folders owned by another source; a full install removes only stale folders with the same runtime_source_id, while an explicit targeted install removes none. Runtime validation can select one direct manifest-backed skill and compare only its complete managed file tree with canonical builder output.", + "pass_condition": "Full installation runs full source-repository validation before building; explicit skill installation validates only selected skills. Runtime manifests record schema, skill, runtime_source_id, source_path, source_repository_root, validation_profile, and installer_version. The builder preflights all selected targets, refuses unmanaged folders, links, invalid ownership, and same-name folders owned by another source; a full install removes only stale folders with the same runtime_source_id, while an explicit targeted install removes none. Runtime inventory emits compact direct-manifest entries and blockers without auditing; selected runtime validation derives the source repository from the current manifest and compares only that skill's complete managed file tree with canonical builder output.", "source_lines": [ "skills/ceratops-skill-lifecycle/scripts/runtime/resolve-lifecycle-bundle.py:resolve_bundle", "skills/ceratops-skill-lifecycle/scripts/runtime/install-managed-skills.py:main", diff --git a/skills/ceratops-skill-lifecycle/references/skills-consistency-review.md b/skills/ceratops-skill-lifecycle/references/skills-consistency-review.md index e94232a..00d5df3 100644 --- a/skills/ceratops-skill-lifecycle/references/skills-consistency-review.md +++ b/skills/ceratops-skill-lifecycle/references/skills-consistency-review.md @@ -15,8 +15,13 @@ installed-runtime surface. scripts/skills-consistency-source-validator.py --repo-root --mode skill --skill ` from the lifecycle bundle. - (D) Managed runtime validator: `python - scripts/runtime/skills-consistency-runtime-validator.py --repo-root - --skill ` from the installed lifecycle bundle. + scripts/runtime/skills-consistency-runtime-validator.py --skill + ` from the installed lifecycle bundle; it derives the source + repository from the selected direct runtime manifest. +- (D) Global inventory helper: `python + scripts/runtime/skills-consistency-runtime-validator.py --inventory` emits + compact JSON for every direct manifest-backed installed skill and malformed + direct-manifest blocker without auditing any skill. - (D) Installer synchronization when repair is authorized: `python scripts/runtime/synchronize-installers.py --target-repo-root ` from the installed lifecycle bundle. @@ -48,9 +53,8 @@ Infer missing inputs from the repository and installed manifests before asking. ### Global Automation Caller -- The `global-skills-consistency-review` automation owns direct installed-skill - discovery and invokes this action once per direct skill directory containing - `.runtime-manifest.json`. +- The `global-skills-consistency-review` automation runs the global inventory + helper and invokes this action once per valid inventory entry. - The automation may aggregate results, but it must not deduplicate skills that share a source repository or make this action perform global discovery. - Report an unreadable direct runtime manifest as a blocker for that installed diff --git a/skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py b/skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py index 98a490c..0f3205b 100644 --- a/skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py +++ b/skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py @@ -36,6 +36,8 @@ "validation_profile", "installer_version", } + + def default_runtime_root() -> pathlib.Path: """Return the direct personal runtime skills root.""" @@ -76,6 +78,40 @@ def read_manifest(skill_dir: pathlib.Path) -> tuple[Mapping[str, object] | None, return (value, None) if isinstance(value, Mapping) else (None, "manifest must be a JSON object") +def routing_manifest_errors( + skill_dir: pathlib.Path, + manifest: Mapping[str, object], +) -> list[str]: + """Return routing errors that block inventory or source-root derivation.""" + + errors: list[str] = [] + missing = sorted(REQUIRED_FIELDS - set(manifest)) + if missing: + return [f"runtime manifest missing {', '.join(missing)}"] + string_fields = ("skill", "runtime_source_id", "source_path", "source_repository_root") + invalid_strings = [ + field + for field in string_fields + if not isinstance(manifest.get(field), str) or not str(manifest[field]).strip() + ] + if invalid_strings: + errors.append(f"runtime manifest has invalid {', '.join(invalid_strings)}") + if manifest.get("schema") != RUNTIME_MANIFEST_SCHEMA: + errors.append("unsupported runtime manifest schema") + if manifest.get("skill") != skill_dir.name: + errors.append(f"manifest skill identity is {manifest.get('skill')!r}") + if manifest.get("validation_profile") not in VALIDATION_PROFILES: + errors.append("unsupported runtime validation profile") + version = manifest.get("installer_version") + if not isinstance(version, int) or isinstance(version, bool) or version < 1: + errors.append("installer_version must be a positive integer") + source_root = manifest.get("source_repository_root") + if isinstance(source_root, str) and source_root.strip(): + if not pathlib.Path(source_root).is_absolute(): + errors.append("source_repository_root must be an absolute local path") + return errors + + def managed_directories( runtime_root: pathlib.Path, selected_skill: str | None = None, @@ -92,6 +128,62 @@ def managed_directories( ) +def runtime_inventory(runtime_root: pathlib.Path) -> dict[str, object]: + """Return compact direct-manifest inventory without auditing any skill.""" + + skills: list[dict[str, str]] = [] + blockers: list[dict[str, object]] = [] + for skill_dir in managed_directories(runtime_root): + manifest, error = read_manifest(skill_dir) + if manifest is None: + blockers.append( + { + "directory": skill_dir.name, + "errors": [f"unreadable runtime manifest: {error}"], + } + ) + continue + errors = routing_manifest_errors(skill_dir, manifest) + if errors: + blockers.append({"directory": skill_dir.name, "errors": errors}) + continue + skills.append( + { + "installed_path": str(skill_dir.resolve()), + "skill": str(manifest["skill"]), + "source_repository_root": str(manifest["source_repository_root"]), + } + ) + return { + "blocked": len(blockers), + "blockers": blockers, + "managed": len(skills), + "skills": skills, + "status": "inventory", + } + + +def repo_root_from_selected_manifest( + runtime_root: pathlib.Path, + selected_skill: str, +) -> tuple[pathlib.Path | None, list[str]]: + """Derive one selected skill's source repository from its direct manifest.""" + + if pathlib.PurePath(selected_skill).name != selected_skill: + return None, ["selected skill must be one direct runtime directory name"] + skill_dir = runtime_root / selected_skill + manifest, error = read_manifest(skill_dir) + if manifest is None: + return None, [f"{selected_skill}: unreadable runtime manifest: {error}"] + errors = routing_manifest_errors(skill_dir, manifest) + if errors: + return None, [f"{selected_skill}: {message}" for message in errors] + try: + return pathlib.Path(str(manifest["source_repository_root"])).resolve(), [] + except OSError as exc: + return None, [f"{selected_skill}: source_repository_root is unresolved: {exc}"] + + def source_context( repo_root: pathlib.Path, ) -> tuple[Mapping[str, object] | None, str | None, str | None, set[str], list[str]]: @@ -170,17 +262,9 @@ def discover_runtime( root_matches = False if declared_source_id != source_id and skill_dir.name not in skill_names and not root_matches: continue - missing = sorted(REQUIRED_FIELDS - set(manifest)) - if missing: - errors.append(f"{skill_dir.name}: runtime manifest missing {', '.join(missing)}") - continue - string_fields = ("skill", "runtime_source_id", "source_path", "source_repository_root") - invalid_strings = [field for field in string_fields if not isinstance(manifest.get(field), str) or not str(manifest[field]).strip()] - if invalid_strings: - errors.append(f"{skill_dir.name}: runtime manifest has invalid {', '.join(invalid_strings)}") - continue - if manifest.get("schema") != RUNTIME_MANIFEST_SCHEMA: - errors.append(f"{skill_dir.name}: unsupported runtime manifest schema") + manifest_errors = routing_manifest_errors(skill_dir, manifest) + if manifest_errors: + errors.extend(f"{skill_dir.name}: {message}" for message in manifest_errors) continue if manifest.get("runtime_source_id") != source_id: errors.append(f"{skill_dir.name}: runtime manifest belongs to another source") @@ -188,12 +272,13 @@ def discover_runtime( if manifest.get("validation_profile") != source_profile: errors.append(f"{skill_dir.name}: runtime validation profile does not match source") continue - version = manifest.get("installer_version") - if not isinstance(version, int) or isinstance(version, bool) or version < 1: - errors.append(f"{skill_dir.name}: installer_version must be a positive integer") + try: + declared_root = pathlib.Path(str(manifest["source_repository_root"])).resolve() + except OSError as exc: + errors.append(f"{skill_dir.name}: source_repository_root is unresolved: {exc}") continue - if not pathlib.Path(str(manifest["source_repository_root"])).is_absolute(): - errors.append(f"{skill_dir.name}: source_repository_root must be an absolute local path") + if declared_root != repo_root: + errors.append(f"{skill_dir.name}: source_repository_root does not match repository") continue records.append({"directory": skill_dir.name, **dict(manifest)}) @@ -317,12 +402,33 @@ def main() -> int: parser = argparse.ArgumentParser( description="Validate managed runtime skills for one source repository or selected skill." ) - parser.add_argument("--repo-root", required=True, type=pathlib.Path) + parser.add_argument("--repo-root", type=pathlib.Path) parser.add_argument("--runtime-root", type=pathlib.Path) parser.add_argument("--skill", help="Validate only this direct manifest-backed runtime skill.") + parser.add_argument( + "--inventory", + action="store_true", + help="Emit compact direct-manifest inventory without auditing skills.", + ) args = parser.parse_args() - repo_root = args.repo_root.resolve() runtime_root = (args.runtime_root or default_runtime_root()).resolve() + if args.inventory: + if args.repo_root is not None or args.skill is not None: + parser.error("--inventory cannot be combined with --repo-root or --skill") + print(json.dumps(runtime_inventory(runtime_root), sort_keys=True)) + return 0 + if args.repo_root is None and args.skill is None: + parser.error("--repo-root is required unless --skill or --inventory is used") + derivation_errors: list[str] = [] + if args.repo_root is not None: + repo_root = args.repo_root.resolve() + else: + repo_root, derivation_errors = repo_root_from_selected_manifest(runtime_root, args.skill) + if repo_root is None: + print(f"errors: {len(derivation_errors)}", file=sys.stderr) + for error in derivation_errors: + print(error, file=sys.stderr) + return 1 records, errors, source_manifest, source_id, skill_names = discover_runtime( repo_root, runtime_root, diff --git a/skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py b/skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py index 062f855..9004c8d 100644 --- a/skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py +++ b/skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py @@ -866,6 +866,7 @@ def check_validation_command_surface() -> list[str]: if ( "compare_managed_tree" not in runtime_validator_text or '"--skill"' not in runtime_validator_text + or '"--inventory"' not in runtime_validator_text or '"--mode"' in runtime_validator_text ): errors.append("runtime validator must compare managed trees through one command") diff --git a/tests/test_skill_runtime_multi_repo.py b/tests/test_skill_runtime_multi_repo.py index 0f44f96..2fb52a8 100644 --- a/tests/test_skill_runtime_multi_repo.py +++ b/tests/test_skill_runtime_multi_repo.py @@ -637,8 +637,6 @@ def test_selected_skill_review_does_not_audit_sibling_skills(tmp_path: pathlib.P [ sys.executable, str(RUNTIME_VALIDATOR), - "--repo-root", - str(repo), "--runtime-root", str(install_root), "--skill", @@ -655,3 +653,40 @@ def test_selected_skill_review_does_not_audit_sibling_skills(tmp_path: pathlib.P "runtime_source_id": "example/compatible", "status": "valid", } + + +def test_runtime_inventory_lists_direct_manifests_and_malformed_blockers( + tmp_path: pathlib.Path, +) -> None: + repo = tmp_path / "compatible" + install_root = tmp_path / "installed" + create_compatible_repo(repo, "example/compatible", ["alpha-tool", "beta-tool"]) + assert run_builder(repo, install_root, "--remove-stale").returncode == 0 + malformed = install_root / "broken-tool" + malformed.mkdir() + (malformed / RUNTIME_MANIFEST).write_text("{\n", encoding="utf-8", newline="\n") + nested = install_root / "unmanaged-tool" / "nested-managed" + nested.mkdir(parents=True) + (nested / RUNTIME_MANIFEST).write_text("{}\n", encoding="utf-8", newline="\n") + + result = subprocess.run( + [ + sys.executable, + str(RUNTIME_VALIDATOR), + "--runtime-root", + str(install_root), + "--inventory", + ], + capture_output=True, + text=True, + check=False, + ) + + assert result.returncode == 0, result.stderr + inventory = json.loads(result.stdout) + assert inventory["status"] == "inventory" + assert inventory["managed"] == 2 + assert inventory["blocked"] == 1 + assert [item["skill"] for item in inventory["skills"]] == ["alpha-tool", "beta-tool"] + assert inventory["blockers"][0]["directory"] == "broken-tool" + assert "unreadable runtime manifest" in inventory["blockers"][0]["errors"][0]