Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,10 @@ generated runtime, source, and docs. Each runtime manifest records schema,
skill, source identity, source path, local source-repository root, validation
profile, and installer version. Skill consistency review compares only parsed
`INSTALLER_VERSION` values before the selected installed-skill check.
The `global-skills-consistency-review` automation enumerates every direct
manifest-backed skill under `$CODEX_HOME/skills` and invokes the single-skill
action once per skill without repository deduplication.
The `global-skills-consistency-review` automation uses the lifecycle runtime
inventory helper to enumerate every direct manifest-backed skill under
`$CODEX_HOME/skills`, then invokes the single-skill action once per valid entry
without repository deduplication.

## Scripts

Expand All @@ -79,7 +80,7 @@ action once per skill without repository deduplication.
| `skills/ceratops-skill-lifecycle/scripts/runtime/install-managed-skills.py` | Installed source-scoped runtime installer; full installs run full source validation and same-source stale cleanup, while targeted installs validate only selected skills and remove no stale folders. |
| `skills/ceratops-skill-lifecycle/scripts/runtime/resolve-lifecycle-bundle.py` | Installed-first resolver with target-checkout fallback only for the initial Ceratops installation. |
| `skills/ceratops-skill-lifecycle/scripts/runtime/synchronize-installers.py` | Copies the authoritative installer into an approved task worktree only when its parsed version is missing or lower, then runs full validation. |
| `skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py` | Discovers one repository's installed managed runtime, validates identity and installer versions, and compares every managed file with canonical builder output. |
| `skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py` | Inventories direct managed runtime manifests or validates one selected skill after deriving its source repository, including identity, installer version, and complete managed-file comparison. |
| `skills/ceratops-skill-lifecycle/scripts/runtime/managed_runtime_builder.py` | Canonical managed-runtime builder used for installation and expected-tree generation. |
| `skills/ceratops-gh-repo-lifecycle/scripts/github_contract_engine/` | Package CLI for contract schemas, consistency, source documents, org/repo validation, shared severity levels, and non-deterministic evidence. |
| `skills/ceratops-gh-repo-lifecycle/scripts/github_pr_workflow/` | Package CLI for prepared-branch PR publication, PR readiness, Codex review wait/resolution, merge orchestration, live merge verification, and post-merge local sync. |
Expand Down
28 changes: 16 additions & 12 deletions skills/ceratops-credit-savings-analysis/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ cannot be identified.
- Treat preventable rework, duplicate investigation, broad reads, noisy output,
oversized validation, stale checks, waits, reversions, and user corrections as
candidate credit waste.
- Treat avoidable model requests as candidate credit waste even when they do not
cause rework, including requests made unnecessary by applicable rules or
already-sufficient evidence.
- Use credit-waste signals as prompts for analysis, not mandatory checks;
inspect only categories visible in the selected evidence window.
- Prefer the smallest durable control: wording, deterministic helper, preflight,
Expand All @@ -53,10 +56,10 @@ cannot be identified.
instructions, skills, automations, or helpers, provide the exact proposed
change before mutating; otherwise name the target artifact and target
behavior.
- Before reporting recommendations, classify candidate controls against
inspected evidence as implemented or still unimplemented; omit implemented
controls unless needed to justify that no still-unimplemented proposal
remains.
- Before reporting findings, classify each recommended control against
inspected evidence as implemented or still unimplemented. Report every
confirmed finding; for implemented controls, state the status without
re-recommending the control.
- Merge recommendations that share the same producer and control.
- When prompt-level savings cases exist, rank the top five evidence-backed
cases, or all available cases when fewer exist, using only information
Expand Down Expand Up @@ -104,13 +107,14 @@ cannot be identified.

### Output Contract

Start with one of:
Start with `Blocked: <specific missing evidence or target>.` when missing
evidence prevents analysis. Otherwise start with both:

- `No avoidable credit spend found in the inspected runs.`
- `No still-unimplemented credit-savings proposals found in the inspected runs.`
- `Found still-unimplemented credit-savings proposals.`
- `Blocked: <specific missing evidence or target>.`
- `Avoidable credit spend: found.` or
`Avoidable credit spend: none found.`
- `Still-unimplemented controls: found.` or
`Still-unimplemented controls: none found.`

Then report only findings with still-unimplemented recommendations, any required
ranked prompt-level table, excluded ordinary failures, and important evidence
limits.
Then report every confirmed finding, any required ranked prompt-level table,
excluded ordinary failures, and important evidence limits. For each finding,
state whether its control is implemented or still unimplemented.
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@ release semantics.

### Inputs To Capture

- GitHub organization (not a personal account), repo name, default branch,
visibility, fork status, and branch naming. Default organization:
`Ceratops-Code`.
- Capture the repo name, default branch, visibility, fork status, and branch
naming.
- Always use `Ceratops-Code` as the GitHub repository owner; do not ask which
owner to use.
- Maintainer merge policy, expected artifact type, release expectations, and
missing inputs required by the GitHub repo, code repo, or artifact contracts.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,9 @@
"skill_validation_command": "python skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py --mode skill --skill <skill-name>",
"full_validation_command": "python skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py --mode full",
"section_validation_command": "python skills/ceratops-skill-lifecycle/scripts/skills-consistency-source-validator.py --mode sections",
"runtime_validation_command": "python skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py --repo-root <repo-root> --skill <skill-name>",
"result_contract": "Exit 0 with an ok line when selected deterministic checks pass; nonzero with line-specific errors when they fail."
"runtime_validation_command": "python skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py --skill <skill-name>",
"runtime_inventory_command": "python skills/ceratops-skill-lifecycle/scripts/runtime/skills-consistency-runtime-validator.py --inventory",
"result_contract": "Validation exits zero with compact JSON when selected deterministic checks pass and nonzero with line-specific errors when they fail; inventory exits zero with compact valid entries and malformed direct-manifest blockers."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the source-validator result contract accurate

When this execution contract is used for the skill_validation_command, full_validation_command, or section_validation_command, it now tells callers to expect compact JSON on success, but skills-consistency-source-validator.py still prints ok: skill ..., ok: sections ..., or ok: .... Any automation or agent following this contract would parse the source-validation success output incorrectly; split the contract so only runtime validation/inventory are JSON and source validation remains an ok line.

AGENTS.md reference: AGENTS.md:L81-L83

Useful? React with 👍 / 👎.

},
"checks": [
{
Expand Down Expand Up @@ -211,7 +212,7 @@
"installed source_repository_root and installer_version",
"selected install scope"
],
"pass_condition": "Full installation runs full source-repository validation before building; explicit skill installation validates only selected skills. Runtime manifests record schema, skill, runtime_source_id, source_path, source_repository_root, validation_profile, and installer_version. The builder preflights all selected targets, refuses unmanaged folders, links, invalid ownership, and same-name folders owned by another source; a full install removes only stale folders with the same runtime_source_id, while an explicit targeted install removes none. Runtime validation can select one direct manifest-backed skill and compare only its complete managed file tree with canonical builder output.",
"pass_condition": "Full installation runs full source-repository validation before building; explicit skill installation validates only selected skills. Runtime manifests record schema, skill, runtime_source_id, source_path, source_repository_root, validation_profile, and installer_version. The builder preflights all selected targets, refuses unmanaged folders, links, invalid ownership, and same-name folders owned by another source; a full install removes only stale folders with the same runtime_source_id, while an explicit targeted install removes none. Runtime inventory emits compact direct-manifest entries and blockers without auditing; selected runtime validation derives the source repository from the current manifest and compares only that skill's complete managed file tree with canonical builder output.",
"source_lines": [
"skills/ceratops-skill-lifecycle/scripts/runtime/resolve-lifecycle-bundle.py:resolve_bundle",
"skills/ceratops-skill-lifecycle/scripts/runtime/install-managed-skills.py:main",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,13 @@ installed-runtime surface.
scripts/skills-consistency-source-validator.py --repo-root <repo-root>
--mode skill --skill <skill-name>` from the lifecycle bundle.
- (D) Managed runtime validator: `python
scripts/runtime/skills-consistency-runtime-validator.py --repo-root
<repo-root> --skill <skill-name>` from the installed lifecycle bundle.
scripts/runtime/skills-consistency-runtime-validator.py --skill
<skill-name>` from the installed lifecycle bundle; it derives the source
repository from the selected direct runtime manifest.
- (D) Global inventory helper: `python
scripts/runtime/skills-consistency-runtime-validator.py --inventory` emits
compact JSON for every direct manifest-backed installed skill and malformed
direct-manifest blocker without auditing any skill.
- (D) Installer synchronization when repair is authorized: `python
scripts/runtime/synchronize-installers.py --target-repo-root
<task-worktree>` from the installed lifecycle bundle.
Expand Down Expand Up @@ -48,9 +53,8 @@ Infer missing inputs from the repository and installed manifests before asking.

### Global Automation Caller

- The `global-skills-consistency-review` automation owns direct installed-skill
discovery and invokes this action once per direct skill directory containing
`.runtime-manifest.json`.
- The `global-skills-consistency-review` automation runs the global inventory
helper and invokes this action once per valid inventory entry.
- The automation may aggregate results, but it must not deduplicate skills that
share a source repository or make this action perform global discovery.
- Report an unreadable direct runtime manifest as a blocker for that installed
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@
"validation_profile",
"installer_version",
}


def default_runtime_root() -> pathlib.Path:
"""Return the direct personal runtime skills root."""

Expand Down Expand Up @@ -76,6 +78,40 @@ def read_manifest(skill_dir: pathlib.Path) -> tuple[Mapping[str, object] | None,
return (value, None) if isinstance(value, Mapping) else (None, "manifest must be a JSON object")


def routing_manifest_errors(
skill_dir: pathlib.Path,
manifest: Mapping[str, object],
) -> list[str]:
"""Return routing errors that block inventory or source-root derivation."""

errors: list[str] = []
missing = sorted(REQUIRED_FIELDS - set(manifest))
if missing:
return [f"runtime manifest missing {', '.join(missing)}"]
string_fields = ("skill", "runtime_source_id", "source_path", "source_repository_root")
invalid_strings = [
field
for field in string_fields
if not isinstance(manifest.get(field), str) or not str(manifest[field]).strip()
]
if invalid_strings:
errors.append(f"runtime manifest has invalid {', '.join(invalid_strings)}")
if manifest.get("schema") != RUNTIME_MANIFEST_SCHEMA:
errors.append("unsupported runtime manifest schema")
if manifest.get("skill") != skill_dir.name:
errors.append(f"manifest skill identity is {manifest.get('skill')!r}")
if manifest.get("validation_profile") not in VALIDATION_PROFILES:
errors.append("unsupported runtime validation profile")
version = manifest.get("installer_version")
if not isinstance(version, int) or isinstance(version, bool) or version < 1:
errors.append("installer_version must be a positive integer")
source_root = manifest.get("source_repository_root")
if isinstance(source_root, str) and source_root.strip():
if not pathlib.Path(source_root).is_absolute():
errors.append("source_repository_root must be an absolute local path")
return errors


def managed_directories(
runtime_root: pathlib.Path,
selected_skill: str | None = None,
Expand All @@ -92,6 +128,62 @@ def managed_directories(
)


def runtime_inventory(runtime_root: pathlib.Path) -> dict[str, object]:
"""Return compact direct-manifest inventory without auditing any skill."""

skills: list[dict[str, str]] = []
blockers: list[dict[str, object]] = []
for skill_dir in managed_directories(runtime_root):
manifest, error = read_manifest(skill_dir)
if manifest is None:
blockers.append(
{
"directory": skill_dir.name,
"errors": [f"unreadable runtime manifest: {error}"],
}
)
continue
errors = routing_manifest_errors(skill_dir, manifest)
if errors:
blockers.append({"directory": skill_dir.name, "errors": errors})
continue
skills.append(
{
"installed_path": str(skill_dir.resolve()),
"skill": str(manifest["skill"]),
"source_repository_root": str(manifest["source_repository_root"]),
}
)
return {
"blocked": len(blockers),
"blockers": blockers,
"managed": len(skills),
"skills": skills,
"status": "inventory",
}


def repo_root_from_selected_manifest(
runtime_root: pathlib.Path,
selected_skill: str,
) -> tuple[pathlib.Path | None, list[str]]:
"""Derive one selected skill's source repository from its direct manifest."""

if pathlib.PurePath(selected_skill).name != selected_skill:
return None, ["selected skill must be one direct runtime directory name"]
skill_dir = runtime_root / selected_skill
manifest, error = read_manifest(skill_dir)
if manifest is None:
return None, [f"{selected_skill}: unreadable runtime manifest: {error}"]
errors = routing_manifest_errors(skill_dir, manifest)
if errors:
return None, [f"{selected_skill}: {message}" for message in errors]
try:
return pathlib.Path(str(manifest["source_repository_root"])).resolve(), []
except OSError as exc:
return None, [f"{selected_skill}: source_repository_root is unresolved: {exc}"]


def source_context(
repo_root: pathlib.Path,
) -> tuple[Mapping[str, object] | None, str | None, str | None, set[str], list[str]]:
Expand Down Expand Up @@ -170,30 +262,23 @@ def discover_runtime(
root_matches = False
if declared_source_id != source_id and skill_dir.name not in skill_names and not root_matches:
continue
missing = sorted(REQUIRED_FIELDS - set(manifest))
if missing:
errors.append(f"{skill_dir.name}: runtime manifest missing {', '.join(missing)}")
continue
string_fields = ("skill", "runtime_source_id", "source_path", "source_repository_root")
invalid_strings = [field for field in string_fields if not isinstance(manifest.get(field), str) or not str(manifest[field]).strip()]
if invalid_strings:
errors.append(f"{skill_dir.name}: runtime manifest has invalid {', '.join(invalid_strings)}")
continue
if manifest.get("schema") != RUNTIME_MANIFEST_SCHEMA:
errors.append(f"{skill_dir.name}: unsupported runtime manifest schema")
manifest_errors = routing_manifest_errors(skill_dir, manifest)
if manifest_errors:
errors.extend(f"{skill_dir.name}: {message}" for message in manifest_errors)
continue
if manifest.get("runtime_source_id") != source_id:
errors.append(f"{skill_dir.name}: runtime manifest belongs to another source")
continue
if manifest.get("validation_profile") != source_profile:
errors.append(f"{skill_dir.name}: runtime validation profile does not match source")
continue
version = manifest.get("installer_version")
if not isinstance(version, int) or isinstance(version, bool) or version < 1:
errors.append(f"{skill_dir.name}: installer_version must be a positive integer")
try:
declared_root = pathlib.Path(str(manifest["source_repository_root"])).resolve()
except OSError as exc:
errors.append(f"{skill_dir.name}: source_repository_root is unresolved: {exc}")
continue
if not pathlib.Path(str(manifest["source_repository_root"])).is_absolute():
errors.append(f"{skill_dir.name}: source_repository_root must be an absolute local path")
if declared_root != repo_root:
errors.append(f"{skill_dir.name}: source_repository_root does not match repository")
continue
records.append({"directory": skill_dir.name, **dict(manifest)})

Expand Down Expand Up @@ -317,12 +402,33 @@ def main() -> int:
parser = argparse.ArgumentParser(
description="Validate managed runtime skills for one source repository or selected skill."
)
parser.add_argument("--repo-root", required=True, type=pathlib.Path)
parser.add_argument("--repo-root", type=pathlib.Path)
parser.add_argument("--runtime-root", type=pathlib.Path)
parser.add_argument("--skill", help="Validate only this direct manifest-backed runtime skill.")
parser.add_argument(
"--inventory",
action="store_true",
help="Emit compact direct-manifest inventory without auditing skills.",
)
args = parser.parse_args()
repo_root = args.repo_root.resolve()
runtime_root = (args.runtime_root or default_runtime_root()).resolve()
if args.inventory:
if args.repo_root is not None or args.skill is not None:
parser.error("--inventory cannot be combined with --repo-root or --skill")
print(json.dumps(runtime_inventory(runtime_root), sort_keys=True))
return 0
if args.repo_root is None and args.skill is None:
parser.error("--repo-root is required unless --skill or --inventory is used")
derivation_errors: list[str] = []
if args.repo_root is not None:
repo_root = args.repo_root.resolve()
else:
repo_root, derivation_errors = repo_root_from_selected_manifest(runtime_root, args.skill)
if repo_root is None:
print(f"errors: {len(derivation_errors)}", file=sys.stderr)
for error in derivation_errors:
print(error, file=sys.stderr)
return 1
records, errors, source_manifest, source_id, skill_names = discover_runtime(
repo_root,
runtime_root,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -866,6 +866,7 @@ def check_validation_command_surface() -> list[str]:
if (
"compare_managed_tree" not in runtime_validator_text
or '"--skill"' not in runtime_validator_text
or '"--inventory"' not in runtime_validator_text
or '"--mode"' in runtime_validator_text
):
errors.append("runtime validator must compare managed trees through one command")
Expand Down
Loading
Loading