Skip to content

Collection catalogs - YAML schema validation, and authoring tooling#57

Merged
dmartinol merged 10 commits into
RHEcosystemAppEng:mainfrom
dmartinol:catalog-add-on
Apr 14, 2026
Merged

Collection catalogs - YAML schema validation, and authoring tooling#57
dmartinol merged 10 commits into
RHEcosystemAppEng:mainfrom
dmartinol:catalog-add-on

Conversation

@dmartinol

Copy link
Copy Markdown
Collaborator

Summary

This branch introduces a machine-readable catalog per pack under /.catalog/ (collection.yaml plus a deterministic collection.json mirror and supporting fragments such as deploy_and_use.md), aligned with COLLECTION_SPEC.md and catalog/schema.yaml.

What changed

  • Catalog artifacts (collection,yaml) generated from golden sources (skills, README, CLAUDE, marketplace) without overwriting those sources.
  • Validation pipeline: collection schema checks, compliance checks, and wiring in Makefile / existing validation so CI can enforce catalog shape and roster consistency.
  • Scripts: catalog bootstrap/scaffold, YAML→JSON mirroring, shared validation library, and pack_registry integration with existing tooling (e.g. generate_pack_data.py updates).
  • Contributor docs: updates to README.md, docs/README.md, and root CLAUDE.md describing the catalog workflow and validation.
  • Agent skills under .claude/skills/: create-collection and collection-compliance to author or refresh catalogs and fix validation failures.
  • Repo hygiene: .pre-commit-config.yaml, small pyproject.toml / uv.lock updates, and a minor GitHub Pages workflow adjustment.

Pack(s) affected

  • all
  • rh-sre
  • rh-developer
  • ocp-admin
  • rh-virt
  • rh-ai-engineer
  • Other / repo-wide

Change type

  • New skill
  • New agent
  • New pack
  • Update existing skill / agent
  • MCP server config (mcps.json)
  • Docs / README
  • CI / tooling

CLAUDE.md compliance

  • Agents orchestrate skills; no direct MCP/tool calls in agents
  • Skills are single-purpose task executors
  • Skills encapsulate all tool access (MCP tools invoked only inside skills)
  • Document consultation: file is read with the Read tool, then declared to the user
  • No credentials hardcoded; env vars used via ${VAR} references
  • Human-in-the-loop confirmation added for any destructive or critical operations

Validation

  • make validate passes locally
  • New/changed skills have valid YAML frontmatter (name, description)
  • New/changed agents have valid YAML frontmatter (name, description)

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
…YAML schema for catalog validation

- Replaced `catalog/collection.schema.json` with `catalog/schema.yaml` for JSON Schema in YAML format.
- Updated references in `CLAUDE.md`, `COLLECTION_SPEC.md`, and various skill documentation to point to the new YAML schema.
- Enhanced validation scripts to accommodate the new schema format and enforce inline length limits for documentation.
- Added new documentation fragments for `mcp_section`, `security_model`, and `documentation_section` to improve clarity on usage and configuration.

This change streamlines the catalog validation process and improves documentation consistency across the repository.

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
@github-actions

Copy link
Copy Markdown

Gemini Code Review

1. Summary

This PR introduces a comprehensive collection catalog specification (COLLECTION_SPEC.md) and schema (catalog/schema.yaml) to standardize metadata across all agentic packs. It bootstraps .catalog/ directories (YAML and JSON mirrors) for all existing packs, adds pre-commit hooks for validation, and introduces two new repository-level skills (collection-compliance and create-collection) to help maintainers and AI assistants manage catalog drift.

2. Project Rules Compliance

There are a few violations of the strict skill design principles in the newly added repository skills (.claude/skills/):

  • Mandatory Section Headings:
    • In both collection-compliance/SKILL.md and create-collection/SKILL.md, the heading ## When to Use is used. Per Principle Rh virt/snapshots skills #7, this must be exactly ## When to Use This Skill.
    • Similarly, ## Example usage should be capitalized as ## Example Usage.
  • Skill Invocation Format:
    • In collection-compliance/SKILL.md, the create-collection skill is referenced multiple times without the mandatory slash format. Per Principle Rh developer/human in the loop #4, it should be referenced as /create-collection.
  • Human-in-the-Loop (HitL):
    • The /create-collection skill modifies and writes files (<pack>/.catalog/collection.yaml). Per Principle Rh virt/agent init #6, skills performing create/modify operations MUST include a ## Critical: Human-in-the-Loop Requirements section requiring explicit confirmation before writing to disk.
  • Build Reminder: Since new skills were added and structural validation rules were updated, please ensure you run make validate and make validate-skill-design-changed locally before merging.

3. Code Quality Issues

  • Typo in Directory Path:
    • file: .claude/skills/collection-compliance/SKILL.md:49
    • The dependency lists - create-collection skill (sibling under .cursor/skills/). This should be .claude/skills/ as .cursor/skills/ does not exist in this PR.

4. Suggestions

  • Schema Completeness: In rh-developer/.catalog/collection.yaml (and others), you are using fields like documentation_section_file, mcp_section_file, and security_model_file. While catalog/schema.yaml allows this because additionalProperties: true is set at the root, you might want to explicitly define these *_file properties in the JSON schema for better IDE autocompletion and stricter validation.
  • Pre-commit Hook Scope: The new .pre-commit-config.yaml runs make validate-skill-design-changed on ^(rh-[-a-z0-9]+|ocp-admin)/skills/[^/]+/SKILL\.md$. You may want to consider adding ^\.claude/skills/[^/]+/SKILL\.md$ to this regex so that your new repository-level skills are also linted for design compliance.

5. Verdict

REQUEST_CHANGES — Please fix the mandatory section headings, add the missing HitL section to the file-modifying skill, and correct the .cursor/skills/ typo. Excellent work on the catalog standardization otherwise!


Automated review by Gemini 3.1 Pro | Rules: CLAUDE.md, SKILL_DESIGN_PRINCIPLES.md

…fragment references

- Updated `COLLECTION_SPEC.md` to clarify the usage of inline markdown and fragment references for `deploy_and_use`, `documentation_section`, `mcp_section`, and `security_model`.
- Modified validation scripts to reject legacy `*_file` keys and enforce the new reference structure.
- Adjusted schema definitions in `catalog/schema.yaml` to reflect the changes in documentation structure.
- Updated collection files in `rh-developer` and `rh-sre` to replace `*_file` keys with direct references.

This change enhances consistency in documentation and validation processes across the repository.

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
@dmartinol

Copy link
Copy Markdown
Collaborator Author
  • Pre-commit Hook Scope: The new .pre-commit-config.yaml runs make validate-skill-design-changed on ^(rh-[-a-z0-9]+|ocp-admin)/skills/[^/]+/SKILL\.md$. You may want to consider adding ^\.claude/skills/[^/]+/SKILL\.md$ to this regex so that your new repository-level skills are also linted for design compliance.

Great hint but out of scope for the current PR.
We will track an issue to remind it.

@dmartinol dmartinol requested a review from r2dedios April 14, 2026 08:07
dmartinol and others added 2 commits April 14, 2026 11:24
- Updated `COLLECTION_SPEC.md` to clarify machine validation requirements and the need for AI semantic alignment reviews against golden sources.
- Revised `SKILL.md` to emphasize the importance of semantic alignment and added new drift conditions for installation and environment variables.
- Modified `deploy_and_use.md` files across multiple modules to replace Claude Code installation instructions with Lola commands for consistency.

These changes improve the clarity and accuracy of documentation, ensuring better compliance with validation standards and installation procedures.

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>

@r2dedios r2dedios left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job :) Just have a few questions to check

Comment thread rh-support-engineer/.catalog/collection.json Outdated
Comment thread rh-ai-engineer/.catalog/deploy_and_use.md
Comment thread rh-support-engineer/.catalog/collection.yaml Outdated
Comment thread rh-support-engineer/.catalog/collection.yaml Outdated
Comment thread rh-virt/.catalog/deploy_and_use.md
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
- Added information about the `RHOAI_MCP_TRANSPORT` environment variable in both `README.md` and `deploy_and_use.md`, clarifying its default setting and export requirements.

Signed-off-by: Daniele Martinoli <dmartino@redhat.com>
@dmartinol dmartinol merged commit 2e53d8f into RHEcosystemAppEng:main Apr 14, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants