test: smoke-test-skill (skill-audit v6 e2e)#10
Conversation
- github actions workflow for eval runs on skill preview PRs - 2027.config.yaml extends skill-audit with org rules (mjs-only scripts) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Designed to trigger skill-audit rules end-to-end: - vague description (CONTENT-005) - ALL-CAPS imperatives without rationale (CONTENT-104) - no "do not use when" guidance (CONTENT-070) - README.md at skill root (HYGIENE-030, error) - LICENSE.txt at skill root (HYGIENE-020, warn)
![2027-evals[bot]](https://avatars.githubusercontent.com/in/3339941?s=60&v=4){kind=small}
| @@ -0,0 +1,28 @@ | |||
| --- | |||
| name: smoke-test-skill | |||
| description: A skill that does various things. | |||
There was a problem hiding this comment.
🟡 The description 'A skill that does various things' provides no specific information about what artifact or side-effect the skill produces. It is vague and describes the skill's purpose in general terms without clarifying any concrete output, return value, file path, format, deployed service, or committed change.
The entire description consists of only 'A skill that does various things.' which lacks any specificity about outputs or artifacts.
| @@ -0,0 +1,28 @@ | |||
| --- | |||
| name: smoke-test-skill | |||
| description: A skill that does various things. | |||
There was a problem hiding this comment.
🟡 The description lacks specific contexts or triggers for when to use the skill.
The description only states 'A skill that does various things.' It provides no information about when to use it or what specific contexts trigger its use.
| @@ -0,0 +1,28 @@ | |||
| --- | |||
| name: smoke-test-skill | |||
| description: A skill that does various things. | |||
There was a problem hiding this comment.
🟡 Description lacks both specific action names and concrete trigger keywords. 'Does various things' is vague and non-descriptive.
"A skill that does various things." - No action/capability named, no trigger words or use-case keywords provided.
| @@ -0,0 +1,3 @@ | |||
| # smoke-test-skill | |||
There was a problem hiding this comment.
🔴 forbidden files present
README.md
|
2027 auto-runs evals against preview deployments of your docs. To enable this, install one of:
Once a preview is deployed, open a new PR and we'll run the eval automatically. Evaluating agent experience using 2027.dev · View dashboard |
1 participant
End-to-end smoke for skill-audit v6 via local queue → staging neon → PR comments.
Adds a single SKILL.md with intentional violations so the audit produces real
findings to comment on inline:
Do not merge.