From 2a422e594c4d2df77937aa1b6b23ffcf449c4c65 Mon Sep 17 00:00:00 2001 From: Shannon Tran <12804177+howshannon@users.noreply.github.com> Date: Tue, 28 Jul 2026 11:13:00 -0700 Subject: [PATCH] Add deterministic Slop Cop report cards --- .github/workflows/security.yml | 1 + Makefile | 1 + README.md | 78 +++++++++--- benchmark/cases.yml | 12 +- scripts/test_ticket_score.py | 40 +++++++ skills/slop-cop/SKILL.md | 58 ++++----- skills/slop-cop/agents/openai.yaml | 8 +- .../references/calibration-anchors.md | 73 +++++------- skills/slop-cop/references/prose-examples.md | 4 +- skills/slop-cop/references/report-template.md | 58 ++++----- .../slop-cop/references/ticket-slogans.json | 66 +++++++++++ skills/slop-cop/references/tickets.md | 69 +++++++++++ skills/slop-cop/scripts/score_ticket.py | 112 ++++++++++++++++++ 13 files changed, 445 insertions(+), 135 deletions(-) create mode 100644 scripts/test_ticket_score.py create mode 100644 skills/slop-cop/references/ticket-slogans.json create mode 100644 skills/slop-cop/references/tickets.md create mode 100644 skills/slop-cop/scripts/score_ticket.py diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml index 5e22ce3..9f2901a 100644 --- a/.github/workflows/security.yml +++ b/.github/workflows/security.yml @@ -37,6 +37,7 @@ jobs: run: | python3 scripts/validate_repo.py python3 scripts/validate_cases.py + python3 scripts/test_ticket_score.py - name: Run static security checks shell: bash diff --git a/Makefile b/Makefile index c184d2a..35601e1 100644 --- a/Makefile +++ b/Makefile @@ -4,6 +4,7 @@ check: validate security benchmark validate: python3 scripts/validate_repo.py + python3 scripts/test_ticket_score.py security: bash scripts/security_scan.sh . diff --git a/README.md b/README.md index 15c2982..7f11216 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ The skill loads only the references needed for the material under review. ### Audit without rewriting ```text -$slop-cop Audit this draft. List the evidence, severity, and recommended fixes without rewriting it. +$slop-cop Audit this draft without rewriting it. ``` ### Grade a post @@ -115,27 +115,65 @@ $slop-cop Review this landing page for generic generated-UI defaults, unsupporte $slop-cop Review this diff for happy-path-only logic, swallowed failures, unsafe boundaries, unnecessary abstractions, and missing tests. ``` -Compatible agents may select Slop Cop when a request matches its description. Explicit invocation is the most reliable way to request a particular review. +### Slop Cop this -## Output +```text +$slop-cop This. +``` -Slop Cop supports three modes: +Every request returns a Slop Cop report-card ticket: a deterministic 0–100 +score, letter grade, evidence-backed charge, fixed slogan, and recommended next +step. It targets the work, not the person. It does not message anyone unless +you explicitly ask it to send the ticket through a connected service. -- **Audit:** findings, quoted evidence, severity, and fixes -- **Grade:** a scored review using calibration anchors -- **Rewrite:** the smallest useful revision, followed by a self-audit +Compatible agents may select Slop Cop when a request matches its description. Explicit invocation is the most reliable way to request a particular review. -For prose, it scores five dimensions from 1–10: +## Output -| Dimension | Review question | -|---|---| -| Directness | Does it state the point without empty setup or stacked hedging? | -| Specificity | Are material claims concrete and supportable? | -| Rhythm | Are sentence and paragraph shapes varied for a reason? | -| Voice | Does it preserve a consistent, identifiable voice? | -| Density | Is every remaining sentence doing useful work? | +Slop Cop supports four modes, but the report-card ticket is always included: -A score below 35 normally needs revision. The score is an editorial aid, not an authorship detector or a substitute for judgment. +- **Audit:** emphasize findings, quoted evidence, severity, and fixes +- **Grade:** emphasize the score and penalty arithmetic +- **Rewrite:** the smallest useful revision, followed by a self-audit +- **Ticket:** emphasize the funny, evidence-backed citation + +Scoring starts at 100 and uses fixed deductions: + +| Violation | Deduction | +|---|---:| +| Blocker | −30 | +| Major | −15 | +| Minor | −5 | +| Note | −1 | + +Each violation is counted once at its highest severity. The report always shows +the arithmetic, so identical findings produce the same score. + +| Grade | Score | Grade | Score | +|---|---:|---|---:| +| A+ | 97–100 | C | 73–76 | +| A | 93–96 | C- | 70–72 | +| A- | 90–92 | D+ | 67–69 | +| B+ | 87–89 | D | 63–66 | +| B | 83–86 | D- | 60–62 | +| B- | 80–82 | F | 0–59 | +| C+ | 77–79 | | | + +The scale runs from **0 = total slop** to **100 = damn, you're not a +robot?** Scores below C+ draw from 30 fixed bad-grade slogans; C+ and above +draw from 30 fixed good-grade slogans. Selection is rules-based, not random, +and Slop Cop never invents a new slogan. The joke is not an authorship claim: +the score measures observable quality problems only. + +Recommended next steps are fixed by score band: + +- 97–100: ship +- 90–96: tweak optional notes +- 80–89: tweak minor violations +- 77–79: revise the highest-impact violation +- 70–76: revise material violations +- 60–69: rewrite affected sections +- 0–59: start over from the strongest factual core
@@ -176,7 +214,7 @@ slop-cop/
│ └── openai.yaml
├── assets/
│ └── brand/
- └── references/
+ ├── references/
├── calibration-anchors.md
├── code.md
├── design.md
@@ -184,7 +222,11 @@ slop-cop/
├── prose-phrases.md
├── prose-structures.md
├── report-template.md
- └── research-sources.md
+ ├── research-sources.md
+ ├── ticket-slogans.json
+ └── tickets.md
+ └── scripts/
+ └── score_ticket.py
```
## Development
diff --git a/benchmark/cases.yml b/benchmark/cases.yml
index 41d5ec8..7cbf20d 100644
--- a/benchmark/cases.yml
+++ b/benchmark/cases.yml
@@ -8,7 +8,7 @@ cases:
opportunity is not to move faster. It is to build a robust foundation
that actually scales. That is the point.
expected_verdict: REVISE
- expected_score_range: [10, 24]
+ expected_score_range: [40, 76]
must_find:
- vague quantifier
- repeated antithesis
@@ -24,7 +24,7 @@ cases:
plain English instead of suspicious shell commands, which is why I kept
the manual review step.
expected_verdict: PASS
- expected_score_range: [40, 49]
+ expected_score_range: [90, 100]
must_preserve:
- all numbers
- Sunday
@@ -37,7 +37,7 @@ cases:
to private enterprise code. A larger replication would narrow that
uncertainty.
expected_verdict: PASS
- expected_score_range: [38, 48]
+ expected_score_range: [90, 100]
must_not_find:
- empty hedging
must_preserve:
@@ -50,7 +50,7 @@ cases:
are doing security wrong.” I removed it because it made an unsupported
claim before presenting the scan results.
expected_verdict: PASS
- expected_score_range: [38, 48]
+ expected_score_range: [90, 100]
must_preserve:
- quoted text
@@ -61,7 +61,7 @@ cases:
streamlines the busywork, and turns evolving priorities into actionable
next steps. I'm curious how far this pivotal shift can go.
expected_verdict: REVISE
- expected_score_range: [12, 26]
+ expected_score_range: [40, 76]
must_find:
- marketing verbs
- empty tic words
@@ -74,7 +74,7 @@ cases:
team struck a delicate balance, applying surgical precision to every
detail so the whole thing feels effortless.
expected_verdict: REVISE
- expected_score_range: [12, 26]
+ expected_score_range: [40, 76]
must_find:
- inflated symbolism
- false-precision adjective
diff --git a/scripts/test_ticket_score.py b/scripts/test_ticket_score.py
new file mode 100644
index 0000000..df35e54
--- /dev/null
+++ b/scripts/test_ticket_score.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+import sys
+
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.dont_write_bytecode = True
+MODULE_PATH = ROOT / "skills" / "slop-cop" / "scripts" / "score_ticket.py"
+SPEC = importlib.util.spec_from_file_location("score_ticket", MODULE_PATH)
+assert SPEC and SPEC.loader
+score_ticket = importlib.util.module_from_spec(SPEC)
+SPEC.loader.exec_module(score_ticket)
+
+assert score_ticket.score(0, 0, 0, 0) == 100
+assert score_ticket.score(4, 0, 0, 0) == 0
+assert score_ticket.score(0, 1, 2, 0) == 75
+
+expected_boundaries = {
+ 100: "A+", 97: "A+", 96: "A", 93: "A", 92: "A-", 90: "A-",
+ 89: "B+", 87: "B+", 86: "B", 83: "B", 82: "B-", 80: "B-",
+ 79: "C+", 77: "C+", 76: "C", 73: "C", 72: "C-", 70: "C-",
+ 69: "D+", 67: "D+", 66: "D", 63: "D", 62: "D-", 60: "D-",
+ 59: "F", 0: "F",
+}
+for value, expected in expected_boundaries.items():
+ assert score_ticket.letter_grade(value) == expected
+
+slogans = score_ticket.load_slogans()
+assert len(slogans["bad"]) == 30
+assert len(slogans["good"]) == 30
+assert len(set(slogans["bad"])) == 30
+assert len(set(slogans["good"])) == 30
+assert score_ticket.slogan(75, 0, 1, 2, 0) == score_ticket.slogan(75, 0, 1, 2, 0)
+assert score_ticket.slogan(76, 0, 0, 0, 24) in slogans["bad"]
+assert score_ticket.slogan(77, 0, 1, 1, 3) in slogans["good"]
+
+print("Deterministic ticket scoring, grades, and 60 fixed slogans validated.")
diff --git a/skills/slop-cop/SKILL.md b/skills/slop-cop/SKILL.md
index 4b6b47d..6931b92 100644
--- a/skills/slop-cop/SKILL.md
+++ b/skills/slop-cop/SKILL.md
@@ -1,17 +1,17 @@
---
name: slop-cop
description: >-
- Audit, grade, and rewrite generic AI-slop patterns in prose, social posts,
- UI/design, and code. Use when asked to de-slop writing, humanize AI copy,
- grade a draft, review a landing page or generated interface, remove vague
- claims and repetitive rhetoric, or inspect AI-written code for generic
- abstractions and happy-path-only logic. Do not use to determine whether AI
+ Audit, grade, rewrite, and issue funny evidence-backed tickets for generic
+ AI-slop patterns in prose, social posts, UI/design, and code. Use when asked
+ to de-slop writing, humanize AI copy, grade a draft, review a landing page or
+ generated interface, remove vague claims and repetitive rhetoric, inspect
+ AI-written code for generic abstractions and happy-path-only logic, or ticket
+ someone for a specific quality offense. Do not use to determine whether AI
authored something.
license: MIT
-compatibility: Works in Agent Skills-compatible environments that support Markdown references.
metadata:
author: howshannon
- version: "0.3.0"
+ version: "0.4.0"
---
# Slop Cop
@@ -41,9 +41,15 @@ proof of AI authorship.
## Choose a mode
- **Audit:** list findings, evidence, severity, and fixes without rewriting.
-- **Grade:** score the requested beat and explain the score.
+- **Grade:** emphasize the score and its arithmetic.
- **Rewrite:** make the smallest changes that remove the problems; then audit
the rewrite before returning it.
+- **Ticket:** emphasize one funny, shareable citation.
+
+Every mode, including a bare “slop cop this,” ends with the same scored
+report-card ticket. Read [report-card tickets](references/tickets.md) and use
+`scripts/score_ticket.py`; never improvise the score, grade, slogan, or next
+step.
Route only to the references needed:
@@ -55,6 +61,7 @@ Route only to the references needed:
- Scoring → [calibration anchors](references/calibration-anchors.md) and
[report template](references/report-template.md).
- Evidence claims → [research sources](references/research-sources.md).
+- Funny citations → [ticket mode](references/tickets.md).
## The two laws
@@ -136,29 +143,26 @@ before deciding severity.
facts rather than inventing them.
6. Re-run the tally and verify facts, quotations, links, numbers, and tone.
-## Grading prose
-
-Score five dimensions from 1–10, total 50. A score below 35 normally needs
-revision.
+## Scoring every review
-| Dimension | Question |
-|---|---|
-| Directness | Does it state the point without empty setup or stacked hedging? |
-| Specificity | Are material claims concrete and supportable? |
-| Rhythm | Are sentence and paragraph shapes varied for a reason? |
-| Voice | Does it preserve a consistent, identifiable human voice? |
-| Density | Is every remaining sentence doing useful work? |
+Start at 100. Deduct 30 per blocker, 15 per major, 5 per minor, and 1 per note,
+then clamp at zero. Count each observable violation once at its highest
+applicable severity. Show the violation counts and full arithmetic so the same
+findings always produce the same score.
-Apply caps only when the tally supports them:
+Map the result to the fixed A+ through F bands in
+[report-card tickets](references/tickets.md). Scores under C+ use one of the 30
+fixed bad-grade slogans; C+ and above use one of the 30 fixed good-grade
+slogans. Select it with the documented formula—never generate a new one.
-- One device used 3+ times: Rhythm may not exceed 5.
-- One device used 5+ times, or two device classes used 3+ times: Rhythm and
- Voice may not exceed 4.
-- Fabricated attribution, quote, statistic, or experience: overall result is a
- blocker regardless of total.
+The displayed scale is `0 = total slop · 100 = damn, you're not a robot?`.
+This is humorous editorial shorthand only. The result measures observable
+quality problems and must never be presented as authorship detection.
-Report the tally, quoted evidence, biggest issue, confidence, score, and the
-single highest-leverage change. Use the report template.
+Always report the score, letter grade, charge, evidence, penalty arithmetic,
+fixed citation, recommended next step, and smallest useful fix. Use the
+[report template](references/report-template.md). A clean review still gets a
+100/A+ report card with “No citation-worthy violations found.”
## Design and code
diff --git a/skills/slop-cop/agents/openai.yaml b/skills/slop-cop/agents/openai.yaml
index eb7c967..77420d3 100644
--- a/skills/slop-cop/agents/openai.yaml
+++ b/skills/slop-cop/agents/openai.yaml
@@ -1,12 +1,12 @@
interface:
display_name: "Slop Cop"
- short_description: "Audit and fix generic AI-slop patterns."
+ short_description: "Score, fix, and ticket generic slop."
icon_small: "./assets/brand/slop-cop-emblem.png"
icon_large: "./assets/brand/slop-cop-emblem.png"
- brand_color: "#FF2E88"
+ brand_color: "#FFAE4D"
default_prompt: >-
- Run Slop Cop on this material. Return the verdict, score or severity,
- evidence tally, biggest offense, highest-impact fix, and revised version.
+ Use $slop-cop to return a deterministic 0–100 report-card ticket, evidence,
+ and the recommended next step for this material.
policy:
allow_implicit_invocation: true
diff --git a/skills/slop-cop/references/calibration-anchors.md b/skills/slop-cop/references/calibration-anchors.md
index 3e30d2a..bf259a7 100644
--- a/skills/slop-cop/references/calibration-anchors.md
+++ b/skills/slop-cop/references/calibration-anchors.md
@@ -1,43 +1,30 @@
-# Calibration Anchors
-
-Reference scores from real posts, graded and corrected by a strict human editor. Use them in Step 4 of scoring: find the band your piece resembles by device-tally and specificity, and land near it. These exist because abstract rules let a grader drift generous; concrete anchors don't.
-
-The pattern to internalize: **specificity keeps a post off the floor, but heavy device-density holds it in the low-to-mid 20s no matter how specific it is. Only specific AND structurally clean writing clears 35.**
-
-## ~11–12, Floor. Pure abstraction, wall-to-wall scaffolding.
-
-No names, numbers, or first-hand examples. Built entirely from negation-reversals, fragment stacks, anaphora, and one-line-punch paragraphs. Reads as a sermon.
-
-- Signature tally: 6+ reversals, 2+ tricolons, most paragraphs a single punchy line, a hyperbole hook, zero specifics.
-- Examples: a leadership post that opens "Everyone is talking about AI" and closes "we're just getting started"; a hymn to delivery leadership ("Vision inspires. Execution transforms.") with no project named.
-- Specificity 2, Rhythm 2, Voice 3. There is nothing under the rhetoric, so it can't be edited into shape, it needs a real story.
-
-## ~19–21, Real examples, but buried under dense scaffolding.
-
-Has a few concrete details, but 6–8 reversals plus announced-insight labels ("The Insight:"), branded title-case examples ("The Fridge Audit"), and tricolons dominate.
-
-- Signature tally: 6–8 reversals, announced insights, title-case labels, a hyperbole hook; specificity ~5–6.
-- Examples: an "AI-native associates" post with four genuine example use-cases smothered by reversal-cadence; a Fable 5 reflection with real capability details wrapped in "not X, it's Y" takeaways and a curiosity-gap CTA.
-- Rhythm capped 3, Voice capped 4. The examples are worth keeping; the scaffolding is the whole fix.
-
-## ~23–25, Genuinely specific, still templated.
-
-Named people, real numbers, a vivid anecdote, wrapped in the "I sat down with X, here are N punchy takeaways" template where each takeaway is a reversal.
-
-- Signature tally: ~6 reversals/antitheses, fragment stacks of stats, some tic-words; specificity 7–8 (this is real and load-bearing).
-- Examples: a robots-drawing demo post (the wobbly-city and "we've sold them a screensaver" lines are excellent) held down by ~4 concrete binaries and mid-thought fragments; a Google Next recap with Aaron Reinitz named, real stats, and the grocer-headshot anecdote, dragged into the mid-20s by six antitheses.
-- The lesson these two teach: specificity 8 does NOT rescue the score. Rhythm and Voice stay capped. This is where a grader is most tempted to over-score, don't.
-
-## ~38–41, Passing. Specific AND clean.
-
-Concrete, checkable detail AND clean scaffolding: at most one or two deliberate devices, distinct casual voice, load-bearing specifics.
-
-- Signature tally: 0–2 reversals total, no fragment-stack texture, no hyperbole hook, no announced insights, real first-hand detail, a voice nobody would generate.
-- Examples: a security-tool post with concrete numbered steps and a self-aware "ps, posting on a Sunday evening" sign-off; a video-tips post with genuinely expert specifics (back cameras beat front, a LUT for warmth, save a preset in Slate) and casual voice ("I yapped about," "it's a slog!").
-- Note: both use light parallelism and a how-to list, but the lists are functional and the specifics are real, so the parallelism is structure, not slop. This is the boundary: a functional list with real content passes; a reversal used as the connective tissue does not.
-
-## How to use these
-
-1. Tally your piece (Step 1).
-2. Find the band whose signature tally and specificity level match yours.
-3. If your instinct says "but this one is better," check: does it actually have fewer devices, or just nicer individual lines? Nicer lines don't move the band. Device count and real specificity do.
+# Deterministic calibration anchors
+
+Scores follow fixed penalties rather than holistic impressions:
+
+- Blocker: −30
+- Major: −15
+- Minor: −5
+- Note: −1
+- Floor: 0
+
+Count one observable problem once at the highest applicable severity. Similar
+wording in separate locations may be one repeated-pattern major rather than
+several local minors. Distinct problems receive distinct deductions.
+
+## Anchors
+
+| Findings | Math | Score | Grade | Typical action |
+|---|---|---:|---|---|
+| None | 100 | 100 | A+ | Ship |
+| 2 notes | 100 − 2 | 98 | A+ | Ship |
+| 1 minor | 100 − 5 | 95 | A | Tweak |
+| 1 major | 100 − 15 | 85 | B | Tweak |
+| 1 major + 2 minors | 100 − 15 − 10 | 75 | C | Revise |
+| 2 majors + 2 minors | 100 − 30 − 10 | 60 | D- | Rewrite sections |
+| 1 blocker + 1 major + 2 minors | 100 − 30 − 15 − 10 | 45 | F | Start over from facts |
+| 4 blockers | max(0, 100 − 120) | 0 | F | Start over |
+
+Specificity does not grant bonus points. It may prevent a violation, but it
+does not offset a separate repeated structure, unsupported claim, or failure
+mode. Preserve strong details while fixing only the cited violations.
diff --git a/skills/slop-cop/references/prose-examples.md b/skills/slop-cop/references/prose-examples.md
index e452e11..4d665f2 100644
--- a/skills/slop-cop/references/prose-examples.md
+++ b/skills/slop-cop/references/prose-examples.md
@@ -78,7 +78,7 @@ This is the miss that instance-by-instance review makes. Each line below is fine
Count the negation-reversals and antitheses: seven-plus in one post, plus fragment stacks ("One room. Off the record. Two days.") and a punchline ending on nearly every paragraph. No single line is wrong. The *cadence* is the tell, one rhythm dressed as variety.
-- Diagnosis: Rhythm capped at 3, Voice capped at 4 (reads as generic thought-leader, not a specific person), Specificity genuinely high (real named speakers, a headcount, the oat/almond/coconut example) at 7. Total lands ~18/50. Slop, even though every fact is true.
+- Diagnosis: Specific details are worth preserving, but the repeated reversal cadence is one major violation and the local fragment/tic-word problems are minors. Apply the fixed deductions and show the arithmetic; true facts do not cancel separate violations.
- Fix: keep the two strongest reversals (the predict-vs-adapt line and the closing thesis, they *are* the argument), rewrite the other five as plain statements, let three paragraphs end without a punch, and add one long breathing sentence. Protect the names and concrete examples; cut the reversal-cadence.
@@ -101,7 +101,7 @@ What's slop, regardless of that specificity:
- **All-negation paragraph:** the entire "Robots embarrass that whole posture…" block.
- **Recurring personal tic:** "I left with more questions than I had going in" (reused from an earlier post).
-Diagnosis: Specificity is genuinely high (7), but Directness (5), Rhythm (3), Voice (4), and Density (4) all suffer because the frame is machine-made. Total ~23/50. Slop.
+Diagnosis: Specificity is strong, but the repeated frame is a major violation and the local density problems are minors. Apply the fixed deductions and show the arithmetic; specificity prevents an unsupported-claim violation but does not award bonus points.
The fix keeps the wiring-diagram city, the depressed Wednesday portrait, and the screensaver line, and rewrites the scaffolding: drop the hook and open on the robots, rejoin the fragments, cut "real" and "is the point," and turn the all-negation paragraph into a plain statement of what the physical test exposed.
diff --git a/skills/slop-cop/references/report-template.md b/skills/slop-cop/references/report-template.md
index 23159c6..a59b8a6 100644
--- a/skills/slop-cop/references/report-template.md
+++ b/skills/slop-cop/references/report-template.md
@@ -1,42 +1,30 @@
# Report template
-## Verdict
-
-PASS | PASS WITH EDITS | REVISE | BLOCKER
-
-## Scope and confidence
-
-- Beat:
-- Mode:
-- Confidence:
-- Exclusion zones:
-- Limits:
-
## Findings
| Severity | Evidence/location | Observable pattern | Why it matters | Recommended change |
|---|---|---|---|---|
-## Device tally (prose only)
-
-- Antithesis / negation reversal:
-- Tricolon:
-- Fragment or punchline stack:
-- Vague authority/hyperbole hook:
-- Label-colon construction:
-- Empty tic word:
-- Repeated paragraph shape:
-- Formatting/punctuation density:
-
-## Score (grade mode)
-
-- Directness: /10
-- Specificity: /10
-- Rhythm: /10
-- Voice: /10
-- Density: /10
-- **Total: /50**
-
-## Highest-leverage change
-
-One concrete change, not a general aspiration.
+For prose, also tally repeated devices. For design and code, tally the
+corresponding observable failures. Count each violation once at its highest
+severity.
+
+## Required report-card ticket
+
+```text
+🚨 SLOP COP REPORT CARD
+To: