Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/thin-sides-fail.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/intent': patch
---

enhance SKILL.md with detailed guidance on leveraging GitHub issues and discussions for skill improvement
86 changes: 71 additions & 15 deletions packages/intent/meta/domain-discovery/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,13 +258,57 @@ sample a subset and extrapolate.
3. **API reference** β€” scan for exports, type signatures, option shapes
4. **Changelog for major versions** β€” API renames, removed exports,
behavioral changes
5. **GitHub issues and discussions** β€” scan for frequently reported
confusion, common misunderstandings, recurring questions. Also look
for what users are implicitly arguing for architecturally β€” not just
"people are confused about X" but "users keep expecting X to work
like Y, which reveals a tension between [design force] and [design force]."
If no web access, check for FAQ.md, TROUBLESHOOTING.md, or docs/faq
as proxies.
5. **GitHub issues and discussions** β€” this is one of the highest-yield
sources for failure modes and skill content. Docs describe intended
behavior; issues reveal actual behavior and real developer confusion.

**How to search.** Use `gh search issues` and `gh search prs` (or the
GitHub web search UI) against the library's repo. Run multiple passes:
- **High-engagement issues:** sort by reactions or comments to find the
problems that affect the most developers. These are skill-worthy
even if already fixed β€” agents trained on older data still hit them.
- **Label-based scans:** look for labels like `bug`, `question`,
`documentation`, `breaking-change`, `good first issue`, `FAQ`,
`help wanted`. Each label category yields different signal:
- `bug` + `closed` β†’ failure modes with known fixes (wrong/correct pairs)
- `question` β†’ developer confusion that skills should preempt
- `breaking-change` β†’ migration-boundary mistakes
- **Keyword searches:** search for the skill's primary APIs, hooks,
and config options by name. E.g. `useQuery stale` or `hydration SSR`.
- **Recent vs. historical:** scan the last 6–12 months of open issues
for current pain points. Then scan older closed issues for patterns
that are now fixed but still appear in agent training data.

**GitHub Discussions** are equally important when the repo uses them.
Discussions surface "how do I..." patterns and architectural questions
that issues don't capture. Search the Discussions tab (or use
`gh api` to query discussions) for:
- Unanswered or long-thread questions (signal: docs are insufficient)
- Threads marked as "Answered" with a non-obvious solution (skill content)
- Recurring themes across multiple threads (systemic confusion)

**What to extract from issues/discussions:**
- Frequently reported confusion patterns β†’ candidate failure modes
- Workarounds that developers use before a fix ships β†’ "wrong pattern"
examples that agents will reproduce
- Recurring "how do I X with Y" threads β†’ composition skill candidates
- Misunderstandings about defaults or config β†’ skill content gaps
- Feature requests with many upvotes that change API design β†’ signals
of where the API surface is unintuitive
- What users are implicitly arguing for architecturally β€” not just
"people are confused about X" but "users keep expecting X to work
like Y, which reveals a tension between [design force] and
[design force]"

**What NOT to extract:** one-off bugs already fixed, feature requests
unrelated to current API surface, issues about build tooling or CI
that don't affect library usage patterns.

**Fallback.** If no web access is available, check for FAQ.md,
TROUBLESHOOTING.md, docs/faq, or KNOWN_ISSUES.md as proxies. Also
scan the repo's `.github/ISSUE_TEMPLATE/` for hints about common
issue categories.

6. **Source code** β€” verify ambiguities from docs, check defaults, find
assertions and invariant checks. For monorepos, read the 2–3 core
packages deeply. For adapter packages, read one representative adapter
Expand All @@ -289,6 +333,10 @@ Log every:
rejects a subtype of X
- Source assertion: any `if (!x) throw`, `invariant()`, or `assert()` with
the error message text
- Issue/discussion pattern: any recurring confusion, workaround, or
misunderstanding surfaced from GitHub issues or discussions β€” note the
issue/discussion URL, the core misunderstanding, and whether it's
resolved or still active

### What to extract from migration guides specifically

Expand Down Expand Up @@ -390,14 +438,16 @@ For each skill, extract failure modes that pass all three tests:

**Where to find them:**

| Source | What to extract |
| -------------------- | ------------------------------------------------------------------ |
| Migration guides | Every breaking change β†’ old pattern is the wrong code |
| Doc callouts | Any "note", "warning", "avoid" with surrounding context |
| Source assertions | `throw` and `invariant()` messages describe the failure |
| Default values | Undocumented or surprising defaults that cause wrong behavior |
| Type precision | Source type more restrictive than docs imply |
| Environment branches | `typeof window`, SSR flags, `NODE_ENV` β€” behavior differs silently |
| Source | What to extract |
| -------------------- | -------------------------------------------------------------------- |
| Migration guides | Every breaking change β†’ old pattern is the wrong code |
| Doc callouts | Any "note", "warning", "avoid" with surrounding context |
| Source assertions | `throw` and `invariant()` messages describe the failure |
| Default values | Undocumented or surprising defaults that cause wrong behavior |
| Type precision | Source type more restrictive than docs imply |
| Environment branches | `typeof window`, SSR flags, `NODE_ENV` β€” behavior differs silently |
| GitHub issues | Recurring bug reports with workarounds β†’ wrong/correct code pairs |
| GitHub discussions | "How do I…" threads with non-obvious answers β†’ missing skill content |

Target 3 failure modes per skill minimum. Complex skills target 5–6.

Expand Down Expand Up @@ -574,6 +624,12 @@ Adapt from this bank of gap-targeted question templates:
What should an agent know about using them together?"
- "The API reference shows [type signature], but the guide examples use
a different shape. Which is accurate?"
- "I found [N] GitHub issues/discussions where developers struggled with
[X]. The common workaround seems to be [Y] β€” is that the recommended
approach, or is there a better pattern that should be documented?"
- "GitHub discussions show developers repeatedly asking how to combine
[feature A] with [feature B]. Is there an intended integration pattern,
or is this a gap in the current API?"

### 4c β€” AI-agent-specific failure modes (2–4 questions)

Expand Down
19 changes: 19 additions & 0 deletions packages/intent/meta/feedback-collection/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,25 @@ Determine the target repo from the skill's package. The repo is typically
derivable from the `repository` field in the package's `package.json`, or
from the `sources` field in the SKILL.md frontmatter.

### Link to existing issues/discussions

Before creating a new issue, search the target repo for existing issues or
discussions that match the feedback. Use `gh search issues` or the GitHub
web search with keywords from the "What Failed" and "Missing" sections.

- If an **open issue** already describes the same problem, comment on it
with the feedback instead of creating a duplicate. Reference the skill
name and version in your comment.
- If a **closed issue** describes a problem the skill still gets wrong
(regression or stale skill content), reference the closed issue in the
new feedback issue body: `Related to #[number] β€” this was fixed in the
library but the skill still describes the old behavior.`
- If a **discussion thread** covers the same topic, link to it in the
feedback issue body so maintainers can see the community context.

This prevents duplicate issues and gives maintainers richer context for
improving skills.

### Privacy check

Before submitting, determine whether the user's project is public or private.
Expand Down
61 changes: 56 additions & 5 deletions packages/intent/meta/generate-skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,49 @@ cannot already know:
- Framework-specific gotchas (hydration, hook rules, provider ordering)
- **Constraints and invariants** β€” ordering requirements, lifecycle rules,
things enforced by runtime assertions
- **Issue/discussion-sourced patterns** β€” real developer mistakes and
confusion surfaced from GitHub issues and discussions (see below)

### 2b β€” Scan GitHub issues and discussions

Before writing the skill body, search the library's GitHub repo for issues
and discussions relevant to THIS skill's topic. This step is important for
both initial generation and regeneration β€” community feedback reveals
failure modes that docs miss.

**Search strategy:**

1. Search issues for the skill's primary APIs, hooks, and config options
by name (e.g. `useQuery invalidation`, `createRouter middleware`)
2. Filter to high-signal threads: sort by reactions/comments, focus on
closed bugs with workarounds and open questions with long threads
3. Search Discussions (if the repo uses them) for "how do I…" threads
related to the skill's topic
4. Check for issues labeled `bug`, `question`, `breaking-change` that
mention concepts this skill covers

**What to incorporate:**

- **Recurring bug workarounds** β†’ add as Common Mistakes entries with
wrong/correct code pairs. Cite the issue URL in the `Source` field.
- **Frequently asked questions** β†’ if the answer is non-obvious, add it
to Core Patterns or as a dedicated pattern section
- **Misunderstandings about defaults** β†’ add to Common Mistakes with the
incorrect assumption as the "wrong" pattern
- **Resolved issues that changed behavior** β†’ if the old behavior is
still in agent training data, add as a migration-boundary mistake

**What NOT to incorporate:**

- One-off bugs already fixed with no broader pattern
- Feature requests for APIs that don't exist yet
- Issues about tooling, CI, or build that don't affect library usage
- Stale threads (>2 years old) about behavior that has fundamentally changed

**Fallback:** If no web access is available, check for FAQ.md,
TROUBLESHOOTING.md, or docs/faq in the repo. Also check whether the
domain_map.yaml already contains issue-sourced failure modes from
domain-discovery β€” use those directly.

### What NOT to extract

Expand Down Expand Up @@ -353,12 +396,20 @@ output in this order:
When regenerating a stale skill (triggered by skill-staleness-check):

1. Read the existing SKILL.md and the source diff that triggered staleness
2. Determine which sections are affected by the change
3. Update only affected sections β€” preserve all other content
4. If a breaking change occurred, add the old pattern as a new Common
2. Scan GitHub issues and discussions opened since the skill was last
generated (use `library_version` or file timestamps as the baseline).
Look for new failure modes, resolved confusion, or changed patterns
related to this skill's topic. Apply the same search strategy from
Step 2b but scoped to the time window since last generation.
3. Determine which sections are affected by the source change AND by
any new issue/discussion findings
4. Update only affected sections β€” preserve all other content
5. If a breaking change occurred, add the old pattern as a new Common
Mistake entry (wrong/correct pair)
5. Bump `library_version` in frontmatter
6. Validate the complete file against Step 5 checks
6. If issues/discussions reveal new failure modes not in the existing
skill, add them to Common Mistakes with issue URLs as sources
7. Bump `library_version` in frontmatter
8. Validate the complete file against Step 5 checks

Do not rewrite the entire skill for a minor source change. Surgical
updates preserve review effort and reduce diff noise.
Expand Down
Loading