[EPIC] DX overhaul: write less, document more, AI-native docs

[mhenrixon]

Problem / Goal

docs-kit exists to make documentation happen. Today the chrome is excellent but the authoring cost is too high, so pages don't get written (daisyui's docs: 3 registered pages, only 1 authored). Three audits (this gem's own docs site, the daisyui + phlex-reactive sites, and zazu/app-4's API docs) converge on the same evidence:

• Prose is written as Ruby, not prose. One paragraph with two inline code spans costs ~11 lines of p do code { … } plain " … " end. The gem's own docs/app/views/docs/pages/installation.rb spends 19 plain/code calls on 6–7 sentences; the file is 140 lines for what Markdown says in ~40.
• Every site hand-rolls the same missing pieces. A props/options table (PropTable) is copy-pasted per site; app-4 has 22 raw table blocks, 216 raw tr/td calls, and 15+ near-duplicate err_row/field_row helpers across 9 files. Both consumer sites carry a byte-identical RuboCop cop and a duplicated render_page.
• API docs restate the same fact up to 7 times. In app-4, one endpoint's example is authored once per client (curl/CLI/JS/Ruby/Python), again as the JSON response, and a third time in a hand-written Markdown to_text twin (929-line invoices.rb). ~25% of the 4,301 docs LOC is restatement, not content.
• Registration is quadruple bookkeeping. slug + title + group + view: string, kept in sync by hand with a 4-level-nested class in another file, plus a 10-line nav lambda every site copies. There is no docs_kit:page generator.
• Nothing is machine-readable. No .md view of a page, no llms.txt, no search, no MCP. app-4 had to invent "Copy for LLM" by hand-writing a Markdown twin of every page — the strongest possible signal that AI consumption is a real need the gem doesn't serve.

Done looks like: a new page is one generator command; prose is Markdown; an API endpoint is one structured declaration that renders every client tab; and every page automatically has a Markdown twin that powers llms.txt, search, "copy for LLM", and an optional built-in MCP server — with zero extra authoring.

Decision

Phlex-native ergonomics + markdown islands + one derivable source of truth. Pages stay Phlex classes (the registry → chrome → TOC pipeline and live demos depend on it), but:

1. Prose becomes Markdown (DocsUI::Markdown / md helper) — kills the p/plain/code chains.
2. The missing components ship in the gem (Table/PropTable, Endpoint, FieldTable/ErrorTable, RequestExample, JsonResponse).
3. The registry becomes one line per page with derived slug/view and auto-derived nav; a docs_kit:page generator scaffolds the rest.
4. Every page gets an automatic Markdown twin (HTML→Markdown over the bounded tag vocabulary the kit itself emits). llms.txt, llms-full.txt, server-rendered search, a copy-as-Markdown button, and an optional MCP endpoint all derive from it. One source of truth; the AI surface costs authors nothing.

Alternatives rejected:

• Markdown-first pivot (pages as .md files) — loses live demos/galleries/reactive components (daisyui's 275 example files, phlex-reactive's demos), breaks every existing page, and turns docs-kit into another static-site generator. Markdown belongs inside pages, not instead of them.
• A new builder DSL (page do … end) — a second API surface that doesn't fix the actual pain (prose verbosity) and churns all sites.
• Authoring-side MCP server — agents writing docs work inside the repo; they need conventions + generators (an AGENTS.md/skill), not a protocol. MCP is for reading published docs; it comes last, built on the export pipeline.

✅ Delivered — all sub-issues shipped and merged (PRs #25–#39, plus #6/#7 for the adjacent #3/#2). Closed 2026-07-03 after a full deferred-scope audit; see the closing comment.

Sub-issues

Phases are ordered by leverage; issues within a phase are independent unless noted. #23 (CI) should merge first — everything else lands through it.

Phase 1 — Authoring ergonomics

• feat(markdown): DocsUI::Markdown — markdown islands for prose authoring #9 Markdown islands (DocsUI::Markdown + md helper)
• feat(page): normalize the authoring API — positional Header title, no more parens gotcha #10 Normalize the component API (Header title, page helpers, parens gotcha)
• feat(table): ship DocsUI::Table + DocsUI::PropTable — stop hand-rolling tables #11 DocsUI::Table + DocsUI::PropTable
• feat(registry): one-line page registration + auto-derived nav #12 Registry v2: one-line page registration + auto-derived nav
• feat(generator): rails g docs_kit:page — page class + registry entry in one command #13 rails g docs_kit:page generator (needs feat(registry): one-line page registration + auto-derived nav #12)

Phase 2 — API-docs kit (the app-4 patterns, upstreamed)

• feat(api-docs): DocsUI::Endpoint + FieldTable/ErrorTable — the endpoint-reference kit #14 DocsUI::Endpoint + FieldTable/ErrorTable (needs feat(table): ship DocsUI::Table + DocsUI::PropTable — stop hand-rolling tables #11)
• feat(api-docs): DocsUI::RequestExample + JsonResponse — one request definition, every client tab #15 DocsUI::RequestExample + DocsUI::JsonResponse
• feat(config): brand_href + dark/light code themes — the knobs API sites shim today #16 Config knobs API sites shim today: brand_href, dark/light code themes

Phase 3 — One source of truth → machine-readable

• feat(markdown-twin): every page renders as Markdown — .md format + copy-page button #17 Markdown twin for every page (.md format + copy button)
• feat(llms): serve /llms.txt + /llms-full.txt from the registry #18 /llms.txt + /llms-full.txt (needs feat(markdown-twin): every page renders as Markdown — .md format + copy-page button #17, feat(registry): one-line page registration + auto-derived nav #12)
• feat(search): server-rendered docs search + palette, inside the docs-nav controller #19 Built-in search: server-rendered + ⌘K palette in docs-nav (needs feat(markdown-twin): every page renders as Markdown — .md format + copy-page button #17, feat(registry): one-line page registration + auto-derived nav #12)

Phase 4 — AI-native

• feat(mcp): optional built-in MCP server — list/get/search docs over Streamable HTTP #20 Optional built-in MCP server (needs feat(markdown-twin): every page renders as Markdown — .md format + copy-page button #17, feat(llms): serve /llms.txt + /llms-full.txt from the registry #18, feat(search): server-rendered docs search + palette, inside the docs-nav controller #19)
• feat(ai-authoring): scaffold AGENTS.md + a write-docs-page skill into consuming sites #21 Agent authoring kit: AGENTS.md + docs-authoring skill (best after Phase 1)

Phase 5 — Health & consolidation

• feat(rubocop): ship the docs-kit RuboCop cops from the gem #22 Ship the RuboCop cops from the gem
• ci: add the gem's CI workflow + the missing generator specs #23 CI workflow + generator specs — merge first
• fix(generator): idempotent install + a consumer re-sync guide #24 Idempotent install generator + consumer re-sync guide (after ci: add the gem's CI workflow + the missing generator specs #23)

Invariants every sub-issue must respect

Chrome is DocsUI:: components only; site values come from DocsKit.configuration with defaults (backwards compatible); pages work with JS off; exactly ONE Stimulus controller (docs-nav) — new interactivity extends it; config.themes == the Tailwind @plugin list; new render-time classes need an @source scan; required setup wires into the install generator AND docs-kit new; TDD (specs before code).

Adjacent in-flight work (not part of this epic)

• DocsUI::Shell inline scripts are not nonce-aware — blocked under a nonce-based CSP #2 CSP nonce for Shell inline scripts — fixed on fix/csp-nonce-shell (unmerged).
• Make DocsUI::Icon's icon library configurable (currently hardcoded to RailsIcons default_library) #3 configurable icon library — fixed on feat/configurable-icon-library (unmerged). Both remove app-4 shims once merged.

Out of scope

• Rendering OpenAPI documents into pages (app-4 generates openapi.yaml from request specs but it is unconnected to the docs; an OpenAPI bridge is a natural follow-up epic once feat(api-docs): DocsUI::Endpoint + FieldTable/ErrorTable — the endpoint-reference kit #14/feat(api-docs): DocsUI::RequestExample + JsonResponse — one request definition, every client tab #15 exist as render targets).
• Fixing the consumer repos themselves (deleting their dead IconHelper, adopting DocsKit::Controller) — follow-ups in those repos once fix(generator): idempotent install + a consumer re-sync guide #24 ships the --sync tool.
• Versioned docs, i18n, analytics.

Execution

Execute sub-issues with /lfg <issue-number>, phase by phase (start with /lfg 23).