Add markdown button and make llms.txt available

[neon60]

Motivation

AI coding assistants and agents increasingly need documentation in machine-readable formats. Two patterns are emerging as standards: per-page Markdown download (so users and agents can copy page content directly into an AI context window) and llms.txt (a combined documentation file that AI agents can discover and ingest). This PR adds both capabilities to make the documentation more accessible to AI-assisted workflows.

Technical Details

Three changes to docs/conf.py, plus a new docs/llms.txt:

Enable the theme download button

Sets "use_download_button": True in html_theme_options, which activates the built-in per-page download action provided by the Sphinx book theme. This allows readers to download the current page as Markdown or RST depending on the source format.

Add a curated llms.txt index

Adds a base docs/llms.txt that serves as a curated index of all documentation pages, including those sourced from RST. Sets html_extra_path = ["llms.txt"] so Sphinx copies it to the output root, making it discoverable at /llms.txt.

Add llms-full.txt generation

Adds a Sphinx build-finished hook (generate_combined_markdown) that filters and combines Markdown and RST source files into llms-full.txt. Each file is filtered through a prose detection pass that strips code fences, directive options, MyST/RST anchors, and inline HTML. Files with fewer than 10 prose lines are skipped entirely to avoid including navigation-only or boilerplate pages. Files in build, static, template, and VCS directories are excluded.

This follows the https://llmstxt.org/ for AI agent documentation discovery, and is compatible with https://docs.readthedocs.com/platform/stable/reference/llms-txt.html.

Test Plan

• Verify the download button appears on documentation pages
• Verify llms.txt is available at {project_url}/llms.txt and contains only the curated index
• Verify llms-full.txt is available at {project_url}/llms-full.txt
• Confirm the curated index section appears at the top of llms-full.txt
• Confirm excluded directories (.git, _build, _static, _templates, .venv) are not represented in llms-full.txt
• Confirm navigation-only pages do not appear in the stitched output

Test Result

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.