Surface deprecated prebuilt detection rules as a sibling nav page

[Mpdreamz]

Why

Elastic's prebuilt detection rules repo contains a _deprecated/ folder of rules that have been retired. These rules were previously invisible to users browsing the docs — there was no way to discover them, understand why they exist, or know what replaced them. Users navigating directly to a deprecated rule page (e.g. from a bookmark or external link) also lost their nav context entirely: nothing in the sidebar was highlighted.

What

Deprecated rules page

When the detection-rules plugin processes a rule folder that contains a _deprecated/ subfolder, it automatically generates a sibling "Deprecated prebuilt detection rules" page in the navigation alongside the active rules overview. No docset.yml changes are required for the page to appear.

The page lists all deprecated rules alphabetically by domain, linking to individual rule pages. Each individual rule page shows a deprecation warning admonition with the date the rule was deprecated.

Optional preface

The security team can supply introductory content for the deprecated rules page by adding a markdown file to the detection-rules docs folder and referencing it in docset.yml:

- file: index.md
  detection_rules: ['../rules', '../rules_building_block']
  deprecated_file: deprecated-rules.md   # optional

When deprecated_file is set, the content of that file prefixes the auto-generated rules listing. This lets the team explain what deprecated rules are, what users should do with them, and link to related guidance — all without touching the docs-builder codebase.

Follow-up: elastic/detection-rules#5968 adds this preface file (draft, cannot merge until this PR is released to production).

Nav active state for hidden pages

Individual rule pages (both active and deprecated) are hidden nav leaves — they exist in the nav tree for ordering/breadcrumb purposes but don't appear as visible links in the sidebar. Before this change, landing on a rule page left the sidebar with nothing selected.

The fix emits a <meta name="docs:nav-active" content="{parentUrl}"> tag server-side when the current page is a hidden nav item. The client reads this and highlights the nearest visible ancestor instead. This restores nav context for rule pages and fixes the same issue on the live site for any hidden page.

Verification

Run locally against the detection-rules repo:

dotnet run --project src/tooling/docs-builder -- serve --path /path/to/detection-rules

• http://localhost:3000/deprecated-detection-rules — lists deprecated rules
• http://localhost:3000/rules/_deprecated/<rule> — deprecation warning, sidebar highlights deprecated rules page
• http://localhost:3000/rules/<category>/<rule> — sidebar highlights prebuilt rules overview

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This PR migrates detection-rule TOML parsing from Tomlet to Tomlyn, introduces deprecation metadata (maturity and date fields), adds dedicated TOC discovery and rendering for deprecated rules within _deprecated subdirectories, and implements navigation state propagation to highlight the correct nav item when viewing pages that are hidden from the navigation tree (such as individual detection rule pages).

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly and clearly describes the main change: surfacing deprecated detection rules as a sibling navigation page.
Description check	✅ Passed	The description comprehensively explains the motivation, implementation details, and verification steps for the feature.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch feature/deprecated-detection-rules

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs (1)

75-87: ⚠️ Potential issue | 🟡 Minor

Remove unused deprecated_detection_rules from parsing logic.

The deprecated_detection_rules key is parsed into the dictionary but never read anywhere in the codebase. The PR auto-discovers deprecated rules via folder scanning, making this key dead code. Remove it from the condition or document why it's retained.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs`
around lines 75 - 87, In the YAML parsing branch inside
TableOfContentsYamlConverters (the else-if that checks key.Value against
"detection_rules" or "exclude" or "deprecated_detection_rules"), remove
"deprecated_detection_rules" from that condition and any related parsing branch
so the parser no longer adds that key into the dictionary; if you intentionally
want to keep it, add a comment explaining why and where it's consumed. Update or
remove any tests or usages that expect deprecated_detection_rules to be present
and ensure the parsing of the remaining keys ("detection_rules", "exclude") is
unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In
`@src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs`:
- Around line 75-87: In the YAML parsing branch inside
TableOfContentsYamlConverters (the else-if that checks key.Value against
"detection_rules" or "exclude" or "deprecated_detection_rules"), remove
"deprecated_detection_rules" from that condition and any related parsing branch
so the parser no longer adds that key into the dictionary; if you intentionally
want to keep it, add a comment explaining why and where it's consumed. Update or
remove any tests or usages that expect deprecated_detection_rules to be present
and ensure the parsing of the remaining keys ("detection_rules", "exclude") is
unchanged.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 81e46472-e23f-4459-8cba-6b14b17737ff

📥 Commits

Reviewing files that changed from the base of the PR and between b94fd12 and 9042a9a.

📒 Files selected for processing (15)

• src/Elastic.Documentation.Configuration/FileSystemFactory.cs
• src/Elastic.Documentation.Configuration/Toc/DetectionRules/DetectionRuleOverviewRef.cs
• src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs
• src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs
• src/Elastic.Documentation.Site/Assets/pages-nav.ts
• src/Elastic.Documentation.Site/Layout/_Head.cshtml
• src/Elastic.Documentation.Site/_ViewModels.cs
• src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs
• src/Elastic.Markdown/Extensions/DetectionRules/DetectionRuleFile.cs
• src/Elastic.Markdown/Extensions/DetectionRules/DetectionRulesDocsBuilderExtension.cs
• src/Elastic.Markdown/HtmlWriter.cs
• src/Elastic.Markdown/Page/Index.cshtml
• src/Elastic.Markdown/Page/IndexViewModel.cs
• src/tooling/docs-builder/Http/InMemoryBuildState.cs
• src/tooling/docs-builder/Http/ReloadGeneratorService.cs

Saw an issue with TOML parsing unrelated to these changes, but still affecting parsing as a whole

[cotti]

I sent #3214 as a possible fix - it works better as a PR for this branch.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs (1)

292-293: 💤 Low value

GetString throws unhelpful exception on missing keys.

Direct indexing table[key] throws KeyNotFoundException without context about which field is missing. For better debuggability in malformed TOML, consider wrapping with a clearer message.

Optional improvement

-private static string GetString(TomlTable table, string key) =>
-    (string)table[key];
+private static string GetString(TomlTable table, string key) =>
+    table.TryGetValue(key, out var value) && value is string s
+        ? s
+        : throw new InvalidOperationException($"Required field '{key}' is missing or not a string");

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs` around lines
292 - 293, The GetString helper currently indexes TomlTable directly causing a
KeyNotFoundException with no context; update GetString (the private static
method) to first check for the key (e.g. ContainsKey or TryGetValue) and if
missing throw an ArgumentException or KeyNotFoundException with a clear message
that includes the missing key name (and optionally the table name/context) and,
when present, safely convert the value to string (with null checks) so malformed
or null values produce a descriptive error instead of an unhelpful exception.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs`:
- Around line 281-290: ReadTactic currently indexes threatTable["tactic"] and
will throw if the key is missing; change it to use
threatTable.TryGetValue("tactic", out var t) and ensure t is a TomlTable before
casting, returning null (or a safe default) if the key is absent or not a
TomlTable; then call GetString(tacticTable, "...") only when tacticTable is
non-null so DetectionRuleTactic construction and GetString calls are guarded
against missing or malformed tactic entries.

---

Nitpick comments:
In `@src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs`:
- Around line 292-293: The GetString helper currently indexes TomlTable directly
causing a KeyNotFoundException with no context; update GetString (the private
static method) to first check for the key (e.g. ContainsKey or TryGetValue) and
if missing throw an ArgumentException or KeyNotFoundException with a clear
message that includes the missing key name (and optionally the table
name/context) and, when present, safely convert the value to string (with null
checks) so malformed or null values produce a descriptive error instead of an
unhelpful exception.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 87f9e368-6350-4e26-af0d-9198120ce3a6

📥 Commits

Reviewing files that changed from the base of the PR and between 9042a9a and 256f308.

📒 Files selected for processing (7)

• Directory.Packages.props
• src/Elastic.Documentation.Configuration/Elastic.Documentation.Configuration.csproj
• src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs
• src/Elastic.Markdown/Elastic.Markdown.csproj
• src/Elastic.Markdown/Extensions/DetectionRules/DetectionRule.cs
• src/Elastic.Markdown/HtmlWriter.cs
• tests/Elastic.Markdown.Tests/DetectionRules/DetectionRuleParsingTests.cs

✅ Files skipped from review due to trivial changes (1)

• src/Elastic.Documentation.Configuration/Elastic.Documentation.Configuration.csproj

🚧 Files skipped from review as they are similar to previous changes (1)

• src/Elastic.Markdown/HtmlWriter.cs

[eric-forte-elastic]

@Mpdreamz just checking, can we merge this to resolve ongoing issues like https://github.com/elastic/detection-rules/actions/runs/25744929992/job/75605440043?

Thanks!

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/tooling/docs-builder/Http/ReloadGeneratorService.cs`:
- Around line 51-52: StartAsync resolves sourcePath using
ReloadableGenerator.Generator.Context.DocumentationCheckoutDirectory?.FullName
?? ReloadableGenerator.Generator.Context.DocumentationSourceDirectory.FullName,
but Reload still uses DocumentationSourceDirectory directly causing inconsistent
targets; update the Reload method to compute sourcePath the same way (use the
nullable DocumentationCheckoutDirectory fallback to
DocumentationSourceDirectory) so both StartAsync and Reload use the identical
sourcePath resolution (reference ReloadableGenerator, Reload method, and
StartAsync).

🪄 Autofix (Beta)

✅ Autofix completed

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 94706b4d-f98f-48e0-9d6c-95b8077ef089

📥 Commits

Reviewing files that changed from the base of the PR and between ba70441 and b136e63.

📒 Files selected for processing (11)

• Directory.Packages.props
• src/Elastic.Documentation.Configuration/Elastic.Documentation.Configuration.csproj
• src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs
• src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs
• src/Elastic.Documentation.Site/Layout/_Head.cshtml
• src/Elastic.Documentation.Site/_ViewModels.cs
• src/Elastic.Markdown/HtmlWriter.cs
• src/Elastic.Markdown/Page/Index.cshtml
• src/Elastic.Markdown/Page/IndexViewModel.cs
• src/tooling/docs-builder/Http/InMemoryBuildState.cs
• src/tooling/docs-builder/Http/ReloadGeneratorService.cs

✅ Files skipped from review due to trivial changes (2)

• src/tooling/docs-builder/Http/InMemoryBuildState.cs
• Directory.Packages.props

🚧 Files skipped from review as they are similar to previous changes (8)

• src/Elastic.Markdown/HtmlWriter.cs
• src/Elastic.Markdown/Page/Index.cshtml
• src/Elastic.Documentation.Site/Layout/_Head.cshtml
• src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs
• src/Elastic.Markdown/Page/IndexViewModel.cs
• src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs
• src/Elastic.Documentation.Configuration/Elastic.Documentation.Configuration.csproj
• src/Elastic.Documentation.Site/_ViewModels.cs

[coderabbitai]

Note

Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it.

Fixes Applied Successfully

Fixed 1 file(s) based on 1 unresolved review comment.

Files modified:

• src/tooling/docs-builder/Http/ReloadGeneratorService.cs

Commit: 3aa8e7a67f326a08037df72527774a435b498f00

The changes have been pushed to the feature/deprecated-detection-rules branch.

Time taken: 1m 54s

[cla-checker-service]

💚 CLA has been signed