fix: strip residual macro HTML from Markdown output (#5)#9
Merged
Conversation
Unhandled Confluence macros are wrapped in <div class="..."> containers. postProcessMarkdown already removed all closing </div> and <span> tags and converted a handful of known macro classes, but left unrecognized *opening* <div ...> tags in the output, leaving "blocks of HTML surrounded with Markdown". Add a residual-HTML cleanup pass (after all specific macro conversions) that strips opening/closing <div> and non-semantic HTML5 sectioning wrappers (section, article, header, footer, nav, aside, figure, figcaption) while keeping their inner content. Tags the converter intentionally emits (<br>, <details>, <summary>, <img>) and table tags are excluded, so real content and raw-HTML fallback tables are preserved. Developed test-first (strip tests confirmed failing before the change) with preserve-set regression locks and an end-to-end macro-wrapper test. Known limitation: like the existing span/div cleanup, this is not fenced-code aware, so a literal <div> inside a code block is also stripped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Aqueeb Qadri <aqueeb@gmail.com>
Add FuzzPostProcessMarkdown seeds (residual/nested divs, sectioning wrappers, divs inside code fences, details preservation) and FuzzPreProcessHTML seeds (macro-wrapped blocks). Active fuzzing (45s each) surfaced no crashers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Aqueeb Qadri <aqueeb@gmail.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #5 — converted Markdown was still leaving "blocks of HTML surrounded with Markdown" for unhandled Confluence macros (e.g. a Change History macro), forcing a second cleanup tool.
Root cause: unhandled macros are wrapped in
<div class="...">.postProcessMarkdownalready stripped all closing</div>and all<span>tags and converted a handful of known macro classes — but had no general removal of residual opening<div ...>tags, so unrecognized macro wrappers leaked.Fix: a residual-HTML cleanup pass (after all specific macro conversions) that removes opening/closing
<div>and non-semantic HTML5 sectioning wrappers (section, article, header, footer, nav, aside, figure, figcaption), keeping inner content. Tags the converter intentionally emits (<br>,<details>,<summary>,<img>) and table tags are excluded — so real content and raw-HTML fallback tables are preserved. No content is dropped (the reporter's accepted "convert to text" option).Before → After (representative macro doc)
Output now contains zero
<div>/<span>tags; issue #6'stestpage.docconverts identically (no regression).Testing (TDD, same rigor as #8)
<details>/<summary>,<img>, and raw fallback<table>tags survive.ConvertHTMLToMarkdownon a macro-wrapped document (change-history table + unknown macro + attachments) → clean Markdown.FuzzPostProcessMarkdownandFuzzPreProcessHTML; active fuzzing 45s each (≈204K + 590K execs) — no crashers.go test ./... -racegreen ·go vetclean ·gofmtclean onmarkdown.go/markdown_test.go.Notes / limitations
<div>inside a ``` code sample is also stripped. Matches current behavior; a code-fence-aware cleanup is a possible follow-up.<div>reliably) — out of scope.🤖 Generated with Claude Code