Skip to content

fix: strip residual macro HTML from Markdown output (#5)#9

Merged
aqueeb merged 2 commits into
mainfrom
fix/stray-html-macros-issue-5
Jul 3, 2026
Merged

fix: strip residual macro HTML from Markdown output (#5)#9
aqueeb merged 2 commits into
mainfrom
fix/stray-html-macros-issue-5

Conversation

@aqueeb

@aqueeb aqueeb commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #5 — converted Markdown was still leaving "blocks of HTML surrounded with Markdown" for unhandled Confluence macros (e.g. a Change History macro), forcing a second cleanup tool.

Root cause: unhandled macros are wrapped in <div class="...">. postProcessMarkdown already stripped all closing </div> and all <span> tags and converted a handful of known macro classes — but had no general removal of residual opening <div ...> tags, so unrecognized macro wrappers leaked.

Fix: a residual-HTML cleanup pass (after all specific macro conversions) that removes opening/closing <div> and non-semantic HTML5 sectioning wrappers (section, article, header, footer, nav, aside, figure, figcaption), keeping inner content. Tags the converter intentionally emits (<br>, <details>, <summary>, <img>) and table tags are excluded — so real content and raw-HTML fallback tables are preserved. No content is dropped (the reporter's accepted "convert to text" option).

Before → After (representative macro doc)

<div class="conf-macro output-block">          ->  | Version | Author |
| Version | Author | ...                            |---------|--------|
<div class="plugin_attachments_container">          | 1.0     | Jane   |
Attachments                                         Attachments

Output now contains zero <div>/<span> tags; issue #6's testpage.doc converts identically (no regression).

Testing (TDD, same rigor as #8)

  • Strip tests written first and confirmed failing, then fixed.
  • Preserve-set locks: <details>/<summary>, <img>, and raw fallback <table> tags survive.
  • End-to-end ConvertHTMLToMarkdown on a macro-wrapped document (change-history table + unknown macro + attachments) → clean Markdown.
  • Fuzz seeds added to both FuzzPostProcessMarkdown and FuzzPreProcessHTML; active fuzzing 45s each (≈204K + 590K execs) — no crashers.
  • go test ./... -race green · go vet clean · gofmt clean on markdown.go/markdown_test.go.

Notes / limitations

  • Like the existing span/div cleanup, this pass is not fenced-code-block aware, so a literal <div> inside a ``` code sample is also stripped. Matches current behavior; a code-fence-aware cleanup is a possible follow-up.
  • Dropping entire named macro bodies (vs. keeping their text) would need a real HTML parser (regex can't match nested <div> reliably) — out of scope.
  • I don't have the reporter's exact export; happy to validate against a real sample. The general wrapper-strip resolves the described symptom.

🤖 Generated with Claude Code

aqueeb and others added 2 commits July 2, 2026 21:39
Unhandled Confluence macros are wrapped in <div class="..."> containers.
postProcessMarkdown already removed all closing </div> and <span> tags and
converted a handful of known macro classes, but left unrecognized *opening*
<div ...> tags in the output, leaving "blocks of HTML surrounded with Markdown".

Add a residual-HTML cleanup pass (after all specific macro conversions) that
strips opening/closing <div> and non-semantic HTML5 sectioning wrappers
(section, article, header, footer, nav, aside, figure, figcaption) while keeping
their inner content. Tags the converter intentionally emits (<br>, <details>,
<summary>, <img>) and table tags are excluded, so real content and raw-HTML
fallback tables are preserved.

Developed test-first (strip tests confirmed failing before the change) with
preserve-set regression locks and an end-to-end macro-wrapper test.

Known limitation: like the existing span/div cleanup, this is not fenced-code
aware, so a literal <div> inside a code block is also stripped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Aqueeb Qadri <aqueeb@gmail.com>
Add FuzzPostProcessMarkdown seeds (residual/nested divs, sectioning wrappers,
divs inside code fences, details preservation) and FuzzPreProcessHTML seeds
(macro-wrapped blocks). Active fuzzing (45s each) surfaced no crashers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Aqueeb Qadri <aqueeb@gmail.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@aqueeb
aqueeb merged commit 76857db into main Jul 3, 2026
5 checks passed
@aqueeb
aqueeb deleted the fix/stray-html-macros-issue-5 branch July 3, 2026 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

html still in Markdown content

2 participants