Skip to content

Redesign heading-health plugin to match Crawlith plugin standards#83

Merged
saurabhsharma2u merged 3 commits into
mainfrom
codex/redesign-heading-health-plugin
Mar 3, 2026
Merged

Redesign heading-health plugin to match Crawlith plugin standards#83
saurabhsharma2u merged 3 commits into
mainfrom
codex/redesign-heading-health-plugin

Conversation

@saurabhsharma2u

Copy link
Copy Markdown
Contributor

Motivation

  • Standardize the heading-health plugin to follow the Crawlith plugin architecture, lifecycle hooks, and secure DB/config patterns described in the agent docs.
  • Improve separation of concerns so analysis, scoring, CLI registration, persistence and output are easier to maintain and test.
  • Provide URL-scoped caching and persistence through the ctx.db API and allow a force-refresh option for recomputations.

Description

  • Restructured the package into the canonical manifest + src/ layout and exported HeadingHealthPlugin from index.ts delegating to src/ modules (plugin.ts, cli.ts, HeadingHealthService.ts, Output.ts, types.ts, analyzer.ts).
  • Implemented lifecycle hooks: onInit registers schema via ctx.db.schema.define, onMetrics computes per-page payloads, reads 24h cached rows via ctx.db.data.find(..., { maxAge: '24h', global: true }), persists via ctx.db.data.save, and publishes a snapshot summary to ctx.metadata, and onReport attaches the snapshot summary to result.plugins.headingHealth.
  • Extracted scoring, aggregation and grouping logic into HeadingHealthService and improved analyzer code with clearer types and utility functions (jaccardSimilarity, entropy calculation, duplicate-risk enrichment); added HeadingHealthOutput for logging and registerHeadingHealthCli for CLI options including --heading-force-refresh.
  • Added/updated typed interfaces in src/types.ts, expanded JSDoc comments across modules, and updated tests to assert DB find/save calls, per-page payload attachment, and report summary behavior.

Testing

  • Ran unit tests with pnpm --filter @crawlith/plugin-heading-health test and all tests passed (5 passed (5)), validating analyzer behavior and plugin hooks. ✅
  • Attempted package build with pnpm --filter @crawlith/plugin-heading-health build but the isolated TypeScript compile failed because the environment could not resolve the workspace package @crawlith/core during an isolated package tsc run; this is an environment workspace resolution issue and does not affect the plugin unit tests. ⚠️

Codex Task

@saurabhsharma2u
saurabhsharma2u merged commit 7e57ab3 into main Mar 3, 2026
1 of 7 checks passed
@saurabhsharma2u
saurabhsharma2u deleted the codex/redesign-heading-health-plugin branch March 3, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant