Skip to content

Add debtlens calibrate for percentile-based threshold suggestions #256

Description

@cursor

Summary

Add debtlens calibrate to analyze a repository's actual metric distributions and propose percentile-based rule thresholds, so first-run adoption starts low-noise instead of with one-size-fits-all defaults.

Motivation

Default thresholds (function length, hook count, branch count, etc.) are deliberately generic. On a large legacy repo they can produce thousands of findings; on a tiny repo they can miss real debt. adopt recommends a rollout but does not tune numeric thresholds to the repo. Percentile-based calibration makes the initial signal match the codebase.

Proposed behavior

  • debtlens calibrate [paths] scans, collects the underlying metric for each threshold-driven rule (e.g. large-function.maxLines), and computes distribution percentiles.
  • Suggests thresholds at a chosen percentile (e.g. p90 by default; --percentile 85), so roughly the worst N% is flagged.
  • Output as a ready-to-merge config snippet (--write merges into debtlens.config.json after confirmation) or as a printed diff.
  • Complements adopt (which picks rules + severity) by tuning the numbers.

Implementation surface

  • New src/cli/commands/calibrate.ts + src/core/calibrate.ts.
  • Detectors expose the raw metric they measure (extend detector metadata in src/core/types.ts or compute via a calibration pass) so calibration knows which config key maps to which distribution.
  • Reuse src/config/mergeConfig.ts / src/config/template.ts for safe writes.
  • Tests: percentile math, suggested-threshold output, --write merge behavior.

Acceptance criteria

  • Calibrate reports current vs suggested thresholds per threshold-driven rule.
  • --percentile controls aggressiveness; default documented.
  • --write merges suggestions without clobbering unrelated config.
  • Docs (docs/false-positives.md adoption section) + CHANGELOG updated.
  • npm run test:all passes.

Difficulty: medium.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions