Skip to content

report: count obfuscator.io corpus in readability metrics#61

Merged
vasie1337 merged 1 commit into
mainfrom
claude/stoic-franklin-zft90c
Jun 8, 2026
Merged

report: count obfuscator.io corpus in readability metrics#61
vasie1337 merged 1 commit into
mainfrom
claude/stoic-franklin-zft90c

Conversation

@vasie1337

Copy link
Copy Markdown
Member

What

The synthetic javascript-obfuscator (obfuscator.io) fixtures in samples/generated/ — 140 files = 20 seeds × 7 obfuscation profiles — gated correctness via manifest.json, but were excluded from the readability metrics. Both the live report binary and the committed SCOREBOARD.md read samples/ non-recursively, so the obfuscator.io samples never showed up in the dashboard.

This adds a per-profile rollup of the generated corpus (aggregated over all seeds, one row per obfuscation technique) to both surfaces, so the obfuscator.io samples count toward readability the same way they count toward correctness.

Metrics shape

  • kept% — byte-weighted Σout/Σin (lower = more dead weight removed)
  • opaque% — mean of per-file opaque-name ratios
  • rounds — worst case across the profile's files
  • convergedyes only if every file reached a fixpoint
profile files kept% brackets opaque%
minimal 20 62% 12 8%
strarr_base64 20 68% 12 5%
strarr_rc4 20 72% 12 4%
controlflow 20 32% 12 8%
deadcode 20 35% 12 8%
numbers_keys 20 53% 12 5%
strong 20 57% 12 5%
all 140 56% 84 6%

Changes

  • src/bin/report.rs — append an "obfuscator.io corpus, aggregated by profile" section to the live dashboard.
  • tests/golden.rs — matching per-profile rollup in render_scoreboard(), so the committed table and live dashboard stay in agreement.
  • tests/snapshots/SCOREBOARD.md — re-blessed with the new section.

Verification

  • Live report runs and prints the new section.
  • scoreboard_is_current passes, and re-runs deterministically without blessing.
  • cargo clippy clean.

https://claude.ai/code/session_01EjhNTCU89wa5zaeRHMnfEc


Generated by Claude Code

The synthetic javascript-obfuscator (obfuscator.io) fixtures in
samples/generated/ gated correctness via manifest.json but were excluded
from the readability metrics: both the live report binary and the
committed SCOREBOARD.md read samples/ non-recursively.

Add a per-profile rollup of the generated corpus (aggregated over all
seeds, one row per obfuscation technique) to both surfaces, so the
obfuscator.io samples count toward readability the same way they count
toward correctness. kept% is byte-weighted, opaque% is the mean per-file
ratio, rounds is the worst case, and converged flags any non-fixpoint.
@vasie1337 vasie1337 merged commit f92ddea into main Jun 8, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants