Add RE2-backed safe-regex substrate (re2 native dependency)#563
Add RE2-backed safe-regex substrate (re2 native dependency)#563kriszyp wants to merge 2 commits into
Conversation
Introduce compileSafeRegex (security/safeRegex.ts), a linear-time regex compiler backed by the re2 native addon, as the safe substrate for regex whose source is operator- or attacker-influenced. RE2's finite-automaton engine cannot catastrophically backtrack, so a pattern like (a+)+$ cannot stall the event loop (ReDoS) the way new RegExp(...) can. The signature mirrors an error-collecting validator; RE2-unsupported features (backreferences, lookaround) surface as compile errors rather than running unsafely -- that rejection is part of the contract. This lands the native dependency on its own so its prebuilt-matrix resolution and the linear-time/feature-rejection contract are reviewed independently of the WAF logic that will consume it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new utility function compileSafeRegex in security/safeRegex.ts that uses the re2 library to compile regular expressions safely, protecting the application against catastrophic backtracking (ReDoS) attacks. It also adds comprehensive unit tests to verify linear-time matching and the rejection of unsupported features like backreferences and lookarounds. The feedback points out a potential issue in the error handling of compileSafeRegex where a non-Error object could be thrown, which would cause a secondary TypeError when accessing .message. It is recommended to check if the caught error is an instance of Error before accessing its properties.
| } catch (error) { | ||
| errors.push(`${where}: invalid or unsupported regex ${JSON.stringify(source)}: ${(error as Error).message}`); | ||
| return undefined; |
There was a problem hiding this comment.
In JavaScript/TypeScript, any value can be thrown (not just instances of Error). If a non-Error value (or null/undefined) is thrown, attempting to access (error as Error).message will result in a secondary TypeError (e.g., Cannot read properties of null (reading 'message')), which would crash the function and bypass the safety of the try-catch block. It is safer to check if the caught error is an instance of Error before accessing its message property, falling back to a string representation otherwise.
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
errors.push(`${where}: invalid or unsupported regex ${JSON.stringify(source)}: ${message}`);
return undefined;
|
Reviewed; no blockers found. |
…ion gate Point compileRuleRegex at the RE2-backed compileSafeRegex (harper-pro #563) so operator-supplied patterns match in guaranteed linear time — closing the ReDoS vector the prototype's `new RegExp(...)` left open (a hostile `(a+)+$` can no longer stall the event loop). Because every path-regex is now a non-backtracking RE2, the JS-RegExp combined-alternation pre-gate and its backreference-safety machinery (hasBackreference + gateable/ungated split) exist only to make an unsafe engine fast, so they are deleted: path-regex rules are matched by a plain linear scan, still allocation-free on the no-match path. For realistic super_user-authored rule sets this scan is negligible; RE2.Set is the drop-in native multi-pattern scan if path-regex volume ever dominates. Contract change: RE2 cannot evaluate backreferences or lookaround, so a rule using them is now rejected as invalid (same as malformed syntax) rather than silently compiled. The M3 adversarial tests are rewritten to pin this rejection and the ReDoS immunity in place of the removed gate's behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The unit/integration/stress workflows install with `npm install --ignore-scripts` and then run Harper. lmdb/msgpackr/rocksdb survive that because they bundle their prebuilt binary in the npm package; re2 instead downloads its binary from a post-install script, so --ignore-scripts leaves re2.node absent and every Harper process that loads the WAF fails with "Cannot find module './build/Release/re2.node'". Add an explicit `npm rebuild re2` step after each such install to materialize the binary (fetches the prebuilt where one exists, source-builds otherwise — e.g. on Node majors re2 has not yet published a prebuilt for). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion gate Point compileRuleRegex at the RE2-backed compileSafeRegex (harper-pro #563) so operator-supplied patterns match in guaranteed linear time — closing the ReDoS vector the prototype's `new RegExp(...)` left open (a hostile `(a+)+$` can no longer stall the event loop). Because every path-regex is now a non-backtracking RE2, the JS-RegExp combined-alternation pre-gate and its backreference-safety machinery (hasBackreference + gateable/ungated split) exist only to make an unsafe engine fast, so they are deleted: path-regex rules are matched by a plain linear scan, still allocation-free on the no-match path. For realistic super_user-authored rule sets this scan is negligible; RE2.Set is the drop-in native multi-pattern scan if path-regex volume ever dominates. Contract change: RE2 cannot evaluate backreferences or lookaround, so a rule using them is now rejected as invalid (same as malformed syntax) rather than silently compiled. The M3 adversarial tests are rewritten to pin this rejection and the ReDoS immunity in place of the removed gate's behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@cb1kenobi did you see @uhop built this |
1 participant
Summary
Introduces
compileSafeRegex(security/safeRegex.ts), a linear-time regex compiler backed by there2native addon, as the safe substrate for any regex whose source is operator- or attacker-influenced. RE2 is a finite-automaton engine with a linear-time matching guarantee, so a pathological pattern such as(a+)+$cannot trigger catastrophic backtracking and stall the event loop the waynew RegExp(...)can (ReDoS).This is deliberately split out from #517 (the WAF component) so the native-dependency introduction is reviewed on its own terms — prebuilt-matrix resolution, install-time behavior, and the linear-time/feature-rejection contract — separate from the WAF matcher logic. The WAF PR will then swap its
compileRuleRegexinternals onto this substrate and delete its interim JS-RegExpalternation-gate machinery.What's in it
security/safeRegex.ts—compileSafeRegex(source, where, errors): RegExp | undefined. Signature mirrors an error-collecting validator. The returned object is anRE2, whichextends RegExp, so it is drop-in for the.test()/.exec()surface at call sites.unitTests/security/safeRegex.test.mjs— 5 tests pinning the two properties that justify the dependency:(a+)+$against a 50k-char non-matching input completes in<250ms(measured ~0.08ms;RegExpwould not return within the test timeout).\1) and lookaround ((?=…)) are unsupported in linear-time RE2 and surface as compile errors, alongside ordinary invalid-syntax errors. Callers treat "unsupported feature" and "invalid syntax" the same way: the rule does not compile.Loading the module at all also exercises that the
re2native binary resolves on the CI matrix.package.json/package-lock.json—"re2": "^1.26.0"plus its install-time subtree (node-gyp/tar/install-artifact-from-github) used only for the source-build fallback.Prebuilt-matrix coverage
re2publishes prebuilt binaries covering every ABI Harper targets (engines: ^22.18.0 || >=24.0.0):Harper's Docker bases (Debian
node:, RHEL UBI, Ubuntu-CUDA) are all glibc → covered; musl is covered too if an Alpine variant is ever used.Residual risks (not blockers)
nan+ fetch-from-GitHub model.re2binaries are Node-major-specific and fetched from GitHub releases at install (not the npm registry). A future Node major landing beforere2publishes its matching ABI would force anode-gypsource build — the ongoing tax of anynan-based addon.build-tools/download-prebuilds.jsis lmdb/msgpackr-specific;re2self-fetches duringnpm ci, so no wiring is needed there.Note for the WAF follow-up
re2also shipsRE2.Set— a native multi-pattern set matcher (new RE2.Set([...]), returns matched pattern indices). That is the correct replacement for #517's combined-alternation pre-gate: no capture-group renumbering, so thehasBackreference/ gateable-vs-ungated split machinery deletes entirely.Tests & checks
.tssource with the realre2binary (Node 24 / ABI 137, linux x64).Generated with Claude Code (Opus 4.8), reviewed by Kris.