-
Notifications
You must be signed in to change notification settings - Fork 9
Move ecosystem-neutral helpers out of Codex-named homes #465
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| """Making untrusted values safe to echo into violation messages. | ||
|
|
||
| Any rule that quotes a manifest or config value in a diagnostic routes | ||
| it through here, whatever ecosystem the rule belongs to. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import re | ||
|
|
||
| # C0, DEL, C1, and the Unicode bidi overrides — any of them can reorder | ||
| # or hide message text in a terminal or a rendered SARIF viewer. | ||
| _CONTROL_CHARS = re.compile(r"[\x00-\x1f\x7f-\x9f\u202a-\u202e\u2066-\u2069]") | ||
|
|
||
| # Everything a message needs to locate the defect fits comfortably here; | ||
| # an adversarial multi-kilobyte value must not become a multi-kilobyte | ||
| # diagnostic in a CI artifact. | ||
| _MAX_DISPLAY = 500 | ||
|
|
||
|
|
||
| def _redact_userinfo(text: str) -> str: | ||
| """Strip credential-shaped userinfo before every ``@`` in *text*. | ||
|
|
||
| Covers scheme-full ("https://u:tok@h/x"), scheme-relative, bare | ||
| ("u:tok@h/x"), and scp-style ("tok@github.com:o/r.git") spellings: | ||
| the segment between the last ``/``/whitespace and an ``@`` is | ||
| redacted when it carries a colon (credential shape) or when the text | ||
| after the ``@`` looks host-like (a dot or colon before the next | ||
| whitespace). Over-redacting an email-shaped value in a path field is | ||
| the safe direction. A linear right-to-left scan — no regex, so there | ||
| is nothing to backtrack, and no length cap for a long token to slip | ||
| past. | ||
| """ | ||
| if "@" not in text: | ||
| return text | ||
| out = [] | ||
| emitted = 0 | ||
| search_from = 0 | ||
| length = len(text) | ||
| while True: | ||
| at = text.find("@", search_from) | ||
| if at == -1: | ||
| out.append(text[emitted:]) | ||
| return "".join(out) | ||
| # The backward window is floored like the forward host cap below: | ||
| # without it, a delimiter-free prefix is rescanned for every ``@`` | ||
| # and the scan goes quadratic on adversarial input. When the | ||
| # clipped window holds no delimiter the credential may extend past | ||
| # it, so redact from the last emit point — never clamp ``start`` | ||
| # into the middle of a secret and emit its head. | ||
| floor = max(emitted, at - 512) | ||
| found = max(text.rfind(ch, floor, at) for ch in ("/", " ", "\t", "\n", "@")) | ||
| start = found + 1 if found != -1 else emitted | ||
| userinfo = text[start:at] | ||
| if not userinfo: | ||
| search_from = at + 1 | ||
| continue | ||
| # Host inspection is capped: a host longer than any real one with | ||
| # no whitespace is treated as host-like, which errs toward | ||
| # redaction — the safe direction. | ||
| head_limit = min(at + 1 + 512, length) | ||
| host_head = text[at + 1 : head_limit] | ||
| ws = next((i for i, ch in enumerate(host_head) if ch.isspace()), None) | ||
| if ws is not None: | ||
| host_head = host_head[:ws] | ||
| host_like = "." in host_head or ":" in host_head | ||
| else: | ||
| host_like = "." in host_head or ":" in host_head or head_limit < length | ||
| if ":" in userinfo or host_like: | ||
| out.append(text[emitted:start]) | ||
| out.append("[redacted]") | ||
| emitted = at | ||
| search_from = at + 1 | ||
|
|
||
|
|
||
| def safe_display(value: object) -> str: | ||
| """A manifest value made safe to echo into a violation message. | ||
|
|
||
| Reports are uploaded as CI artifacts and ingested as SARIF, so an | ||
| author's pasted ``user:token@host`` URL must not ride along — the | ||
| userinfo is redacted, keeping the locator. Control characters are | ||
| replaced so a crafted value cannot smuggle terminal escapes through | ||
| the text formatter, and the result is length-bounded. | ||
| """ | ||
| raw = str(value) | ||
| truncated = len(raw) > _MAX_DISPLAY | ||
| # Truncate before scanning so the display cap bounds the *work*, not | ||
| # just the output — redaction over the full value is quadratic-ish in | ||
| # the worst case, and a multi-megabyte manifest value must not buy | ||
| # minutes of CPU for one diagnostic. | ||
| text = raw[:_MAX_DISPLAY] | ||
| if truncated: | ||
| # The cut can sever a credential ahead of its ``@`` — the window | ||
| # then holds a bare colon-bearing segment redaction would never | ||
| # match. Treat the cut like an ``@``: redact a colon-bearing tail | ||
| # segment. Over-redacting a truncated tail is the safe direction. | ||
| start = max(text.rfind(ch) for ch in ("/", " ", "\t", "\n", "@")) + 1 | ||
| if ":" in text[start:]: | ||
| text = text[:start] + "[redacted]" | ||
| text = _CONTROL_CHARS.sub("\N{REPLACEMENT CHARACTER}", text) | ||
| text = _redact_userinfo(text) | ||
| if truncated or len(text) > _MAX_DISPLAY: | ||
| text = text[:_MAX_DISPLAY] + "…" | ||
| return text |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When an existing custom rule imports either previously exposed non-underscore helper (
from skillsaw.formats.codex import safe_existsorsafe_is_symlink), this import list no longer binds it, so custom-rule loading raises anImportErrorand aborts the lint. The repository explicitly preserves legacy import locations for custom rules intests/test_module_layering.py:9-14; keep compatibility re-exports here while makingpaths.pycanonical.AGENTS.md reference: AGENTS.md:L88-L89
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Declining: skillsaw.formats.codex does not exist on main or in any release — it was introduced on this unreleased feature branch, so no external code can import from it. Moving the helpers to their intended home before first release is exactly why no deprecation shim is warranted.