Add final trusted TQQQ snapshot acceptance - #222
Conversation
Co-Authored-By: Codex <noreply@openai.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0e6f3fe2a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| prices = tqqq_r1_snapshot._normalized_prices( | ||
| pd.read_csv( | ||
| BytesIO(members["prices.csv"]), | ||
| dtype={tqqq_r1_snapshot.PRICE_FIELD: "string"}, | ||
| ), | ||
| require_exact_columns=True, |
There was a problem hiding this comment.
When each data row contains four fields under the three-column header, pd.read_csv silently treats the leading field as an implicit index, while _normalized_prices(..., require_exact_columns=True) still sees exactly the three named columns. If the shifted session column contains all receipt sessions—for example, shifted rows for both 2010-01-04 and 2010-01-05—the later membership check also passes, so the loader accepts and returns raw prices.csv bytes that violate the three-column snapshot contract. Explicitly reject implicit indexes or otherwise validate every row's raw field count before accepting the snapshot.
Useful? React with 👍 / 👎.
| manifest["session"] != receipt["session"] | ||
| or manifest["session"] not in calendar["sessions"] |
There was a problem hiding this comment.
When the authenticated snapshot contains an extra weekday that is absent from the authenticated XNYS calendar, the package is still accepted as long as the single receipt session appears in both artifacts. The snapshot validator only excludes weekends, so this permits holiday or otherwise non-exchange observations to enter the trusted price history without calendar verification; require all verified_sessions to be contained in the calendar evidence rather than checking only manifest["session"].
Useful? React with 👍 / 👎.
| frozenset[str], | ||
| ]: | ||
| try: | ||
| names = tuple(sorted(os.listdir(directory_fd))) |
There was a problem hiding this comment.
When an untrusted snapshot directory contains a very large number of entries, os.listdir materializes every name and sorted duplicates that work before the verifier can reject the unexpected layout. This bypasses the module's otherwise bounded-input design and can exhaust memory or CPU without possessing any valid hash anchor. Enumerate incrementally and stop after more than the four expected members rather than loading the entire directory.
Useful? React with 👍 / 👎.
|
Fail-closed terminal evidence for immutable head
The permanent stop rule requires |
1 participant
Summary
_normalized_prices(..., require_exact_columns=True, ...)result the single source of canonical snapshot sessions; receipt membership uses only itsfrozenset[str]os.fspath(snapshot_dir)withoutnormpath,resolve, orrealpath; reject any..component, trailing separator, or trailing/.before openTests-first evidence
On the audit-only PR #221 exact head
f2d813e7edbe15094c62e62f453e9ca150dd5b5d, the two frozen regression tests both failed withDID NOT RAISE:symlinked-ancestor/../snapshotwas lexically collapsed and acceptedOn fresh main, the new capability test module was RED because the trusted package did not exist. The complete candidate then passed:
PYTHONPATH=src python3 -m pytest -q tests/test_tqqq_r1_snapshot.py tests/test_tqqq_trusted_snapshot_package.py— 67 passedPYTHONPATH=src python3 -m pytest -q— 612 passed, 3 skippedpython3 -m ruff check src/us_equity_snapshot_pipelines/tqqq_trusted_snapshot_package.py tests/test_tqqq_trusted_snapshot_package.py— passedpython3 -m compileall -q src tests— passedScope and compatibility
Exactly two new files; no dependency, workflow, schema, configuration, provider/data/replay, or live behavior changes. Snapshot roots containing
.., a trailing separator, or trailing/.are now intentionally unsupported and fail closed.size=0.This initial head is the complete immutable final candidate for the single high-risk review/CI gate; no remediation or review rerun is permitted.