fix(tee): keep head + tail when raw output exceeds max_file_size#1696
Open
iliaal wants to merge 1 commit intortk-ai:developfrom
Open
fix(tee): keep head + tail when raw output exceeds max_file_size#1696iliaal wants to merge 1 commit intortk-ai:developfrom
iliaal wants to merge 1 commit intortk-ai:developfrom
Conversation
The tee log is the recovery hatch when filters truncate. It used head-only truncation at max_file_size (1MB by default), which discarded exactly the information that tail-heavy commands (test runners, build summaries, error trailers) put at the end. So a make test run with 17000+ progress lines would tee the first 1MB of progress and lose the pass/fail summary. Switch to a 25/75 head/tail split: keep the first quarter of the budget (prologue, command echo, banner) and the last three quarters (where the useful output lives for tail-heavy commands), with a marker between documenting how many bytes were dropped. UTF-8 char boundaries are respected on both ends. Updated existing UTF-8 boundary tests to exercise the new split, and added a tail-preservation test that simulates a make test run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before
write_tee_filetruncated raw output atmax_file_size(1MB default) by keeping the first 1MB of bytes and dropping the rest. The tee log is the recovery hatch agents fall back to when the live filter dropped a needed signal, but for tail-heavy commands the head is the part that doesn't matter.Triggered by
rtk make teston a php-src checkout: ~17000 phpt progress lines followed by a pass/fail summary. The make filter kept only the first ~16 progress lines (separate fix onfeat/php-tooling). The tee log captured 1MB of progress lines and also dropped the summary. Same head-bias on a tail-heavy command, two layers down.After
write_tee_filekeeps a 25/75 head/tail slice with a--- N bytes truncated from middle ---marker between them. Default 1MB budget gives ~256KB of head plus ~768KB of tail. The slicer rounds both cuts to UTF-8 char boundaries, so multi-byte input never panics.Tail gets the larger share because tee mostly captures build and test output, where summaries and failure trailers live at the end. The head stays because the prologue often shows the command echo and config that explain what was running.
Scope rationale
max_file_size,RTK_TEE=0,force_tee_hintbehave the same.feat/php-toolingbecause that branch already touches PHP tooling. This PR fixes the recovery layer for every tail-heavy command, not just make.Backward compatibility: marker text changed from
--- truncated at N bytes ---to--- N bytes truncated from middle ---. Visible if anyone parses tee logs programmatically.Test plan
cargo test --bin rtk: 1721 passed, 0 failedcargo clippy --all-targets: no new warningscargo fmt --all: no churntest_write_tee_file_keeps_tail: 3000-line phpt-style run, verifies head + marker + tail all survivetest_write_tee_file_truncation_utf8_boundaryandtest_write_tee_file_truncation_emoji: new split exercised with multi-byte chars at both cuts