crl-release-25.4: pebble: log per-table/blob output sizes for flushes and compactions#6119
Merged
Conversation
Member
|
This change is |
|
Linter isn't happy because formatting |
fqazi
reviewed
Jul 1, 2026
| }) | ||
| } | ||
| } | ||
| // TestParseEndWrittenBytes exercises the full match -> parse path for compaction |
fqazi
approved these changes
Jul 1, 2026
Flush and compaction event log lines previously printed the output sstable file numbers and a single aggregate size, e.g. `-> L6 [000123 000124] (3.0MB)`. Compaction lines did not print output blob files at all, even though `CompactionInfo.Output.Blobs` was already populated. This change prints size details for each output table and each output blob file, which is cheap information that helps with debugging. Three helpers are added in `event.go`: - `FormatTablesWithSizes` renders each table as `filenum(size+refsize)`, omitting `+refsize` when the blob reference size is zero. - `FormatBlobsWithSizes` renders each blob file as `filenum(size)`, plus the MVCC garbage percentage when present. - `formatOutputBlobs` builds the shared ` blob(s) [...] (...)` segment used by both the flush and compaction log lines. Both `FlushInfo.SafeFormat` and `CompactionInfo.SafeFormat` now use these helpers; the aggregate total is also shown as `size+refsize`, and compaction lines now include the output blob segment. The `tool/logs` compaction/flush log parser extracts the aggregate output size from these lines. Its regexes are updated to anchor the byte capture on the closing `]` of the file list, so they skip the per-file sizes now embedded in the list (and the trailing blob segment), and to accept `+` without surrounding spaces. Both the compaction and flush end-line parsers now use `unHumanizeSum` to handle the `table + blob` sum. This is backward compatible with older log formats; new tests cover the new format and the match-to-parse path. Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
RaduBerinde
force-pushed
the
pebble-log-per-table-blob-output-sizes-for-flushes-25.4
branch
from
65e93f6 to
b5162a6
Compare
RaduBerinde
deleted the
pebble-log-per-table-blob-output-sizes-for-flushes-25.4
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flush and compaction event log lines previously printed the output sstable
file numbers and a single aggregate size, e.g.
-> L6 [000123 000124] (3.0MB). Compaction lines did not print output blob files at all, eventhough
CompactionInfo.Output.Blobswas already populated.This change prints size details for each output table and each output blob
file, which is cheap information that helps with debugging. Three helpers are
added in
event.go:FormatTablesWithSizesrenders each table asfilenum(size+refsize),omitting
+refsizewhen the blob reference size is zero.FormatBlobsWithSizesrenders each blob file asfilenum(size), plus theMVCC garbage percentage when present.
formatOutputBlobsbuilds the sharedblob(s) [...] (...)segment used byboth the flush and compaction log lines.
Both
FlushInfo.SafeFormatandCompactionInfo.SafeFormatnow use thesehelpers; the aggregate total is also shown as
size+refsize, and compactionlines now include the output blob segment.
The
tool/logscompaction/flush log parser extracts the aggregate outputsize from these lines. Its regexes are updated to anchor the byte capture on
the closing
]of the file list, so they skip the per-file sizes nowembedded in the list (and the trailing blob segment), and to accept
+without surrounding spaces. Both the compaction and flush end-line parsers
now use
unHumanizeSumto handle thetable + blobsum. This is backwardcompatible with older log formats; new tests cover the new format and the
match-to-parse path.
Co-Authored-By: roachdev-claude roachdev-claude-bot@cockroachlabs.com