Skip to content

serialiser: tiered growth in FastByteBuffer.ensureCapacity#271

Open
num0001 wants to merge 1 commit into
fair-acc:mainfrom
num0001:feat/fastbytebuffer-tiered-grow
Open

serialiser: tiered growth in FastByteBuffer.ensureCapacity#271
num0001 wants to merge 1 commit into
fair-acc:mainfrom
num0001:feat/fastbytebuffer-tiered-grow

Conversation

@num0001
Copy link
Copy Markdown

@num0001 num0001 commented May 22, 2026

Replace the single +12.5% / clamp(1 KiB, 100 KiB) policy with three regimes:

  • small (< 8 KiB): double, with 1 KiB floor preserved
  • medium (< 8 MiB): +12.5% (unchanged from prior policy)
  • large (>= 8 MiB): +50% capped at 16 MiB

Previously, the 100 KiB max-add clamp turned large-buffer growth into a linear (O(n^2)) regime: a 10 MiB buffer doubling needed ~100 reallocs. Multiplicative-capped growth bounds this to a logarithmic number of reallocs while keeping single-step memory waste below 16 MiB.

The small-tier 1 KiB floor preserves the cache-reuse behaviour exercised by the existing testFastByteBufferResizing test.

@num0001 num0001 had a problem deploying to configure coverage May 26, 2026 09:03 — with GitHub Actions Failure
@num0001 num0001 had a problem deploying to configure coverage May 26, 2026 09:03 — with GitHub Actions Error
@RalphSteinhagen
Copy link
Copy Markdown
Member

@num0001 structurally this looks OK, could you please follow-up the following two items before I can merge this:

  1. check/verify that the CI passes, there seem to be some issues that prevent building/testing.
  2. Remove the Claude co-author DCO -- if you contribute code, this is effectively your own

Regarding the latter: we are OK with the use of LLM such as Claude -- I consider them as spell-, grammar-, and thesaurus-checker on steroids” that are good for this type of refactoring. At the same time, this does not drop the requirement that you need to know and be able to explain what it did because the authorship is ultimately yours.
I.e. while I am OK with using LLM -- the final QA/checks need to remain to be done by humans and shall not be delegated to LLMs only. LLMs are not humans and should be acknowledged as such.

If you are more interested in this topic, there is an OPed by us here.

@num0001
Copy link
Copy Markdown
Author

num0001 commented May 27, 2026

Thanks for the clear feedback — I'll address both items.

On authorship: agreed, and the OpEd link is a useful read. I'll drop the Co-Authored-By trailer and force-push. To address the underlying point directly: I used Claude as a research/refactoring aid (locating the TODO, drafting growth-policy options, walking through the testFastByteBufferResizing cache-reuse trace by hand to confirm the existing test still passes). The final design choices — keeping the 1 KiB floor in the small tier, adopting your multiplicative-capped suggestion for the large tier, the 8 KiB / 8 MiB / 16 MiB constants — are decisions I can defend and explain; happy to do so if anything's unclear during review.

On CI: I'll pull the failing job logs and report back here once I have a fix. If the failures are environmental (JDK/dependency) rather than caused by this change, I'll flag that explicitly rather than paper over them.

Replace the single +12.5% / clamp(1 KiB, 100 KiB) policy with three regimes:
 - small  (<  8 KiB): double, with 1 KiB floor preserved
 - medium (<  8 MiB): +12.5% (unchanged from prior policy)
 - large  (>= 8 MiB): +50% capped at 16 MiB

Previously, the 100 KiB max-add clamp turned large-buffer growth into a
linear (O(n^2)) regime: a 10 MiB buffer doubling needed ~100 reallocs.
Multiplicative-capped growth bounds this to a logarithmic number of
reallocs while keeping single-step memory waste below 16 MiB.

The small-tier 1 KiB floor preserves the cache-reuse behaviour
exercised by the existing testFastByteBufferResizing test.

Signed-off-by: num0001 <numberonedev6@gmail.com>
@num0001 num0001 force-pushed the feat/fastbytebuffer-tiered-grow branch from 92b5cc8 to 3d899a7 Compare May 27, 2026 10:58
@num0001 num0001 had a problem deploying to configure coverage May 27, 2026 11:12 — with GitHub Actions Failure
@num0001 num0001 had a problem deploying to configure coverage May 27, 2026 11:12 — with GitHub Actions Failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants