Skip to content

feat: replace Buffer-based base64 paths with engine-neutral paths#98

Draft
Phillip9587 wants to merge 1 commit into
jshttp:masterfrom
Phillip9587:exodus-bytes
Draft

feat: replace Buffer-based base64 paths with engine-neutral paths#98
Phillip9587 wants to merge 1 commit into
jshttp:masterfrom
Phillip9587:exodus-bytes

Conversation

@Phillip9587
Copy link
Copy Markdown
Contributor

Migrate base64 encode/decode paths away from Node.js-specific Buffer to @exodus/bytes for cross-engine compatibility. Add base64 benchmarks for Buffer, @exodus/bytes, base64-js, and Uint8Array.fromBase64()/toBase64() with TextEncoder/TextDecoder, using runtime guards.

https://github.com/ExodusOSS/bytes does the heavy lifting of choosing the right implementation per environment.

Performance Comparison accross engines:

https://docs.google.com/spreadsheets/d/1GnQYzrzEdF3Ea1hJUoEkEZEVNUI3ve8PKPFhLWcEoBI/edit?gid=0
https://github.com/ExodusOSS/bytes/blob/main/Performance.md

I decided to use the strict utf8 methods that throws on invalid UTF-8 byte sequences. We can also use the loose methods that uses the replacement char instead of throwing but I think the strict method is more correct.

Migrate base64 encode/decode paths away from Node.js-specific Buffer to @exodus/bytes for cross-engine compatibility.
Add base64 benchmarks for Buffer, @exodus/bytes, base64-js, and Uint8Array.fromBase64()/toBase64() with TextEncoder/TextDecoder, using runtime guards.
@Phillip9587 Phillip9587 requested a review from blakeembrey May 13, 2026 19:22
@codecov
Copy link
Copy Markdown

codecov Bot commented May 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (1ba386f) to head (174b60f).

Additional details and impacted files
@@            Coverage Diff            @@
##            master       #98   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            1         1           
  Lines           55        61    +6     
  Branches        20        22    +2     
=========================================
+ Hits            55        61    +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blakeembrey
Copy link
Copy Markdown
Member

@Phillip9587 Using Buffer still seems incredibly fast when I check out this PR locally, are you seeing something different?

  decode base64-js - src/base64.bench.ts > base64 decode - short
    1.14x faster than decode Buffer
    2.38x faster than decode Uint8Array.fromBase64
    2.90x faster than decode @exodus/bytes

  encode Buffer - src/base64.bench.ts > base64 encode - short
    1.70x faster than encode @exodus/bytes
    2.50x faster than encode Uint8Array.toBase64
    3.28x faster than encode base64-js

  decode Buffer - src/base64.bench.ts > base64 decode - long
    1.97x faster than decode Uint8Array.fromBase64
    2.34x faster than decode @exodus/bytes
    2.51x faster than decode base64-js

  encode Buffer - src/base64.bench.ts > base64 encode - long
    2.25x faster than encode Uint8Array.toBase64
    2.38x faster than encode @exodus/bytes
    10.60x faster than encode base64-js

Makes me feel like it should still be Buffer and fallback on Uint8Array? Do you want older browser support without Uint8Array?

@blakeembrey
Copy link
Copy Markdown
Member

We may also want to consider applying size-limit if this is intended for browsers and other environments. I'm not sure pulling in that dependency is worth it.

@Phillip9587
Copy link
Copy Markdown
Contributor Author

@blakeembrey Hey, I took a look at your refactoring of my PR in #95. Please give me some time to explain my reasoning behind this PR versus #95. It’s currently holiday in Germany, so I’m away for the weekend.

@blakeembrey
Copy link
Copy Markdown
Member

Of course, no rush! In case it helps, my context is that it's a trade-off between package size and performance. Since Buffer is fastest anyway, and non-Buffer environments are package size constrained, it seemed best balance to ship something fast and small with the option of having those other environments opt-in via encode/decode options or polyfills to the larger package size.

@Phillip9587
Copy link
Copy Markdown
Contributor Author

I ran the same comparison locally, and I agree that on Node, Buffer is always going to be the fastest option. The reason I would still use @exodus/bytes is that this PR is changing the package from using a Node-specific primitive to using runtime-neutral byte/string primitives.

It's not just about performace it is whether we want basic-auth itself to own all the runtime-specific encoding logic and fallback behavior. I don't want to reinvent the wheel here by adding our own mix of Buffer, TextEncoder, TextDecoder, atob, Uint8Array.fromBase64, and fallback branches, then also be responsible for making sure they all behave consistently across Node, browsers, workers, and other runtimes.

Buffer also has more permissive parsing semantics than I'd want as the validation boundary here: invalid UTF-8 is repaired with U+FFFD instead of throwing, and base64 decoding is intentionally forgiving.

For basic-auth, the input size is tinyy as we are decoding an Authorization header, not large payloads. Performance still matters, but at this scale I think the trade-off is worth the consistency we get in auth parsing and formatting across runtimes.

We are also only using the two focused exports we need, @exodus/bytes/utf8.js and @exodus/bytes/base64.js. That means we are pulling in a small subset of what the package provides, rather than the whole surface area, which keeps the bundle impact small in environments where bundle-size matters.

I hope this explains why I prefer @exodus/bytes here. It gives us well tested Uint8Array-first helpers for base64 and UTF-8, with strict decoding behavior and we can avoid maintaining a homegrown cross-runtime layer. To me, that is the cleaner maintenance boundary.

@blakeembrey
Copy link
Copy Markdown
Member

The easy solution here is to allow encode/decode to be passed as an option, and document it in the README.

The dependency makes the package both slower and larger. From 529 B to 3.43 kB (almost 7x).

I understand the concerns around maintaining multiple code paths, but I'm not convinced it's worth the trade off. What consistency and formatting are you concerned about across runtimes? Since this packages primary consumer is node.js, and presumably any non-node runtime will be a more up to date, having only two supported paths (Buffer and Uint8Array) should be plenty.

Another thing that's possible with semver is to adopt this later if it's requested, while it's not going to be possible to remove it without a major. I'd personally opt for the smaller and faster package, and upon request from users, evaluate expanding scope.

@blakeembrey
Copy link
Copy Markdown
Member

blakeembrey commented May 20, 2026

I'd prefer using only Uint8Array and skip Buffer entirely over adding a dependency for this if we're really worried about the two different code paths. The only reason for keeping Buffer was the clearly superior performance, and to a much lesser extent supporting old node versions. I think there'd be a reasonable argument to even go node 25+ for the major (if we're dropping Buffer), as we can always expand support to older node versions upon request.

@Phillip9587
Copy link
Copy Markdown
Contributor Author

Makes sense. I’m good with both directions.

The consistency concern I had was mainly at the codec boundary: strict UTF-8 vs loose with replacement characters, permissive base64 handling, and avoiding subtle differences between runtime APIs. But I agree that this is probably better handled as an escape hatch instead of pulling in a dependency by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants