Skip to content

perf(stdlib): parse string.format flags once, dispatch on integer specifier#317

Merged
davydog187 merged 3 commits into
mainfrom
perf/string-format-flags-bitmask
Jun 1, 2026
Merged

perf(stdlib): parse string.format flags once, dispatch on integer specifier#317
davydog187 merged 3 commits into
mainfrom
perf/string-format-flags-bitmask

Conversation

@davydog187
Copy link
Copy Markdown
Contributor

perf(stdlib): parse string.format flags once, dispatch on integer specifier

Plan: .agents/plans/B12-string-format-flags-bitmask.md
Closes #309

Goal

Speed up the string.format per-specifier hot path in
lib/lua/vm/stdlib/string.ex with zero change to observable output:

  1. Parse format flags exactly once, at parse time, into an integer
    bitmask, so the apply path reads precomputed bits instead of
    re-scanning the flags binary with String.contains?/2 on every
    specifier application.
  2. Store the conversion char as a raw integer code point (?d/?f/…)
    so the apply_format_spec/2 case dispatches on BEAM
    integer patterns rather than one-byte binaries.

Success criteria

  • string.format produces byte-for-byte identical output for all
    existing tests — mix test test/lua/vm/string_test.exs green
    (152 passed, 41 properties). The plan named
    test/lua/vm/stdlib/string_test.exs, but the format coverage lives
    in test/lua/vm/string_test.exs; see Discoveries.
  • Flags parsed exactly once into a fixed-shape integer mask; no
    String.contains?(flags, ...) re-scan remains in
    apply_width_flags/3 (now reads flags &&& @flag_minus /
    @flag_zero).
  • Conversion specifier stored/matched as an integer code point; the
    case dispatches on ?d/?f/… and the invalid-option error
    re-renders the char (<<specifier>>) so the message is unchanged.
  • Flags parsed-but-ignored (+, space, #) still carried in the
    mask but not consulted downstream — output unchanged.
  • mix compile --warnings-as-errors clean.
  • Full mix test passes (2114 passed, 19 skipped, 1 excluded).
  • mix run benchmarks/string_format.exs runs to completion; numbers below.

Benchmark (Apple M4, mode: quick) — lua (chunk)

Workload Before (ips / avg) After (ips / avg) Delta
long literal-heavy 591.35 / 1.69 ms 692.16 / 1.44 ms ~+17% ips
width-flagged specs 253.20 / 3.95 ms 359.39 / 2.78 ms ~+30% ips
many specifiers 183.66 / 5.44 ms 225.48 / 4.44 ms ~+23% ips

The width-flagged workload (which exercises the padding path on every
conversion, where the String.contains?/2 re-scan lived) sees the
largest win, as expected.

Changes

 lib/lua/vm/stdlib/string.ex | 67 +++++++++++++++++++++++++++++----------------
 1 file changed, 43 insertions(+), 24 deletions(-)

Verification output tail

mix test test/lua/vm/string_test.exs
Result: 152 passed (41 properties, 111 tests)

mix test
Result: 2114 passed (60 doctests, 51 properties, 2003 tests), 19 skipped, 1 excluded

Out of scope

Discoveries

  • The plan's verification referenced test/lua/vm/stdlib/string_test.exs,
    which does not exist. The string.format coverage lives in
    test/lua/vm/string_test.exs; that file plus the full mix test run
    were used as the correctness gate.
  • benchmarks/string_format.exs lists luaport as an optional dep that
    needs C Lua headers (absent here); its dep compile aborts mix run
    before the runtime skip-guard fires. The lua-vs-luerl numbers above
    were captured with luaport temporarily excluded from mix.exs for
    the bench run only; mix.exs is unmodified in this PR.

…cifier

Parse format flags a single time at parse time into an integer bitmask
instead of re-scanning the flags binary with String.contains?/2 on every
specifier application, and store the conversion char as a raw code point
so apply_format_spec/2 dispatches on BEAM integer patterns rather than
one-byte binaries. Pure internal refactor: formatted output is unchanged.

Plan: B12
Closes #309
Copy link
Copy Markdown
Contributor Author

@davydog187 davydog187 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: clean-with-nits

Automated review (skeptical senior-reviewer pass). Read-only; no self-approve since PR author == reviewer, so this is a comment.

Verified this is a genuine perf-only refactor with no observable output change. Specifics checked:

Correctness (output is byte-for-byte equivalent)

  • Old parse_flags/2 guarded on ~c(-+ 0#) = exactly [?-, ?+, ?\\s, ?0, ?#]. The new per-char clauses cover precisely those five bits. Equivalent set. ✓
  • Order never mattered: the old code only ever consulted the flags binary via String.contains?(flags, "0") / (flags, "-"), so collapsing to an unordered integer mask is faithful. ✓
  • All 14 dispatch branches converted consistently to integer code points (?d?q) — no "d"/?d mismatch left behind (lib/lua/vm/stdlib/string.ex:432-447). ✓
  • Invalid-option error re-renders the char via <<specifier>> (string.ex:447), so invalid option '%z'-style messages are unchanged. ✓
  • apply_width_flags/3 (string.ex:858-859) now reads flags &&& @flag_minus/@flag_zero; all three padding branches (left-justify, right-justify, sign-aware zero-pad at :864) preserved one-to-one. ✓
  • The only surviving String.contains? (string.ex:633) is the float . check in expand_float — unrelated to flags, correctly untouched. ✓

Tests — coverage pins the behavior: example tests for every specifier (d/i/u/f/e/E/g/G/x/X/o/c/s/q) plus property tests for %d/%x/%X/%o/%s/%c, width, left-justify (%-Nd), zero-pad (%0Nd), precision (%.Nf/%.Ns), and the sign-aware zero-pad branch (%06d over -42, string_test.exs:398). The UTF-8 %6s/%-6s cases pin byte-width padding. Output drift would turn these red. All CI green (test matrix x2, Dialyzer, Lua 5.3 suite).

Scope — clean: only .agents/plans/B12-…md + lib/lua/vm/stdlib/string.ex. No bleed into the width (#316) or float-io_lib (#319) sibling regions.

Findings

  • [nit] Commits 26ec3bd and aa875d5 use chore(B12): — plan id as the commit scope, which the repo conventions (.agents/skills/ship-a-plan + CLAUDE.md) explicitly forbid; scope should name the subsystem. The substantive perf commit (0f5fa90) is correct: perf(stdlib): … subject with Plan: B12 in the body. Non-blocking, but worth squashing/renaming the bookkeeping commits on the next pass.

No blockers, no majors. No plan-id leakage into source/comments, no Co-Authored-By AI trailer. Ship it.

@davydog187 davydog187 merged commit d395302 into main Jun 1, 2026
5 checks passed
@davydog187 davydog187 deleted the perf/string-format-flags-bitmask branch June 1, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(stdlib): parse string.format flags into bitmask, integer specifier

1 participant