Commit f9172e1
committed
Document non-obvious lexer dispatch conditions
Three review-noted spots that were terse in the code:
- The remaining_tokens() loop guard now spells out why both EOF
and `null === token_type && bytes_already_read > 0` are needed
(EOF on clean end-of-input vs invalid byte mid-stream, with
the `> 0` guard letting the very first iteration through).
- The identifier/keyword fast path now explains `$byte > "\x7F"`
(UTF-8 multi-byte starter; MySQL identifiers allow U+0080-U+FFFF)
and `next_byte !== "'"` (only single quotes form the special
hex/bin/n-char literal starters; `"` never does, regardless of
SQL mode).
No behavior change.1 parent 30c557c commit f9172e1
1 file changed
Lines changed: 6 additions & 1 deletion
Lines changed: 6 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2301 | 2301 | | |
2302 | 2302 | | |
2303 | 2303 | | |
| 2304 | + | |
| 2305 | + | |
2304 | 2306 | | |
2305 | 2307 | | |
2306 | 2308 | | |
| |||
2421 | 2423 | | |
2422 | 2424 | | |
2423 | 2425 | | |
2424 | | - | |
| 2426 | + | |
| 2427 | + | |
| 2428 | + | |
| 2429 | + | |
2425 | 2430 | | |
2426 | 2431 | | |
2427 | 2432 | | |
| |||
0 commit comments