# reform-prep-1: parser spans + import/function diagnostics

[vpisarev]

Groundwork for the syntax-reform automigrator (and better diagnostics along the
way): source locations are now true spans, not points, and import/function
errors point at the right thing. Nothing here changes language behavior — only
locations and diagnostics.

Spans (the foundation)

Previously every token carried only its begin (line, col); the parser stamped
a point, so a caret was always a single ^. Now the lexer returns, per batch,
(tokens, batch_begin, batch_end) — after nexttokens' body runs, pos sits
at the batch end, so getloc(pos) is the end loc — and the parser applies that
span to every token. So loc_t is a real span, a caret shows the token width,
and loclist2loc folds true node spans:

val x = lenght + 1
        ^~~~~~        (was a single ^)

Internal recursive nexttokens() calls (string interpolation, numeric-suffix,
brace grouping) take .0; no per-arm changes; the lexer's internal point locs
are untouched.

Diagnostics point at the name

• Imports (a common error — misspelled or off-path module): the caret lands
  on the whole (possibly dotted) module name, not the first token or the from
  keyword, with a Levenshtein did you mean whose candidates are the *.fx
  files on the include path:

  import Strig
         ^~~~~   error: module Strig is not found; did you mean 'String'?
  

• Functions: a function's own diagnostics (implicit-return-type warning,
  redeclaration, the overload candidate listing) point at the NAME, not fun;
  operators span operator X; dotted method names span Class.method.

• Parse errors now carry the same caret excerpt as type errors (they are
  frontend diagnostics with accurate locations); EOF errors degrade to no caret.

• ExpCast/ExpTyped fold to the whole x :> T / x : T construct.

Helper edit_distance moved to Ast.fx so the parser (unknown modules) and the
type checker (unknown identifiers) share one copy; consumed_loc(before, after)
folds a node's span out to the end of a sub-parser that returns no loc.

Reform-relevant findings (filed, not fixed)

• FB-020 — typ_t carries no source location: type-name diagnostics land on
  the preceding :, and a span-based migrator cannot locate 't / half /
  vector occurrences via the AST. The 't → [t] reform will need token-level
  scanning for types, or locs added to typ_t. The biggest gap.
• FB-021 — .op type errors surface inside the library (__dot_mul__
  instantiation site), not at the user's .*.
• FB-022 — a failed fold-valued val cascades because the desugared shape
  sidesteps diag-1's DefVal recovery.

docs/problematic_spans_log.txt is the full audit (live compiler output for
each reform construct).

Verification

test_all 147, fxtest all (unit + negative 87 + ir + cfold + corpus O0/O3),
-pr-resolve census 356 unchanged, rettype gate clean, bootstrap fixpoint holds.
~90 negative goldens migrated to carets (reviewed as a format change — wider
carets, name-column citations).

Not in scope (see docs/reform_prep1_report.md)

WP-2 migration census, WP-3 A/B renders, WP-4 list[t] parser spike, and the
tools/span_check.py acceptance oracle — measurement/tooling deliverables for
the design sessions, plus the FB-020/021/022 fixes.