Compiler: Smalltalk source → Tinytalk bytecode

[softwarewrighter]

Today every demo in `examples/dN_*.bas` is hand-assembled bytecode:
the driver POKEs bytecodes into scratch RAM, then `GOSUB 12000`
dispatches them through the VM. This costs ~30-60 lines of fragile
BASIC per demo and means new demos spend more time on loader
plumbing than on the logic they're trying to show.

A Smalltalk-source compiler — parser + codegen producing the same 14
Tinytalk bytecodes — would let future demos be `.st` files. Concretely:
`examples/fibonacci.st` instead of `examples/d8_fibonacci.bas`.

Scope

Lexer / parser

• Tokenize: integer literal, identifier, keyword (trailing `:`),
  binary selector (`+ - * / < > = , ;` etc.), block delimiters
  `[ ]`, statement terminator `.`.
• Parse statements: `expr.` and `^ expr.` (return).
• Parse expressions: receiver + cascade of unary / binary /
  keyword messages with standard Smalltalk precedence
  (unary > binary > keyword).
• Block literals `[ :a :b | body ]` (only after v2 blocks).
• Class declaration syntax: at minimum a file format declaring
  class X with instance variables A B C and methods M N P.

Codegen

• PUSH_INT for integer literals.
• PUSH_TEMP / STORE_TEMP / PUSH_FIELD / STORE_FIELD with a
  per-method symbol table.
• SEND for unary / binary / keyword sends with correct arg count.
• RETURN_TOP / POP / JUMP / JUMP_IF_FALSE for control flow.
• Literal pool indices (depends on v1 strings, v1: strings + literal pool + Transcript primitive #4).

Image build

• Replace per-demo `image_dN.bas` POKE / DATA tables with the
  compiler's output: class table + method dictionaries + bytecode
  pool in a single image format the VM loads.
• CLI: `stc examples/foo.st > image/foo.bin` or similar.

Where to host

1. In Smalltalk — dogfooded; v1+v2 unlock enough to self-host.
   Bootstrap by hand-assembling the compiler once, then it
   self-hosts.
2. In Pascal or Rust — pragmatic for the v0..v2 era; ship
   `tools/stc.pas` (or in `pa24r` style, a Rust crate) that
   produces an image at CLI time.

(2) ships sooner and unblocks demos; (1) is the philosophically
correct end state.

Demos this unlocks

• All ten v0-feasible demos in Demo backlog: more v0 demos (no dialect upgrade required) #1 become `.st` files.
• Higher-tier demos (`eliza-lite`, `mud-room`, `caesar-cipher`,
  `roman-numerals`, `test-runner`) can be written in real
  Smalltalk syntax once v1 strings + this compiler exist.
• D5 calc could grow into a real source-driven REPL: user types
  Smalltalk, compiler emits bytecode, VM runs it.

Web demo

`web-sw-cor24-smalltalk`
would either:

• Continue bundling per-demo BASIC images, but generate them at
  build time from `.st` sources via the compiler, or
• Ship the compiler in the WASM bundle so the browser compiles
  user-typed Smalltalk to bytecode — a real REPL.

The latter is the killer-app version of the live demo.

Related: dialect roadmap (#2), v1 strings (#4).

[softwarewrighter]

Tracked. Coverage of this issue is split across sagas:

v0-bootstrap saga (current, in flight)

• Step 015 smalltalk-source-and-compiler — minimal stc covering D2-shaped source: class with slots, methods, primitive directive, ^ expr, var := expr, atoms (self / int / slot), left-to-right binary sends. CLI is tools/stc.awk < x.st > image.bas.
• Step 016 migrate-demos-to-st — extends syntax as needed for D1, D3..D7. Adds keyword sends (for D3 ifTrue:ifFalse:, D4 max:), JUMP_IF_FALSE lowering for D4 / D6, and subclass declarations for D7.

v1-dialect saga (planned, post-v0.1)

• String literals + literal pool (v1: strings + literal pool + Transcript primitive #4 anchor work).
• Standard Smalltalk precedence (unary > binary > keyword).
• Cascades.
• Block literals (depends on v2 blocks per Dialect roadmap: v1 strings → v7 processes #2).

Out of scope for this repo:

• Self-hosted compiler (Smalltalk-in-Smalltalk) — needs v2 blocks first.
• Compiler in WASM bundle for browser REPL — belongs in web-sw-cor24-smalltalk.

docs/status.md 'Future sagas' section now points at this issue. Closing this comment thread; the issue stays open until the v1 dialect saga lands the keyword + block + precedence pieces.

[softwarewrighter]

First slice has shipped: `tools/stc.awk` (commit 2936962) plus
58a4e9a
which migrated all seven foundational demos to `.st` source. Notes
on what works and what's still ahead:

Currently supported (v0-aligned)

• `class CLASS [extends PARENT] [slots V1 V2 ...]` declarations.

• `method SELECTOR` bodies with `primitive N` and a small
  statement set (`^ expr`, `var := expr`, `if expr`).

• Expressions: integer literal, `self`, `(expr)`, identifier,
  binary ops `+ - * < =`, unary chains. Standard Smalltalk
  precedence (unary > binary > keyword).

• All 7 demos compile and run end-to-end. Confirmed against the
  CLI:

  demo	expect	got
  d1_add	7	7
  d2_counter	2	2
  d3_boolean	42	42
  d4_max	5	5
  d6_fact	120	120
  d7_bounded	5	5

  D5 calc verified interactively in the web sandbox.

Hosting choice

Took option (3) — a third path I didn't list: `awk`. Smaller and
more portable than (1) Smalltalk-self-host or (2) Pascal/Rust.
Works fine for v0 but probably caps out at v1+ scope (string
literals, blocks, hygienic temp/arg shadowing).

Still open from the original scope

• Block literals `[ :a :b | body ]` — needs v2 dialect.
• Keyword sends (`a max: b`, `obj at: k put: v`) — currently
  binary-only at the parse layer.
• Cascades (`obj msg1; msg2; msg3`) — would unlock more natural
  expression-style demos.
• Literal pool for strings — gates by v1: strings + literal pool + Transcript primitive #4.
• A real source-driven REPL (web Dialect roadmap: v1 strings → v7 processes #2) where the user types
  Smalltalk in the browser and the compiler runs there too. Would
  benefit from porting / re-implementing stc.awk in Rust so it
  ships in the WASM bundle.

Web demo

`web-sw-cor24-smalltalk`'s
`build.rs` now invokes `tools/stc.awk` for each demo's `.st` at
build time. Source pane shows the canonical `.st` (resolved
web #1).