Skip to content

architecture.md

Latest commit

 

History

History
289 lines (232 loc) · 15.8 KB

architecture.md

File metadata and controls

289 lines (232 loc) · 15.8 KB
Raw

Architecture

This page is for contributors who want to understand or modify the engine. It traces a template from text to rendered output and points at the types that do each job.

High‑level pipeline

flowchart TD
    src["Template text"] --> lex

    subgraph parse["DocumentParser.Parse (steps 1-3)"]
        lex["1. LEX<br/>HeddleLexer (ANTLR, mode stack)"] -->|tokens| prs["2. PARSE<br/>HeddleParser - SLL, fallback LL"]
        prs -->|parse tree| walk["3. WALK<br/>HeddleMainListener + ParseContext"]
        prs -.->|syntax errors| errs["HeddleSyntaxErrorListener"]
    end

    subgraph compile["HeddleCompiler.Compile (steps 4-5)"]
        cmp["4. COMPILE (HeddleCompiler)<br/>extensions + compiled accessors"] --> mem["member paths -> expression-tree delegates"]
        cmp --> cs["@(...) C# -> Roslyn delegates"]
    end

    walk -->|"definitions, output chains"| cmp
    cmp --> rt["RuntimeDocument (+ IProcessStrategy)"]
    rt --> render["6. RENDER<br/>HeddleTemplate.Generate(data)<br/>ScopeRenderer -> string"]
    errs -.->|collected| result["HeddleCompileResult"]
    cmp -.-> result
Loading

The orchestration entry points are DocumentParser.Parse (steps 1–3) and HeddleTemplate.Compile → HeddleCompiler.Compile (steps 4–5).

1. Lexing

The lexer is generated from HeddleLexer.g4 (which imports CSharp.g4 for C# tokens). It is mode‑based: a stack of lexer modes makes the language context‑sensitive so that the same characters mean different things in different places (e.g. }} ends a subtemplate but is plain text at the top level).

Modes (entered/exited via pushMode/popMode/mode):

Mode Entered by Purpose
(default) — Top‑level text + directive starts (@%, @<<, @, raw). A top‑level {{ here is a plain unconnected token — it does not push SUB_BLOCK.
SUB_BLOCK {{ after a call (CALL_RETURNED), inside a definition (DEF), or after an extension name (OUT_MODE) Subtemplate body; }} pops.
DEF @% Definition block: <name>, : base, :: type, -> default, ( opens a prop list, %@ pops.
DEF_PROPS ( in DEF Definition prop list (v2 typed props): prop names/types between ( … ).
IMPORT_MODE @<< Import path between {{ }}.
OUT_MODE @ Extension name; ( opens parameter.
CALL ( Parameter tokens (ids, ., ::, :), nested (; inner @ → CS.
CALL_RETURNED ) What follows a call: : chain, @ next, {{ body, raw, etc.
CS inner @ in a parameter Embedded C# expression; balances (/), ends at matching ).
CS_NESTED nested ( in CS Nested parentheses inside an embedded C# expression.
INTERP_STR / INTERP_VERBATIM_STR / INTERP_HOLE $"…" / $@"…" inside CS C# interpolated‑string literals and their { … } holes.

The mode transitions (solid = push, dashed = pop/return to a mode):

stateDiagram-v2
    state "DEFAULT" as Top
    [*] --> Top
    Top --> DEF: "@%"
    Top --> IMPORT_MODE: "@&lt;&lt;"
    Top --> OUT_MODE: "@"

    DEF --> SUB_BLOCK: "{{"
    DEF --> OUT_MODE: "-&gt; (default output)"
    DEF --> Top: "%@"

    OUT_MODE --> SUB_BLOCK: "{{ (after ext name)"

    SUB_BLOCK --> Top: "}}"
    IMPORT_MODE --> Top: "}}"

    OUT_MODE --> CALL: "("
    CALL --> CS: "inner @"
    CALL --> CALL_RETURNED: ")"
    CS --> CALL_RETURNED: "matching )"

    CALL_RETURNED --> OUT_MODE: ": (chain) or @"
    CALL_RETURNED --> SUB_BLOCK: "{{ (call body)"
    CALL_RETURNED --> Top: "text / raw"
Loading

Comments (@* … *@), trimmed whitespace (@\), and definition whitespace are routed to the hidden channel so they never reach the parser but are still available for tooling (the listener collects hidden‑channel positions as SkippedTokens). This is why comments can appear mid‑construct — between any two tokens (e.g. inside a call) — though never inside a single token. See the Language Reference → lexer modes for the author‑facing view.

2. Parsing

The parser is generated from HeddleParser.g4. The top‑level rule is heddle; the interesting rules are definition, outblock, chain, call, member_expression, csharp_expression, and subtemplate.

DocumentParser controls prediction strategy:

  • It first parses in SLL mode (fast). If SLL throws ParseCanceledException (ambiguity) it falls back to LL_EXACT_AMBIG_DETECTION (full LL) via ParseDiagnosticMode and records a HeddleCompileWarning noting the SLL failure. If SLL instead merely reported errors, it re‑parses the same way in full LL mode, but records no warning on that path.
  • When TemplateOptions.ProvideLanguageFeatures is set (editor/tooling mode), it parses directly in LL mode and produces a token list for syntax highlighting instead of optimizing for throughput.

Syntax errors are gathered by HeddleSyntaxErrorListener into the ParseContext, then copied onto CompileContext.CompileErrors.

3. Tree walking

A ParseTreeWalker drives HeddleMainListener, which builds the ParseContext: the set of definitions (DefinitionBlock/DefinitionItem), output chains (OutputChain/OutputItem), imports, and the raw/text spans. This is the structured representation the compiler consumes.

4–5. Compilation

HeddleCompiler turns the parse context into an execution‑ready document (RuntimeDocument): a tree of extension instances, each wired to a compiled value accessor. The template itself is not transpiled to C# or to IL — its structure becomes an object graph; only the value accessors are compiled. For statically‑typed models nothing is interpreted or reflected at render time; dynamic (dynamic/ExpandoObject) models instead bind their member paths through cached DLR call sites (see below).

  • It instantiates the right extension for each call (resolved by name from the registered set), and calls InitStart to thread types through the chain and compile nested subtemplates; extensions needing a second pass use CompleteInit (e.g. PartialExtension).
  • Member paths (@(A.B.C)) compile to expression‑tree delegates: ModelParameter builds an Expression<Func<object,object>> over the resolved property chain (null‑safe for reference types) and .Compile()s it. Reflection is used only here, at compile time, to locate the properties — never at render time for statically‑typed models. (Dynamic models compile instead to DLR call sites that resolve members at render time, cached per receiver type.)
  • Embedded C# expressions (@( … )) are emitted into C# source via the .tcs templates in src/Heddle/LanguageTemplates (CSharpPreparseTemplate.tcs, CSharpClassTemplate.tcs) and compiled by Roslyn (Microsoft.CodeAnalysis.CSharp) into (model, chained, root) delegates (CompiledParameter). This is also where C# expressions are bound to the model's types; the CompileScope/CSharpContext track imported namespaces (@using) and the model type (@model, :: Type). CompileScope.Compile() runs that Roslyn pass.
  • Errors from either path are collected as HeddleCompileErrors rather than thrown, and surfaced through HeddleCompileResult.

The result is a RuntimeDocument exposing an IProcessStrategy (Strategy) — the execution‑ready render tree. So "compiling a template" means two kinds of code generation (expression‑tree delegates for member access, Roslyn delegates for embedded C#) wired into one document — not a single whole‑template Roslyn compile.

6. Rendering

HeddleTemplate.Generate creates a ScopeRenderer and a root Scope, then invokes _processStrategy.Render(scope). Extensions write through scope.Renderer.Render(...) (the streaming path, RenderData) or return strings (ProcessData) when a parent needs the value (e.g. inside a chain). The renderer's buffer is size‑adaptive across calls (a per‑instance high‑water mark with ~10 % margin; the field is length‑based on net8+ and count‑based on older targets).

Scope is a small readonly struct carrying ModelData, ChainedData, ParentModelData, CallerData, RootData, and the Renderer; its pure transforms (Model, Parent, Chain, …) construct the child scopes extensions hand to their subtemplates. See Writing Custom Extensions for the author‑facing contract.

Performance characteristics

The repository's BenchmarkDotNet suite measures Heddle against four other .NET template engines (Fluid, Scriban, DotLiquid, Handlebars.Net) — all four rendering byte‑identical parity‑checked output over a component‑heavy composition workload — plus ASP.NET Core Razor, which renders a larger, different page and is not under the parity assertion ([MemoryDiagnoser] enabled). In the run of 2026‑07‑11 (commit 8341bb67; AMD Ryzen 9 9950X, .NET 10.0.9, BenchmarkDotNet 0.15.8) Heddle rendered that page in 32.50 μs / 227.86 KB — the fastest of the six and tied‑least on allocation (within 0.3 KB of Handlebars.Net); the next engine (Fluid) took 2.0× as long and Scriban 11.7× with 5.07× the allocation. The full render and compile‑cost tables, environment header, and raw artifacts live in the README Performance section and docs/benchmarks/2026-07-11. The reasons Heddle leads on the render path are structural, not incidental:

  • Execution‑ready document, not per‑call activation. Each template becomes a RuntimeDocument / IProcessStrategy with extension instances already resolved and typed, and every value accessor pre‑compiled (expression‑tree delegates for member paths, Roslyn delegates for embedded C#) — no reflection or re‑parsing per render. Rendering a dozen components is a dozen direct RenderData calls — it does not spin up partials, view components, section buffers, or DI scopes per component the way Razor does. The benchmark page exercises exactly this: many @area_component(...), @assets_component(...), @head_scripts(), etc. calls.
  • Streaming, low‑allocation output. Extensions write directly to a single ScopeRenderer whose buffer is size‑adaptive across renders (a per‑instance high‑water mark), so a warmed‑up template stops reallocating its output buffer. list pre‑sizes from ICollection<T>.Count when available.
  • Struct Scope, aggressive inlining. The per‑node data view is a readonly struct with [MethodImpl(AggressiveInlining)] transforms, avoiding per‑scope heap allocation as the renderer descends into elements and subtemplates.
  • Composition is near‑free at run time. Splitting a page into independent reusable templates recombined by a layout (see the benchmark's @<<{{layout.heddle}} import + <body:body> override) renders through one pre‑built extension node per definition invocation — no per‑render lookup, activation, or buffer indirection — unlike Razor sections, whose layout/section binding adds indirection. See Language Reference → inheritance.

The trade‑off is up‑front compilation: the first compile runs ANTLR (parse), expression‑tree compilation (member accessors), and Roslyn (embedded C#), so it is not cheap — the model is "compile once, render many." In the same 2026‑07‑11 run, cold‑compiling the layout + home templates took 264.99 μs / 1,339.67 KB for Heddle versus single‑digit microseconds for the Liquid engines (Fluid 3.65 μs, Scriban 4.68 μs, DotLiquid 7.21 μs) — a cost amortized across every cached render. Compile cost is benchmarked via TemplateParseBenchmarks and the runners in src/Heddle.Performance/Runners; full table in the README.

Source layout (src/)

Path Contents
src/Heddle Core engine.
Heddle/Core Extension base classes, InitContext.
Heddle/Extensions The 24 built‑in extensions.
Heddle/Language Parser host: DocumentParser, HeddleMainListener, ParseContext, OutputItem, DefinitionItem, CallParameter.
Heddle/Runtime HeddleCompiler, RuntimeDocument, CompileContext, CompileScope, IExtension, TemplateResolver.
Heddle/Data Scope, TemplateOptions, HeddleCompileResult, ExType, LinearList, render types.
Heddle/Attributes The extension/model attributes.
Heddle/Strings Fast string building (ExStringBuilder).
Heddle/LanguageTemplates .tcs resources used to emit C# for Roslyn.
src/Heddle.Language ANTLR grammar + generated lexer/parser + editor assets.
src/Heddle.Tests xUnit tests + .heddle fixtures.
src/Heddle.Performance BenchmarkDotNet benchmarks.

The grammar and generated code

The grammar lives in src/Heddle.Language:

Generated C# (checked in under generated/) is produced by ANTLR 4.13.1. To regenerate after editing the .g4 files, run generate_cs.cmd (requires Java; the script downloads the ANTLR 4.13.1 jar from antlr.org on first run):

java -jar "antlr-4.13.1-complete.jar" -Dlanguage=CSharp "HeddleLexer.g4" "HeddleParser.g4" -o "generated" -lib "generated" -package Heddle.Language

The project references Antlr4.Runtime.Standard 4.13.1 at run time. The js/ and ace_build/ directories are excluded from the C# compile (Heddle.Language.csproj).

Editor / tooling integrations

  • JavaScript parser — generate_js.cmd produces a JS lexer/parser under js/ from the same grammar (for in‑browser editing).
  • Ace editor — ace_build/ and build_ace.sh build an Ace mode using the JS parser for syntax highlighting on the web.
  • Language features (tooling) — setting TemplateOptions.ProvideLanguageFeatures makes DocumentParser emit a token list — the basis for editor classification (highlighting), error tagging, and completion / quick info for .heddle files.
  • TextMate grammar — coloring-scheme/heddle.tmLanguage.json is a portable highlighter (HTML + inline C# embedded) whose scopes mirror the Ace mode, for VS Code/Monaco/Shiki and documentation sites. See Syntax Highlighting.

For build/test/packaging mechanics, continue to Building & Testing.