Skip to content

Query API

Jump to bottom
David Sisco edited this page Jan 11, 2026 · 4 revisions

Query API

The Query static class provides a fluent CSS-like selector API for finding nodes in a syntax tree.

Basic Usage

using TinyTokenizer.Ast;

var tree = SyntaxTree.Parse("foo(x) + bar(y)");

// Select nodes matching a query
var idents = tree.Select(Query.AnyIdent).ToList();  // [Ident("foo"), Ident("x"), Ident("bar"), Ident("y")]

// Use in editor
tree.CreateEditor()
    .Replace(Query.Ident("foo"), "baz")  // Matches: Ident("foo")
    .Commit();

Named Queries

Match specific token content:

Query.Ident("main")            // Identifier with text "main"
Query.Symbol(".")              // Dot symbol
Query.Operator("=>")           // Arrow operator
Query.Numeric("42")            // Number literal "42"
Query.String("\"hello\"")      // String literal
Query.TaggedIdent("#define")   // Tagged identifier

Any-Kind Queries

Match any token of a kind:

Query.AnyIdent                 // All identifiers
Query.AnyNumeric               // All numbers
Query.AnyString                // All strings
Query.AnyOperator              // All operators
Query.AnySymbol                // All symbols
Query.AnyTaggedIdent           // All tagged identifiers
Query.AnyComment               // All comments
Query.Any                      // Any single node

Block Queries

Match block delimiters:

Query.BraceBlock               // All { } blocks
Query.BracketBlock             // All [ ] blocks
Query.ParenBlock               // All ( ) blocks
Query.AnyBlock                 // Any block type

Keyword Queries

Match language keywords defined in the schema:

// All keywords (requires schema with keyword definitions)
Query.AnyKeyword               // All keyword tokens

// Specific keyword text
Query.Keyword("if")            // Matches "if" keyword
Query.Keyword("return")        // Matches "return" keyword

// Keywords in a category
Query.KeywordCategory("Types")        // int, float, double, void...
Query.KeywordCategory("ControlFlow")  // if, else, while, for, return...
Query.KeywordCategory("Modifiers")    // public, private, static, const...

Keywords are defined via Schema.Create().DefineKeywords() — see Schema#Keyword Definitions.

Boundary Assertions

Match document boundaries:

Query.BOF                      // Beginning of file (first token at root)
Query.EOF                      // End of file (last token at root)

Newline Assertions

Line-based matching is done via trivia, not via dedicated newline tokens.

Query.Newline                   // Node occurs after a newline
Query.NotNewline                // Exact negation of Query.Newline

Semantics:

  • A node matches Query.Newline when either:
    • The node owns leading newline trivia, OR
    • The previous sibling owns trailing newline trivia.

This is useful with repetition:

// Consume tokens on the current line (terminator is not consumed)
Query.Any.Until(Query.Newline)

Filters

Refine queries with predicates:

// Text-based filters
Query.AnyIdent.WithText("foo")              // Exact match
Query.AnyIdent.WithTextContaining("test")   // Contains substring
Query.AnyIdent.WithTextStartingWith("_")    // Starts with prefix
Query.AnyIdent.WithTextEndingWith("Async")  // Ends with suffix

// Custom predicate
Query.AnyIdent.Where(n => n.Width > 5)      // Width > 5
Query.AnyNumeric.Where(n => int.Parse(n.Text) > 100)

Pseudo-Selectors

Limit matches:

Query.AnyIdent.First()         // First match only
Query.AnyIdent.Last()          // Last match only
Query.AnyIdent.Nth(2)          // Third match (0-indexed)
Query.AnyIdent.Skip(1)         // Skip first match
Query.AnyIdent.Take(3)         // Take first 3 matches

Composition

Combine queries:

// Union (OR) — matches if either matches
Query.AnyIdent | Query.AnyNumeric

// Intersection (AND) — matches if both match
Query.AnyIdent & Query.Leaf

// Variadic OR
Query.AnyOf(Query.AnyIdent, Query.AnyNumeric, Query.AnyString)

// Match when none match
Query.NoneOf(Query.Ident("if"), Query.Ident("else"), Query.Ident("while"))

Negation

Zero-width negative assertion:

// Negative lookahead
Query.Not(Query.Ident("if"))         // Not the identifier "if"
Query.AnyIdent.Not()                 // Fluent syntax

Sequences

Match consecutive nodes:

// Static method
Query.Sequence(Query.AnyIdent, Query.ParenBlock)  // ident then parens

// Fluent chaining
Query.AnyIdent.Then(Query.ParenBlock)

// Multiple elements
Query.Sequence(
    Query.AnyIdent,
    Query.Symbol("."),
    Query.AnyIdent,
    Query.ParenBlock
)  // Matches: obj.method()

Repetition

Match repeated patterns:

Query.AnyIdent.Optional()        // Match 0 or 1
Query.AnyIdent.ZeroOrMore()      // Match 0 or more
Query.AnyIdent.OneOrMore()       // Match 1 or more
Query.AnyIdent.Exactly(3)        // Match exactly 3
Query.AnyIdent.Repeat(2, 5)      // Match 2 to 5

// Repeat until terminator (terminator not consumed)
Query.Any.Until(Query.Newline)

Lookahead

Zero-width assertions about following nodes:

// Positive lookahead — must be followed by
Query.AnyIdent.FollowedBy(Query.ParenBlock)     // func(...)

// Negative lookahead — must not be followed by
Query.AnyIdent.NotFollowedBy(Query.ParenBlock)  // Not a function call

Navigation Queries

Queries based on tree structure:

// Sibling checks
Query.Sibling(1)                       // Next sibling exists
Query.Sibling(-1)                      // Previous sibling exists
Query.Sibling(1, Query.AnyIdent)       // Next sibling is identifier

// Fluent sibling checks
Query.AnyIdent.NextSibling()           // Has next sibling
Query.AnyIdent.PreviousSibling()       // Has previous sibling
Query.AnyIdent.NextSibling(Query.ParenBlock)  // Next sibling is paren block

// Parent/ancestor checks
Query.Parent(Query.BraceBlock)         // Parent is a brace block
Query.Ancestor(Query.BraceBlock)       // Any ancestor is a brace block

// Match by parent/ancestor
Query.BraceBlock.AsParent()            // Match nodes whose parent is brace block
Query.BraceBlock.AsAncestor()          // Match nodes with brace block ancestor

Content Matching

Match content between delimiters:

// Match everything between < and >
Query.Between(Query.Operator("<"), Query.Operator(">"))

Exact Node Reference

Match a specific node instance:

var tree = SyntaxTree.Parse("foo bar baz");
var node = tree.Select(Query.Ident("foo")).First();  // Ident("foo")

// Query that only matches this exact node
Query.Exact(node)

// Useful in editor when you have a node reference
tree.CreateEditor()
    .Replace(Query.Exact(node), "qux")  // Replaces only the matched "foo"
    .Commit();
// Result: "qux bar baz"

Block Boundary Queries

Block queries can select their boundaries for insertion:

// Get block boundary positions
Query.BraceBlock.First().Start()   // Opening delimiter {
Query.BraceBlock.First().End()     // Closing delimiter }

// Use with InsertBefore/InsertAfter
tree.CreateEditor()
    .InsertAfter(Query.BraceBlock.First().Start(), "// first line")   // After {
    .InsertBefore(Query.BraceBlock.First().End(), "// last line")     // Before }
    .Commit();

Boundary positions explained

Position Code Result
Inside start InsertAfter(block.Start(), x) { x ...}
Inside end InsertBefore(block.End(), x) {... x }
Before block InsertBefore(block, x) x { }
After block InsertAfter(block, x) { } x

Named Node Queries

For syntax nodes implementing INamedNode:

// Find functions by name
Query.Syntax<FunctionSyntax>().Named("main")

// Use with InsertBefore/InsertAfter
tree.CreateEditor()
    .InsertBefore(Query.Syntax<FunctionSyntax>().Named("foo"), "// doc\n")
    .Commit();

Block Container Queries

For syntax nodes implementing IBlockContainerNode:

// Insert into named blocks using convenience methods
var funcQuery = Query.Syntax<FunctionSyntax>().Named("main");

tree.CreateEditor()
    .InsertAfter(funcQuery.InnerStart("body"), "\n    console.log('enter');")
    .InsertBefore(funcQuery.InnerEnd("body"), "\n    console.log('exit');")
    .Commit();

// Or use the explicit Block().Start()/End() form
tree.CreateEditor()
    .InsertAfter(funcQuery.Block("body").Start(), "\n    console.log('enter');")
    .InsertBefore(funcQuery.Block("body").End(), "\n    console.log('exit');")
    .Commit();

Quick Reference Table

Combinator Description Example
Query.Ident("x") Specific identifier Query.Ident("main")
Query.Symbol(".") Specific symbol Query.Symbol(".")
Query.Operator("=>") Specific operator Query.Operator("=>")
Query.Numeric("42") Specific number Query.Numeric("3.14")
Query.Keyword("if") Specific keyword Query.Keyword("return")
Query.AnyIdent Any identifier Query.AnyIdent
Query.AnySymbol Any symbol Query.AnySymbol
Query.AnyOperator Any operator Query.AnyOperator
Query.AnyNumeric Any number Query.AnyNumeric
Query.AnyString Any string Query.AnyString
Query.AnyKeyword Any keyword Query.AnyKeyword
Query.KeywordCategory("X") Keywords in category Query.KeywordCategory("Types")
Query.ParenBlock ( ) block Query.ParenBlock
Query.BraceBlock { } block Query.BraceBlock
Query.BracketBlock [ ] block Query.BracketBlock
Query.Any Any single node Query.Any
Query.Sequence(...) Match A then B Query.Sequence(a, b)
a | b Match A or B Query.AnyIdent | Query.AnyNumeric
.Optional() Match 0 or 1 Query.AnyOperator.Optional()
.ZeroOrMore() Match 0+ Query.AnyIdent.ZeroOrMore()
.OneOrMore() Match 1+ Query.AnyIdent.OneOrMore()
.Exactly(n) Match exactly n Query.AnyIdent.Exactly(3)
.Repeat(min, max) Match range Query.AnyIdent.Repeat(2, 5)
.Until(q) Repeat until Query.Any.Until(Query.Newline)
.FollowedBy(q) Positive lookahead Query.AnyIdent.FollowedBy(Query.ParenBlock)
.NotFollowedBy(q) Negative lookahead Query.AnyIdent.NotFollowedBy(Query.ParenBlock)
.Then(q) Fluent sequence Query.AnyIdent.Then(Query.ParenBlock)
Query.Exact(node) Exact node Query.Exact(myNode)
.First() First match Query.AnyIdent.First()
.Last() Last match Query.AnyIdent.Last()
.Nth(n) Nth match Query.AnyIdent.Nth(2)
.Where(pred) Filter Query.AnyIdent.Where(n => ...)
.Start() Block opener boundary Query.BraceBlock.Start()
.End() Block closer boundary Query.BraceBlock.End()
.InnerStart(name) Named block start funcQuery.InnerStart("body")
.InnerEnd(name) Named block end funcQuery.InnerEnd("body")

See Also

Overview

TinyAst

Low-Level Tokenization

Reference

Clone this wiki locally