Skip to content

monospaceai/evaldata

Repository files navigation

evaldata

CI Coverage License: Apache 2.0

Evaluate AI-generated SQL with pytest.

evaldata runs text-to-SQL evals in your existing test suite.

It checks semantic equivalence of SQL queries, diffs result sets in your warehouse, and uses an LLM judge for ambiguous cases.

Why evaldata

  • Semantic equivalence. Parse both queries, normalize their ASTs, and compare canonical forms. No execution, no LLM — when it can't confirm, it returns unknown.
  • Execution in your warehouse. Run the query on DuckDB, Postgres, Databricks, or Snowflake and compare the results, accounting for row order, NULLs, float tolerance, and types.
  • It's just pytest. Every eval is a test, run in your suite and your CI on every PR. No new runner, notebook, or dashboard.
  • An LLM judge when you need one. For ambiguous questions, missing reference answers, or explanations to grade, use a grader model with explicit criteria.

Quickstart

uv add evaldata   # core, includes the DuckDB adapter

An eval is a pytest test: a case (a question and its expected answer), a solver (the system under test that writes the SQL), and a scorer (how the answer is judged).

Below, the AI's SQL is written differently from the reference query — reordered predicates, different casing — but means the same thing. observed_equivalence() confirms the match with AST normalization; no query runs.

from evaldata import CallableSolver, EvalCase, assert_eval, eval_case, observed_equivalence
from evaldata.platforms import duckdb_platform

platform = duckdb_platform(name="shop", path="shop.duckdb")


@eval_case(
    input="Name the US customers with an id above 1.",
    expected={"kind": "gold_query", "sql": "SELECT name FROM customers WHERE country = 'US' AND id > 1"},
    platform=platform,
)
def test_us_customers(case: EvalCase) -> None:
    solver = CallableSolver(lambda c: "select NAME from customers where id > 1 and country = 'US'")
    assert_eval(case, solver, scorers=[observed_equivalence()])
uv run pytest
 case               result   detail
 ──────────────────────────────────
 test_us_customers  PASS

 1 passed, 0 failed

The full runnable version is in examples/01_deterministic/test_showcase.py.

To test a real model instead of fixed SQL, swap the solver for PromptSolver(model="openai/gpt-4o-mini") (needs the evaldata[litellm] extra). To judge equivalence without a warehouse, swap the scorer for judged_equivalence(model).

More use cases

Install

uv add evaldata                # core (includes the DuckDB adapter)
uv add "evaldata[postgres]"    # + Postgres adapter
uv add "evaldata[databricks]"  # + Databricks adapter
uv add "evaldata[snowflake]"   # + Snowflake adapter
uv add "evaldata[cortex]"      # + Snowflake Cortex Analyst solver
uv add "evaldata[litellm]"     # + litellm, to call a model from PromptSolver

DuckDB, Postgres, Databricks, and Snowflake are the adapters available today. A BigQuery adapter is planned.

Documentation

Full documentation: monospaceai.github.io/evaldata

Examples

Runnable examples in examples/:

Example Shows
Showcase Semantic equivalence with an execution fallback — no setup
Deterministic Every expected-type and scorer, with fixed SQL
Local AI A self-hosted Ollama model
Hosted AI A hosted model, mocked so it runs without a key
Databricks The same cases on a live Databricks SQL Warehouse
LLM judge Judged equivalence, mocked so it runs without a key
Benchmark Load a Spider/BIRD dataset and measure execution accuracy
Snowflake The same cases on a live Snowflake warehouse — live-only, needs SNOWFLAKE_* credentials
Cortex Analyst Snowflake Cortex Analyst — live-only, needs SNOWFLAKE_* credentials
dbt A dbt project's text-to-SQL, stubbed so it runs offline
dbt Semantic Layer dbt Semantic Layer (MetricFlow) queries, scored locally on DuckDB

See examples/README.md for details.

Contributing

git clone https://github.com/monospaceai/evaldata.git
cd evaldata
uv sync                       # core + dev tooling
uv run pre-commit install
just check                    # lint + typecheck + tests with coverage (runs everything)

just check runs lint, typecheck, and tests with coverage (held at 100%). See the justfile for the full set of commands.

Platform e2e tests

Adapter conformance for real platforms is marked e2e. CI provisions Postgres as a service container and runs the suite on every push, so the Postgres adapter is exercised against a real engine on every change.

Run it locally against Postgres with:

docker compose up -d                  # postgres:17 on localhost:5432
uv run --extra postgres pytest -m e2e # connection via POSTGRES_TEST_* env (defaults match compose)