🚀 Kvatch Runtime

A lightweight runtime for executing federated data plans.

⚠️ v1 is in preparation. Until v1.0.0, minor API adjustments may still occur.

---

📦 Installation

go get github.com/kvatch-hub/kvatch-runtime

---

🚀 Quick Start

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/kvatch-hub/kvatch-runtime/model"
    "github.com/kvatch-hub/kvatch-runtime/pkg/runtime"
)

func main() {
    rt, err := runtime.NewDefault()
    if err != nil {
        log.Fatal(err)
    }

    plan := model.Plan{
        Name: "postgres-example",
        Output: model.Output{
            DatasetName: "author_books",
            Verbose:     true,
        },
        Connectors: []model.Connector{
            {
                Name: "books_db",
                Type: model.ConnectorTypePostgres,
                Connection: &model.PostgresConnectorConfig{
                    Host:     "localhost",
                    Port:     5432,
                    Database: "books",
                    Username: "example_user",
                    Password: "example_password",
                    SSLMode:  "disable",
                },
            },
            {
                Name: "authors_db",
                Type: model.ConnectorTypePostgres,
                Connection: &model.PostgresConnectorConfig{
                    Host:     "localhost",
                    Port:     5432,
                    Database: "authors",
                    Username: "example_user",
                    Password: "example_password",
                    SSLMode:  "disable",
                },
            },
        },
        Datasets: []model.Dataset{
            {
                Name:          "books",
                ConnectorName: "books_db",
                Type:          model.DatasetTypeSQL,
                Options: &model.SQLDatasetOptions{
                    Query:   "SELECT name as title, author, review FROM books",
                    Timeout: 30,
                },
            },
            {
                Name:          "authors",
                ConnectorName: "authors_db",
                Type:          model.DatasetTypeSQL,
                Options: &model.SQLDatasetOptions{
                    Query:   "SELECT id, slug, name, bio FROM authors",
                    Timeout: 30,
                    Dedupe:  []string{"id", "slug"},
                },
            },
            {
                Name:          "author_books",
                ConnectorName: "federated",
                Type:          model.DatasetTypeSQL,
                Options: &model.SQLDatasetOptions{
                    Query: `
SELECT a.name, b.title as title, b.review
FROM authors a
JOIN books b ON a.slug = b.author
`,
                    Timeout: 30,
                    Dedupe:  []string{"id", "slug"},
                },
                Children: []model.DatasetChild{
                    {DatasetName: "authors"},
                    {DatasetName: "books"},
                },
            },
        },
    }

    if err := plan.Validate(); err != nil {
        log.Fatalf("invalid plan: %v", err)
    }

    resp, err := rt.ExecutePlan(context.Background(), model.ExecutePlanRequest{
        Plan: plan,
    })
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Columns:", resp.Columns)
    fmt.Printf("Rows:\n")
    for _, row := range resp.Data {
        fmt.Printf("  %v\n", row)
    }
}

What this does

• Queries two completely separate Postgres databases
• Loads them into the runtime
• Joins them using SQL
• Returns a unified result set

No ETL. No pipelines. Just declarative data access.

See examples/postgres for a runnable version using .env.

---

▶️ Run the Postgres Example

git clone https://github.com/kvatch-hub/kvatch-runtime
cd kvatch-runtime

cp examples/postgres/.env.example .env
go run ./examples/postgres

---

📦 Features

• 🔗 Federated queries across multiple data sources
• 🧩 Pluggable connectors (API, Postgres, CSV, Google Sheets, etc.)
• ⚡ In-memory execution engine
• 🧱 Strongly typed runtime model
• 🛠 Built for extensibility

---

🧭 Public API Boundary

The packages intended for external use are:

• pkg/runtime
• model
• configs

Everything under internal/ is implementation detail and may change without notice.

---

🔌 Connector Support (current)

• Stable/primary: postgres, sqlite, api, localfile, local_directory
• Available: googlesheet, git, s3, google_api
• Special runtime-only connector: federated

For early releases, treat non-primary connectors as evolving APIs.

---

💡 Examples

• Track a crypto portfolio → examples/crypto_portfolio
• Query federated datasets using SQL → examples/postgres

---

💡 Vision

Kvatch aims to make querying distributed data as simple as writing SQL — without moving or duplicating it.

---

📄 License

MIT License © 2026 James Wooltorton