Skip to content

add pluggable Interpolator and ConnectionCache extension points#2

Open
odemkovych wants to merge 3 commits into
mainfrom
feat/sqlds-extension
Open

add pluggable Interpolator and ConnectionCache extension points#2
odemkovych wants to merge 3 commits into
mainfrom
feat/sqlds-extension

Conversation

@odemkovych

Copy link
Copy Markdown

Summary

Adds two extension points to SQLDatasource so downstream plugins can replace internal behavior without forking sqlds:

  • SQLDatasource.Interpolator — owns the full SQL-rewrite call. Plugins use this to install AST-aware interpolation or carry plugin-defined macro context on their own wrapper type.
  • SQLDatasource.ConnectionCacheFactory — returns a ConnectionCache used by Connector for the per-ConnectionArgs *sql.DB pool. Plugins use this to install TTL/LRU caches and capture per-cache config (size cap, dependencies) by closure.

Both are additive and fully backwards-compatible: nil values resolve to defaults that preserve the pre-extension behavior byte-for-byte. Existing call sites compile and run unchanged.

Motivation

We maintain a downstream Grafana data-source plugin built on sqlds. Two needs have repeatedly pushed us toward forking:

  1. Custom interpolation — we need an AST-aware rewrite pipeline that carries plugin-defined context across macros. The current Interpolate(driver, q) free function offers no extension surface beyond the legacy sqlutil.MacroFunc map.
  2. Bounded connection cache — long-running instances accumulate per-ConnectionArgs *sql.DB entries in the internal sync.Map forever. A TTL/LRU policy needs to replace storage, not wrap it.

Rather than carry a fork, these patches expose minimal interfaces so plugins (ours and others') can swap in custom implementations.

API surface

Interpolator

type Interpolator interface {
    Interpolate(ctx context.Context, ds *SQLDatasource, query *sqlutil.Query, rawJSON json.RawMessage) (string, error)
}

type DefaultInterpolator struct{} // delegates to sqlutil.Interpolate — pre-extension default

SQLDatasource.Interpolator is the assignment point; nilDefaultInterpolator{}.

ConnectionCache

type CachedConnection interface {
    DB() *sql.DB
    Settings() backend.DataSourceInstanceSettings
    Close() error
}

type ConnectionCache interface {
    Load(key string) (CachedConnection, bool)
    Store(key string, v CachedConnection)
    Range(f func(key string, v CachedConnection) bool)
    Dispose()
}

func NewSyncMapCache() ConnectionCache // default, behaviorally identical to the old sync.Map

SQLDatasource.ConnectionCacheFactory func() ConnectionCache — invoked once per Connector; nilNewSyncMapCache(). A ConnectorOption / WithCache(...) is exposed for direct NewConnector callers; NewConnector gains a trailing variadic opts ...ConnectorOption parameter (additive).

The unexported dbConnection struct satisfies CachedConnection via three adapter methods; its internal field-access patterns elsewhere in the package are unchanged.

Contract: Load must return the exact value previously passed to Store — no wrapping or decoration. Internal code type-asserts back to the concrete struct; a wrapping impl panics loudly at the call site (documented).

Compatibility

  • No exported symbols removed or renamed.
  • NewConnector's new param is variadic — existing call sites compile unchanged.
  • With nil Interpolator and nil ConnectionCacheFactory, behavior is byte-for-byte identical to the pre-patch path (the default interpolator delegates to sqlutil.Interpolate; NewSyncMapCache runs no goroutines and performs no eviction).
  • Nil-cache guards in Connector's internal accessors keep external &Connector{} test fixtures usable.

Tests

  • interpolator_test.go — default impl parity, nil-ds handling, custom impl wiring through handleQuery.
  • cache_test.go — default impl contract (Load/Store/Range/Dispose), overwrite semantics, early-stop Range, concurrent Store/Load under -race, compile-time assert that dbConnection satisfies CachedConnection.
  • connector_cache_test.go — default cache when no option, WithCache routes the bootstrap entry into the custom cache, ConnectionCacheFactory invoked exactly once per Connector, nil factory falls back to the default, existing call sites unaffected.

All existing tests pass unchanged.

Files

README.md                  | +40    (extension-point docs)
interpolator.go            | +44    (new — Interpolator iface + default)
interpolator_test.go       | +92    (new)
cache.go                   | +100   (new — ConnectionCache iface + default)
cache_test.go              | +145   (new)
connector.go               | ~50    (cache field + WithCache option)
connector_cache_test.go    | +116   (new)
datasource.go              | ~39    (Interpolator/ConnectionCacheFactory fields)
datasource_connect_test.go | ~14    (fixture updates for cache field)

Test plan

  • go test ./... -race (covered by CI)
  • Confirm NewConnector callers in dependent plugins still compile (variadic opts are additive)
  • Confirm default-path datasources (no Interpolator, no ConnectionCacheFactory set) behave identically to main

odemkovych and others added 3 commits June 15, 2026 14:29
Add the Interpolator interface and SQLDatasource.Interpolator field. A custom Interpolator owns the full
rewrite call, so plugin-attached state and plugin-defined macro contexts
live on the plugin's own wrapper type rather than on sqlds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a ConnectionCache extension point so plugins can install a custom
cache (TTL, LRU, …) for the per-ConnectionArgs *sql.DB pool managed by
Connector. The default (NewSyncMapCache) preserves the pre-extension
behaviour byte-for-byte.

Surface:
- CachedConnection interface (DB, Settings, Close) — exposed by three
  adapter methods on the existing unexported dbConnection struct; the
  struct's fields and internal access patterns are unchanged
- ConnectionCache interface (Load, Store, Range, Dispose) — non-generic;
  contract documents the no-wrap requirement
- NewSyncMapCache() — default sync.Map-backed impl, no background work
- ConnectorOption + WithCache(ConnectionCache); NewConnector gains a
  trailing variadic opts parameter (additive, existing call sites
  compile unchanged)
- SQLDatasource.ConnectionCacheFactory func() ConnectionCache — nil
  resolves to the default; the factory is invoked once per Connector

Connector.connections sync.Map is replaced by an unexported cache
ConnectionCache field. Internal accessors delegate to the cache and
type-assert Load's result back to the concrete dbConnection (safe under
the no-wrap contract; fails loudly otherwise). Nil-cache guards in the
internal accessors keep external &Connector{} test fixtures usable.

Tests cover the default impl contract, concurrent Store/Load with
-race, Dispose-closes-every-entry, factory wiring, and bootstrap entry
placement via WithCache. Adds the connection-cache spec and tasks under
openspec/changes/add-connection-cache.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@odemkovych odemkovych self-assigned this Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant