Custom Codec Registry: Interface-Based Design

[SeanTAllen]

Custom Codec Registry: Interface-Based Design

Discussion #146 explored three options for making the codec registry user-extensible. After analysis, none of them are the right answer. Here's the approach we're going with.

The Core Problem (quick recap)

FieldDataTypes is a closed union. Custom codecs can only return types already in the union, which makes them nearly useless. The best you can do is decode an unknown OID to String instead of Array[U8] val. That's Option B from #146, and it's a mediocre API.

Option A (wrapping custom types in CustomFieldData with Any val inside) gives users two-level matching: first match on CustomFieldData, then cast the inner value. That's ugly and defeats the purpose of Pony's type system. It also has a concrete correctness problem: _FrontendMessage.bind()'s size-calculation pass would silently skip CustomFieldData values, producing corrupted wire messages.

The Actual Answer: Separate Result Types from Parameter Types

Results and parameters have different semantics. Parameters need a closed set of types because the encode path (_FrontendMessage.bind(), _ParamEncoder) matches on Pony types to derive OIDs and produce wire bytes. Results don't have that constraint. The server tells us the OID and format code, the codec decodes the bytes, and the user gets a typed value. There's no reason the decode side needs to be locked to the same closed union as the encode side.

So we use different types for each direction:

• Parameters stay as FieldDataTypes (the existing closed union). The encode path is completely unchanged.
• Results use a new FieldData interface (an open type that any val with string() can satisfy).

interface val FieldData
  fun string(): String iso^

All 13 existing types already have string() via Stringable, so they conform structurally. Field.value becomes FieldData instead of FieldDataTypes. Codec.decode() returns FieldData. Custom codecs return their own types directly.

This is a breaking change for any code that explicitly references FieldDataTypes as the type of a result field value. Code that just matches on field.value (which is all the examples and every consumer I've seen) continues to work unchanged.

What This Looks Like for Users

A custom codec for PostgreSQL point (OID 600):

class val Point is FieldData
  let x: F64
  let y: F64

  new val create(x': F64, y': F64) =>
    x = x'
    y = y'

  fun string(): String iso^ =>
    recover
      String.join(["("; x.string(); ","; y.string(); ")"].values())
    end

primitive PointBinaryCodec
  fun format(): U16 => 1

  fun decode(data: Array[U8] val): FieldData ? =>
    let x = F64.from_bits(data.read_u64(0)?.bswap())
    let y = F64.from_bits(data.read_u64(8)?.bswap())
    Point(x, y)

  fun encode(value: FieldDataTypes): Array[U8] val ? =>
    error // decode-only for now

Registration and use:

let registry = CodecRegistry
  .with_codec(600, PointBinaryCodec)

Session(server_info, db_info, notify where registry = registry)

Users match on their concrete types directly:

match field.value
| let p: Point => use(p.x, p.y)
| let s: String => // ...
| let i: I32 => // ...
end

No wrappers. No casting. Flat matching, same pattern as built-in types.

Field.eq()

Field.eq() currently has explicit arms for all 13 built-in types, with else false. Those arms stay and work exactly as they do today.

For custom types, we add an opt-in equality interface:

interface val FieldDataEquatable
  fun field_data_eq(that: FieldData box): Bool

Custom types that want equality comparison implement FieldDataEquatable. The else branch in Field.eq() checks for this interface before falling to false. Built-in types don't need to change. They're handled by the existing explicit arms.

CodecRegistry API

_with_codec becomes a public method (not a constructor) for fluent chaining:

let registry = CodecRegistry
  .with_codec(600, PointBinaryCodec)
  .with_codec(790, MoneyTextCodec)

Session gets an optional CodecRegistry parameter with a default:

Session(server_info, db_info, notify where registry = my_registry)

The existing fallback behavior is unchanged: unknown text-format OIDs decode to String, unknown binary-format OIDs to Array[U8] val.

What's Deferred

Custom parameter encoding. The encode path (_FrontendMessage.bind(), _ParamEncoder) matches on Pony types from the FieldDataTypes union to derive OIDs and produce wire bytes. Making that extensible would require either refactoring bind() to delegate to the codec registry, or introducing a new parameter type that carries an OID alongside the value. Both are significant changes. Users who need to send values for custom PostgreSQL types can use String with text format today. Custom parameter encoding gets its own design work when there's a concrete use case driving it.

Codec interface split. The Codec interface currently has both encode() and decode(). Decode-only custom codecs would implement encode() as error. Splitting into separate Decoder/Encoder interfaces would be cleaner. It's a minor wart for now.

Phase 3 Scope

1. New FieldData interface
2. New FieldDataEquatable interface for opt-in custom type equality
3. Codec.decode() return type changes from FieldDataTypes to FieldData
4. CodecRegistry.decode() return type changes to FieldData
5. Field.value type changes to FieldData
6. Field.eq() updated with FieldDataEquatable check in else branch
7. _with_codec renamed to with_codec as a public method (not constructor) for fluent chaining
8. Session constructor gets optional CodecRegistry parameter, threaded through the state machine to _SessionLoggedIn
9. Package docstring updated with custom codec documentation
10. New examples/custom-codec/ example + examples/README.md update
11. Tests, CLAUDE.md update, release notes (classification: changed)

Design origin: discussion #146

[SeanTAllen]

Imeplementation plan: #148