Skip to content

Migrate pokemon v2/v3 scan to Huma (OpenAPI 3.1) — POC#368

Merged
jfberry merged 47 commits into
mainfrom
feat/huma-pokemon-v2-v3-scan
Jun 18, 2026
Merged

Migrate pokemon v2/v3 scan to Huma (OpenAPI 3.1) — POC#368
jfberry merged 47 commits into
mainfrom
feat/huma-pokemon-v2-v3-scan

Conversation

@jfberry

@jfberry jfberry commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Proof-of-concept migrating POST /api/pokemon/v2/scan and POST /api/pokemon/v3/scan from hand-rolled gin handlers to Huma, so the API is self-documenting via OpenAPI 3.1. Goal: evaluate discoverability on our most complex endpoints (DNF filters + PVP) before deciding whether to migrate the rest.

  • Docs: Stoplight UI at /docs, spec at /openapi.json (both public; operations are secured).
  • Documented response types: new pointer-based PokemonResult; the previously-opaque Pvp interface{} is now a fully documented {little, great, ultra} PvpRankings structure. Request-side DNF filter structs are documented too.
  • Serializer parity: Huma uses its own serializer (not gin's), so -tags go_json doesn't apply to it — overridden to use goccy/go-json to match the current gin setup.
  • Auth: X-Golbat-Secret is now a documented apiKey security scheme + Huma middleware mirroring AuthRequired() (empty secret still disables auth).
  • Legacy retired: gin PokemonScan2/3 handlers and the dead GetPokemonInArea2/3 + PokemonScan3Result removed. internalGetPokemonInArea2/3, GrpcGetPokemonInArea2/3, and the v1/search builders are untouched.

Wire compatibility

Designed to be wire-compatible with the legacy responses (v2 has a real production consumer):

  • Nullable fields are *T without omitempty, so missing values still serialize as null (identical to guregu/null). A golden parity test asserts every shared scalar key is byte-identical to the legacy builder.
  • v2 stays a bare array; v3 keeps its {pokemon, examined, skipped, total} wrapper; both preserve HTTP 202.
  • Disabled Huma's $schema/Link: describedBy response injection (cfg.CreateHooks = nil) so v3 bodies don't gain a $schema field. Guarded by an e2e test.

Intentional divergences (documented): pvp shape changed from the legacy dynamic map (omitted empty leagues; null when disabled) to the fixed three-league struct (always emits all three, null for empty). Huma also validates request bodies more strictly (structured 422 vs gin's lenient bind) and omits the ; charset=utf-8 on Content-Type.

Test Plan

Automated (CI): builder/golden-parity, v2 bare-array & v3 wrapper shape, auth allow/reject, OpenAPI discoverability, and an httptest e2e (auth + $schema absence) all pass under go test ./... and go build -tags go_json ./....

Manual — to run against the live DB + real caller:

  • Boot against the populated MariaDB; confirm no startup panic and /docs renders both operations with the PVP schema.
  • Point the real v2 consumer at /api/pokemon/v2/scan and confirm it works unchanged (this is the key validation).
  • Diff a captured real v2 response against the pre-migration output (expect byte-identical except the documented pvp/header divergences).
  • Spot-check v3 with PVP enabled to confirm little/great/ultra populate as expected.

Design + results doc: docs/superpowers/specs/2026-05-30-huma-pokemon-v2-v3-scan-design.md

🤖 Generated with Claude Code

jfberry and others added 29 commits June 5, 2026 10:27
Proof-of-concept design to evaluate Huma's OpenAPI discoverability:
replace POST /api/pokemon/v3/scan with a Huma operation, fixed-league
PVP struct, pointer-based nullable fields (wire-compatible), and a
goccy/go-json serializer override for parity with the current gin setup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold v2 into the migration so we can validate against a real production
v2 consumer. Shared clean response types (PokemonResult, PvpEntry,
PvpRankings) with v2 keeping its bare-array envelope and v3 its wrapper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seven TDD tasks: add huma dep, shared PokemonResult/PVP types + builder,
v2/v3 clean entry functions, Huma infra (goccy serializer + secret auth),
operation registration + retire gin handlers, OpenAPI assertion test,
manual wire-compat verification against the real v2 consumer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Declared via `go get` without `go mod tidy` so it survives until the
first code import in a later task (tidy would otherwise prune an
unimported module). Promoted to a direct require when huma_api.go lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…okemonResult

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Discoverability achieved (OpenAPI test), scalar wire-compat verified
(golden parity + e2e), $schema injection regression caught and fixed.
Remaining manual step: real-server + real-v2-consumer replay needs a
populated MariaDB in the user's environment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Huma marks every struct field required by default; the legacy gin handlers
used BindJSON which required nothing. Set FieldsOptionalByDefault so the
documented request contract matches the lenient behavior callers rely on.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Revert the blanket FieldsOptionalByDefault toggle in favour of explicit
per-field `required:"false"` tags, matching how the code already models
optionality (nil-checked pointer fields). Contract: min/max (bounding box)
required; limit, filters, and all filter attributes optional; range
objects require both min and max when present. Test pins the required set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Traced the filter lookup: keys are tried as {id,form}, {id,-1}, {-1,-1}
but never {-1,form}, so a pokemon entry with a form and no id can never
match. The id is the meaningful field; form is optional (null = any form
of that id). Mark id required, keep form optional. Test updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The v2/v3 request embedded the internal geo.Location, whose JSON fields
are Latitude/Longitude. Under Huma's strict validation this 422'd the
lat/lon shape every other Golbat endpoint uses. Introduce ApiLatLon: a
SchemaProvider + custom UnmarshalJSON that accepts lat/lon (canonical) or
latitude/longitude (compat), converting to geo.Location internally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a gin-level logger (gated on logging.debug) that records the raw
request body and the response status+body for /api/pokemon/*/scan. Huma
validates and returns 422 before the operation handler runs, so this
transport-level hook is the only place that sees rejected payloads and
the schema-validation reason. Off unless logging.debug is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The legacy pvp map only included a league key when it had rankings; the
new fixed-league struct emitted all three with null for empty leagues,
which can break consumers that iterate pvp (a null league where a key was
previously absent). Add omitempty to the league fields so empty leagues
are dropped, making the output match the legacy map for ranked pokemon.
The OpenAPI schema still documents all three leagues.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trim ApiLatLon's SchemaProvider to document only the canonical lat/lon.
The latitude/longitude alias is still accepted at runtime via
UnmarshalJSON but no longer shown in the OpenAPI. The custom schema is
still required to keep additionalProperties permissive so the
undocumented alias passes validation. Test asserts the coordinate schema
exposes lat/lon and not latitude/longitude.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the new response types to keep the Api prefix (PokemonResult ->
ApiPokemonResult, PvpRankings -> ApiPvpRankings, PvpEntry -> ApiPvpEntry,
PokemonScanResultV3 -> ApiPokemonScanResultV3) and delete the duplicate
legacy ApiPokemonResult/buildApiPokemonResult that used null.X + an
interface{} pvp. v1 scan, search, and GetOnePokemon already referenced
ApiPokemonResult/buildApiPokemonResult by name, so they now use the
documented pointer-based struct with no code changes (wire-compatible:
null.X and pointers serialize identically; pvp omitempty matches the old
map). Replace the now-moot legacy-vs-new parity test with a golden
snapshot pinning the exact wire format shared by every endpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse ApiPokemonDnfMinMax8 (int8) into ApiPokemonDnfMinMax (int16) so
the public schema exposes one range type instead of leaking the internal
integer width. All filter range fields (iv/atk/def/sta/level/gender/size,
plus cp/pvp) now use the single type. The int8 PokemonLookup fields are
cast up to int16 at the comparison sites in isPokemonDnfMatch (a free
register sign-extend; the per-pokemon lookup struct stays int8 to avoid
any memory cost). gRPC convertToMinMax8/16 collapse to one convertToMinMax.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The HTTP request/response body logging for the scan endpoints can be very
large, so move it off the general logging.debug flag onto its own
api_request_logging option (default off). Lets debug stay on without the
body dumps, and the logging can be re-enabled on demand to diagnose a
specific caller. Documented in config.toml.example.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fort scan (draft-badged) + pokemon search/id + tier 3 reads + tier 4
operational endpoints. Pointer-based response structs everywhere (pokemon
template), reusing existing builders and the PR #368 Huma infrastructure.
Stacked on feat/huma-pokemon-v2-v3-scan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
13 tasks in 3 phases: shared infra (fort range unification, ApiLatLon,
draft-badge + geofence helpers) -> fort scan (draft) -> pokemon search/id
+ tier 3 -> tier 4. Pointer-based result structs with golden snapshots;
geofence endpoints via RawBody + NormaliseFenceFromBytes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jfberry and others added 14 commits June 5, 2026 10:27
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All planned endpoints (fort scan draft-badged, pokemon search/id, tier 3,
tier 4) now served by Huma; v1 scan + /raw + health/version stay on gin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register available-pokemon (GET, 202) and retire the gin handler. The
result struct (ApiPokemonAvailableResult) is plain ints, so no pointer
conversion needed — just doc tags.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The quest-status, clear-quests, and pokestop-positions bodies are JSON
(GeoJSON geometry/feature or a Golbat fence), but Huma's RawBody []byte
documented them as application/octet-stream (a binary string). Switch to
Body json.RawMessage so the OpenAPI shows application/json. json.RawMessage
is assignable to []byte, so NormaliseFenceFromBytes is called unchanged;
malformed JSON still returns 400 (Huma body-parse error).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jfberry jfberry force-pushed the feat/huma-pokemon-v2-v3-scan branch from 06555dd to 40f98d6 Compare June 5, 2026 09:41
jfberry and others added 4 commits June 5, 2026 10:45
When logging.api_request_logging is on, log the request/response for every
Huma-served /api operation (renamed humaScanRequestLogger ->
humaApiRequestLogger, isHumaScanPath -> isHumaApiPath matching /api/). The
middleware sits ahead of only the Huma routes, so it naturally excludes the
remaining gin routes and the /docs + /openapi.json doc routes. Log prefix
is now "[huma api]".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…docs

- Add required:"false" to request fields the legacy gin BindJSON treated as
  optional: DNF range bounds (pokemon + fort), gym search limit and filter
  fields, pokemon search min/max (center-only mode), and the ids query body.
  Without these, Huma's required-by-default schemas 422-rejected real legacy
  client bodies (e.g. {"iv":{"min":90}}).
- Correct fort DNF list-filter docs: omitted/null means no constraint; an
  explicitly empty list matches nothing. The old wording documented the
  opposite of the matcher's nil-check behavior.
- Correct the gym-search clamp comment: the legacy clamp was a no-op
  (range-loop copy); the effective 500km clamp is a deliberate change.
- New api_docs config option (default on) controls whether /docs,
  /openapi.json and /schemas are registered. They are intentionally served
  without the api secret.
- Regression tests: schema required-field pins, lenient legacy bodies over
  the HTTP pipeline, and the api_docs gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… copy

ApiPvpEntry is field-identical to gohbem.PokemonEntry (only the doc tags
differ), and Go struct conversion ignores tags, so ApiPvpEntry(e) replaces
the manual 10-field mapping. Same wire output; a future gohbem field change
now fails to compile instead of silently dropping the field from the API.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Set http.Server.IdleTimeout to 60s. Without it net/http holds idle
  keep-alive connections open until the client closes, each pinning a
  goroutine and a file descriptor — the cause of the stepped goroutine/FD
  plateau under many persistent clients.
- Rework the auth-gated /debug/pprof routes via a small helper, mirroring
  net/http/pprof's own registration: add the missing index, /allocs,
  /threadcreate and symbol POST, and route the named profiles through
  pprof.Handler instead of pprof.Index. Kept hand-rolled (not gin-contrib)
  so they stay behind AuthRequired().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jfberry jfberry merged commit c9371a5 into main Jun 18, 2026
5 checks passed
@jfberry jfberry deleted the feat/huma-pokemon-v2-v3-scan branch June 18, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant