feat(context-agent): registry-fed property bitmap + context-signal targeting#363
feat(context-agent): registry-fed property bitmap + context-signal targeting#363ohalushchak-exadel wants to merge 7 commits into
Conversation
…rgeting
Two capabilities for the production context-match agent:
1. Registry persistent store (opt-in via REGISTRY_ENABLED). When enabled,
the agent runs a background registry sync loop and feeds the engine's
global property bitmap from a live PropertyIndex instead of the static
PROPERTY_RIDS env stopgap. Supports memory and Redis/Valkey index
backends, optional file-backed cursor, panic-recovered sync goroutine,
and a /live staleness check driven by a per-poll success beacon
(SyncerConfig.OnSuccessfulPoll).
2. Context-attribute signal targeting. PackageContextConfig gains a
ContextSignals profile (any_of / none_of) evaluated against the
signal:{owner}:{key_types}:{values} keyspace in a single deduped MGet
per request. The KeyType taxonomy is restricted to context attributes
(url_hash, geo, topic, artifact-ref public IDs); identity key types are
rejected at three layers — write-time validation, read-time ExpandKeys,
and engine fail-closed — so the context path can never issue an
identity-keyed lookup. Topic values are taxonomy-namespaced; the
per-cfg cartesian expansion is capped and fails closed on trip.
pkgconfigstore gains a batched MGet (direct + cache) so candidate configs
load in one round-trip. Signal evaluation is skipped end-to-end when no
candidate carries a profile.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The registry-fed property bitmap added a cmd/context-agent dependency on the registry and registry/redisstore modules via local replace directives (../../registry). The Docker build context omitted that tree, so `go build` could not resolve the replacement. Copy registry/ alongside the other module dirs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Approving. Identity-vs-context boundary holds end-to-end — Cfg.Validate, ExpandKeys, and signalsPass are three independent gates and ExpandKeys re-checks IsAllowed before composing any key, so the write-side Store.Set bypass acknowledged in the diff is caught on read.
Things I checked
signalstore.AllowedKeyTypesmatchesArtifactRefType ∪ {country, region, metro, topic}minus rawurl— identity types (eid,email, theUIDTypeenum intmproto/types_gen.go:88-101) are intentionally absent. (targeting/signalstore/signalstore.go:60-86)- Topic namespacing
{tax.source}:{tax.id}:{topicID}matchestopicstore.Taxonomy.String()attopicstore/topicstore.go:87-92, so the writer and reader are guaranteed to key the same bytes from one source of truth. Unaccepted-taxonomy silent drop mirrors the existing topic-match path atengine.go:144-153. HashURLcanonicalization (tmproto/urlcanon.go:33-70, Blake3-256 + std base64 over scheme/www/query/fragment-stripped lowercase form) is the right shape for the writer/reader to agree on;ArtifactRefTypeURLcorrectly hashes before key composition andKeyType("url")is rejected by Validate (delimiter collision).- Fail-closed posture: NoneOf evaluated before AnyOf so a cap-trip on a blocklist still rejects. Pinned by
TestMatchProfile_NoneOfCapTripFailsClosedintargeting/signalstore/signalstore_test.go:209-240. MGet error fails-closed every package with a profile while bare packages still evaluate —TestContext_SignalMGetError_FailsClosed. Shutdown()idempotency:context.CancelFuncis safe to call repeatedly, receive on a closedsyncDonereturns immediately,closeRedistolerates a repeatClose(). No race. (cmd/context-agent/registry_setup.go:170-181)syncErr atomic.Pointer[error]andlastSuccess atomic.Int64are read/written across goroutines correctly; the local nil-check before*errderef inLivenessCheck.Fnis sound.pkgconfigstorecache MGet alignment:missIDs[j] == packageIDs[missIdx[j]]by construction so theAddkey incache.go:97is equivalent to the input id.Service.Putvalidation rejects identity-keyed and missing-signal-id cfgs at write time —TestService_PutRejectsInvalidContextSignalscoversKeyType("eid")returningErrCfgUnsafe.
Follow-ups (non-blocking — file as issues)
- Delimiter injection in
signal:*keys.Key()writescustomartifact-ref values, topic IDs, andregion/metro.valueverbatim with no escape — a publisher-supplied value containing,or:can shadow a legitimately-written compound-tuple key when both sides happen to land on the same byte sequence. The KeyTypes segment is a partial discriminator (different cfg shapes produce different middle segments), so the realistic collision needs the writer's data to already carry,inside a single value — narrow, but real. Fix is one of: reject any value containing:or,insideappendUnique, or haveKey()return""on bad bytes (same fail-closed shape as the empty/mismatched case atsignalstore.go:174). (targeting/signalstore/signalstore.go:189-194,targeting/engine_signals.go:48,55,82) - No profile-level or request-side cardinality cap.
maxKeysPerCfg=4096bounds a single cfg, butPlanLookupcan union N cfgs at near-cap each, andreq.ArtifactRefsis unbounded at the engine boundary —appendUniquededupes but a maliciously-sized request feeds the cartesian.Service.Putalso accepts arbitrarily many cfgs per profile. Recommend: caplen(data[kt])at extract, caplen(out)inPlanLookup, and enforce a smalllen(AnyOf)+len(NoneOf)ceiling inProfile.Validate. All three should fail-closed withErrCfgUnsafeto inherit the existing engine pattern. SignalMGetis a required method onContextStorage. The interface lives at the module root (targeting/context_storage.go:97) so this is a public Go API change for any external implementer.ContextConfigswas added optionally via feature-detect (contextConfigBatcher) —SignalMGetcould have been the same. The branch commit isfeat(context-agent):with no!orBREAKING CHANGE:footer; if release-please treatstargetingas a stable surface, the next tag will need a SemVer minor at minimum and arguably a major. Worth confirming the release-please policy here.- Whole-batch error observability regression. When the storage implements
contextConfigBatcherand the whole batch errors,e.metrics.StoreError(ctx, "context_config", err)is recorded once for the whole candidate set; the pre-batch path recorded one per failing package. Either emit N events for the batch path, or document the metric semantic change. (targeting/engine_signals.go:166-175) OnSuccessfulPollfires once per page, not once per poll cycle. The docstring atregistry/syncer.go:23-27reads as a per-iteration beacon but the hook is inside the page loop, so a 10-page bootstrap fires 10 times. Benign for liveness (countlets operators distinguish), but pin the semantic in the test. (registry/syncer.go:206-208)- Spec confirm:
geo.metrois currently flattened to"{system}:{value}"(engine_signals.go:138-147).MetroAreaSystemexists as an enum intmproto/types_gen.go:30-35butGeoismap[string]anyand the schema forgeo.metroshape isn't in this tree. Worth a one-liner confirming againstadcp/schemas/tmp/context-match-request.jsonupstream that the wire shape is{system, value}and not a flat string. signal_owner_idnaming. Reader/writer use this name uniformly here, but if upstream AdCP signals-agent vocabulary calls itsignal_provider_idor similar, reconcile before any cross-agent wire surface lands.
Minor nits (non-blocking)
debug.Stack()called twice.registry_setup.go:153,156— both calls capture the same goroutine stack. Collapse to one local and reuse.signalLogger = slog.Default()falls through to the global. The rest of the engine has no logger field, so this is a one-off surface. If more diagnostic sites land, thread a logger throughContextEngineConfiginstead.appendUniqueis O(n²). Fine at current request sizes; revisit if the artifact-ref cap ever lifts.
LGTM. Follow-ups noted below.
|
The automated review encountered an issue (possibly reached max turns, timed out, or failed to post the final This is an automated message from the Argus AI review workflow. |
There was a problem hiding this comment.
Substantial feature add. Privacy boundary holds end-to-end — signalstore.Cfg.Validate (targeting/signalstore/signalstore.go:116-129), ExpandKeys (targeting/signalstore/signalstore.go:212-254), and Service.Put (targeting/pkgconfigstore/pkgconfigstore.go:65-67) each independently reject any KeyType outside allowedKeyTypes, and extractSignalLookupData (targeting/engine_signals.go:35-91) never projects identity dimensions in the first place. Three load-bearing layers, no escape hatch. Clean fail-closed posture on ErrCfgUnsafe propagation (none_of evaluated before any_of so a cap trip on a blocklist still surfaces). One blocking question below before approve.
Things I checked
- ArtifactRefType → KeyType projection (
targeting/engine_signals.go:108-132) matches the canonical AdCP set 1-for-1, withurlcorrectly folded intourl_hashviatmproto.HashURL. - Cartesian cap (
signalstore.go:228-230) is checked progressively inside the loop, before thekeysallocation — no allocation-DoS path. MatchProfilenone_of/any_of ordering is correct and tested atsignalstore_test.go:244.signalsPassreturns false on bothErrCfgUnsafeandfetchErr, mirroring URL-filter / topic-match patterns elsewhere in the engine.- Registry sync:
OnSuccessfulPollfires after cursor save (registry/syncer.go:206), correct liveness semantics. Panic recovery inrunSyncercaptures and stores; cancel propagation throughShutdownis idempotent. - Credential paths (
REGISTRY_REDIS_PASSWORD,REGISTRY_FEED_TOKEN) flow only intoredis.Options{Password}and the bearer header. No echo throughfmt.Errorfwrapping —cfg.RedisAddrishost:port, not a URI carrying credentials. - TLS on registry Redis (
registry_setup.go:251-256) —tls.Config{MinVersion: TLS12}with emptyServerNameis acceptable: stdlib derives ServerName from the host portion ofAddr. Only fails open on IP-literal Addr; example.env uses hostnames.
Open question (would flip me to approve)
registryPropertyBitmap adapter likely cannot match a spec-compliant property_rid. cmd/context-agent/registry_setup.go:236-249. The spec defines ContextMatchRequest.PropertyRID as a UUID-v7 string (tmproto/types_gen.go:119: "Property catalog UUID (UUID v7). Globally unique, stable identifier..."). The registry's Property.PropertyRID is uint64 (registry/property_index.go:13). The adapter tries (a) LookupByID(rid) — matches against property_id, slug-shaped — and (b) ParseUint + LookupByRID(n) — matches the uint64. A UUID-v7 string fails both: it isn't a slug, and ParseUint rejects the hyphens. With REGISTRY_ENABLED=true, every spec-compliant inbound short-circuits at the top of Evaluate and the agent serves empty offers.
This is opt-in (default REGISTRY_ENABLED=false) and the fail mode is fail-closed, so it's not a leak — but it likely means the registry path serves nothing until resolved. What's the intended bridge? Two ways this could already be fine:
- Your registry feed populates
Property.PropertyIDwith the UUID-v7 form soLookupByID(rid)matches (the field-name collision is just confusing). - Your operators run with
property_ridnumeric internally and inbound requests carry the numeric form, accepting the spec divergence.
A line in the example env explaining the assumed registry shape would resolve this. If neither bridge holds, the adapter needs a third lookup dimension.
Follow-ups (non-blocking — file as issues)
- Doc/code drift on
storage.SignalMGetnil signals.targeting/contextagent/storage.go:85-90returns(nil, nil)whens.signals == nil;DecodeValuesreturnsnilon length mismatch;MatchProfile(data, nil)is then called. Anone_of-only profile passes vacuously — no key hit → no block. Production wiressignals: rawStoreatsetup.go:402so unreachable today, butbuildStorage's doc claims this "skips every package... as a fail-closed safety measure." Either reject nil atbuildStorage, or return an error fromSignalMGetwhensignals == nil && len(keys) > 0sofetchErrpropagates andsignalsPassfails-closed. - No request-wide cap on
PlanLookupkeys.signalstore.go:331-369— per-cfg cap is 4096, but withNcandidate packages × multi-key any_of cfgs the deduped MGet could plan4096 × Nkeys. RealisticNkeeps this bounded; amaxKeysPerPlan(e.g. 65536) trippingErrCfgUnsafefor the request would be defense in depth. - Topic namespacing is a private invariant. Spec ContextSignals carries
TaxonomySource/TaxonomyIDon the envelope, not per-topic. The{source}:{id}:{topicID}convention intargeting/engine_signals.go:74-85is necessary for multi-taxonomy disambiguation but isn't a spec form — anyone authoring cfgs out-of-band must know to prefix. Worth a docstring onPackageContextConfig.ContextSignals. - Registry liveness window during slow first bootstrap.
lastSuccessis seeded at bundle-return time (registry_setup.go:119), butHydrateruns synchronously before that. WithBootstrapLimit=10000, a slow first-page fetch plus retries could approach the 5-minute staleness threshold before the firstOnSuccessfulPollfires. Worth either stampinglastSuccessfrom inside the goroutine just before the first FetchFeed, or documenting the threshold relative to bootstrap budget. runSyncerpanic recovery usesb.loggerwithout a nil-check (registry_setup.go:149-160). Always non-nil from main, but a test callingrunSynceragainst a hand-built bundle nil-derefs insiderecover. Defensiveif b.logger != nilcloses the gap.- Defense-in-depth on
signalstore.Key()escaping (signalstore.go:173-196). Publisher-controlledtopicvalues containing,produce malformed read keys. Denial-of-match, not key-smuggling, butappendUniquecould reject values containing,or:outside the knownsystem:value/source:id:topicshapes.
Minor nits (non-blocking)
- Memory-backend file cursor mismatch.
cmd/context-agent/registry_setup.go:90-92lets memory-backend index + file-backed cursor coexist. A restart resumes against the cursor with an empty index, then bootstraps incrementally instead of doing the full reload the memory-backend comment promises. Worth either rejecting the combination or documenting the resulting behavior.
Verdict: LGTM after the property_rid adapter question is resolved — if your registry feed populates PropertyID with the UUID-v7 form, the existing LookupByID branch is the answer and we ship.
- signalstore: drop comma-bearing values in appendUnique so a request value can't shadow a compound-tuple key (delimiter guard); add a request-wide PlanLookup key cap (maxKeysPerPlan) and a per-profile cfg-count cap in Validate, both fail-closed via ErrCfgUnsafe. - contextagent/storage: SignalMGet returns an error when no signal store is wired but keys were requested, so a none_of-only profile fails closed instead of passing vacuously (matches buildStorage's documented contract). - registry_setup: capture debug.Stack() once in the panic recover; nil-check b.logger so a hand-built bundle in tests can't nil-deref; document the memory-backend + file-cursor resume behavior. - syncer: clarify OnSuccessfulPoll fires per feed page, not per drain cycle. - entity: document the topic namespacing invariant on PackageContextConfig.ContextSignals. Tests added for the profile cfg-count cap, request-wide plan cap, comma-value drop, and the nil-signal-store fail-closed path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
LGTM. Follow-ups noted below. Architecturally right: hot path skips the entire signal stage when zero candidates carry a profile, single deduped MGet per request, fail-closed everywhere a buyer's brand-safety cfg could otherwise silently pass.
Things I checked
- Three-layer privacy guard against identity key types: write-time
signalstore.Cfg.Validate(signalstore.go:124), write-time gate atpkgconfigstore/pkgconfigstore.go:65inService.Put, read-timeExpandKeysIsAllowed(signalstore.go:230), enginesignalsPassfail-closed (engine_signals.go:281-297). Verified byTestContext_SignalInvalidKeyType_FailsClosedandTestPlanLookup_PropagatesCfgError. extractSignalLookupData(engine_signals.go:36-92) never projects identity dimensions intoLookupData— every write site goes through a fixed switch with no identity arms.tmproto.ContextMatchRequest(types_gen.go) carries no identity-bearing field. The Go-side struct enforces the wire contract.- ArtifactRefType→KeyType projection is complete.
ArtifactRefTypeURLis correctly normalized viatmproto.HashURLontoKeyTypeURLHash(engine_signals.go:48-50);ArtifactRefTypeURLHashmaps to the sameKeyTypeURLHash(engine_signals.go:121-123).KeyTypeURLis deliberately absent — raw URLs would collide with the,delimiter. - Topic namespacing reuses
topicstore.Taxonomy.String()("source:id" form) — same form the existing topic-match path keys on, so signal and topic-match cannot drift apart on the prefix scheme. Tested byTestContext_SignalTopicTaxonomyNamespacingandTestContext_SignalTopicUnacceptedTaxonomyDrops. - Comma-bearing values are dropped at a single sink (
engine_signals.go:105-115) before reachingsignalstore.Key, so the,tuple delimiter is unshadowable from publisher-controlled bytes (artifact_ref custom values, country, region, metro, topic).TestContext_SignalCommaValueDroppedexercises the custom-ref path end-to-end. maxKeysPerCfg=4096checked after every multiplication onint(signalstore.go:238-241) — no allocation reaches a memory-unsafe size.maxKeysPerPlan=65536is the request-wide backstop and fails closed for every profile-bearing package (TestPlanLookup_RequestWideCapFailsClosed).maxCfgsPerProfile=256is operator-controlled, not request-controlled.signalsPass(engine_signals.go:268-300) fail-closes in every error path I traced:fetchErr != nil,MatchProfileerr,ErrCfgUnsafe. Logger nil-checked at line 287 so a hand-built engine in tests can't nil-deref.TestContext_SignalMGetError_FailsClosedpins it.storage.SignalMGetreturns an error whens.signals == nilbut keys were requested (contextagent/storage.go:86-99) — without this, anone_of-only profile would pass vacuously when no signal store is wired. Right shape.registryBundle.Shutdownis genuinely idempotent:cancelis stdlib-idempotent,<-syncDoneon a closed channel returns immediately on repeat,closeRedistolerates re-close.TestRegistryBundle_ShutdownIdempotentexercises double-call and nil receiver.runSyncer(registry_setup.go:158-181) recovers panics intosyncErrand surfaces them via the LivenessCheck (lines 207-230).debug.Stack()captured once in the recover, per the third commit's fix.buildRegistryHydrate failure paths all callcloseRedis(redisCloser)before returning (registry_setup.go:103-114). No leak on partial-failure.clonePackageContextConfigclones the newContextSignalspointer viacloneSignalProfile+cloneSignalCfg(cache.go:146, 150-176) — profile, AnyOf/NoneOf slices, and per-cfg KeyTypes slice are all freshly allocated, so a cache-returnedProfilecan't be mutated into a stored one.OnSuccessfulPollcallback wired per page inregistry/syncer.go:207-209(not per drain cycle), drivinglastSuccessso a quiescent-but-healthy feed still proves liveness. Documented at SyncerConfig.OnSuccessfulPoll (syncer.go:20-29).- The signal gate runs BEFORE the URL block/allow filter in the per-package loop (engine.go:306 vs :310). Intentional — fail-closed on signal errors drops the package before brand-safety even runs — but inverts the conventional "policy before personalization" ordering. Acceptable; worth a comment.
Follow-ups (non-blocking — file as issues)
ExpandKeysshort-circuits on empty data before validating remaining key types (signalstore.go:223-242). For a cfg withKeyTypes=[url_hash, eid]and an emptydata[url_hash], iteration 0 returns(nil, nil)at line 235 before iteration 1 reaches the disallowedeid. In anone_ofposition that cfg would silently not-block instead of propagatingErrCfgUnsafe. Not wire-exploitable —extractSignalLookupDatanever populatesdata[identity_key], and the write-timeValidategate catches this shape today. But the PR's "three-layer" claim is read-time-shape-dependent for one cfg shape. Fix is a single pre-pass: validate everyKeyTypein a top-of-function loop before anydata[kt]lookup, mirroringCfg.Validateat line 116. One-liner change; carries a test for the mixed-key none_of case.PlanLookupis all-or-nothing on cap trip / unsafe cfg (signalstore.go:353-389). One misconfigured cfg in one package blackholes every signal-gated package in the request. Fail-closed direction is correct, but a noisy single cfg can take out the whole signal stage. Consider per-profile isolation in a follow-up — log the offending owner/index so the operator can locate the bad cfg without a transcript.cachedReader.MGetdoes not validateinner.MGetlength alignment (pkgconfigstore/cache.go:86-97). Direct reader enforces this at pkgconfigstore.go:142-144; cache wrapper trusts the inner length and indexesmissIdx[j]blind. Defense-in-depth — add the same length check.OnSuccessfulPollfires whencursor.Savefails under the 3-failure threshold (registry/syncer.go:192-209). The save-failure branch incrementssaveFailuresand logs but flows through to the hook. Liveness reports healthy while persisted cursor is stale; the escalation to a hard return only triggers on the third consecutive failure. Acceptable for liveness intent (in-memory state did advance) but worth either moving the hook above the save block or commenting the deliberate choice.- Default empty
REGISTRY_KEY_PREFIX(cmd/context-agent/registry_config.go:95,registry/redisstore/store.go:60-71). Empty prefix warns once and defaults; two deployments sharing a Valkey without setting it collide. Hard-fail in production builds would be safer than warn-and-default. - Metro key shape
{system}:{value}is engine-invented (engine_signals.go:151-165). Reader and writer must agree byte-for-byte. Pin the contract in theentity.goContextSignals docstring alongside the existing topic-namespacing note, otherwise the next writer-side change will drift silently. - Signal-gate-before-URL-filter ordering (
engine.go:306vs:310). Defensible — fail-closed signal errors should drop the package before the URL filter runs — but inverts the conventional cheapest-reject-first ordering. One-line comment explaining the choice keeps it from being re-litigated. maxKeysPerCfg=4096is tight for a topic-only cfg with a high-cardinality publisher (signalstore.go:270). Realistic publishers send <100 topics so it's theoretical; flag if a buyer needsKeyTypes=[topic]against a publisher fanning out heavy IAB taxonomies.- Double Hydrate (
registry_setup.go:103-114thenregistry/syncer.go:87-99). Idempotent viaidx.hydrated.CompareAndSwap, so this is safe — just confusing on read. Pick one site to own the call.
Minor nits (non-blocking)
Key(0, …)is permitted.signalstore.go:184-207:SignalOwnerID=0producessignal:0:…. Not a bug, but if the writer convention disallows owner 0 (most do),Cfg.Validatecould reject it.
The third commit's title says "address review nits" but the request-wide plan cap and the comma-value signalstore drop landed there — substantive privacy-hardening, not nits. Worth flagging in the squash message if this lands as one commit.
LGTM.
There was a problem hiding this comment.
Approving. The three-layer identity-key-type rejection holds end-to-end — I walked every path from ContextMatchRequest to a Valkey MGet and could not reach the identity keyspace. None_of cap-trip propagates ErrCfgUnsafe correctly. Comma drop at extraction prevents writer-key shadowing. Clean fix shape.
Things I checked
signalstore.Cfg.ExpandKeys(targeting/signalstore/signalstore.go:229-231) is the only call site that buildssignal:*keys outside tests —Keyis otherwise unreachable from the data path. Identity key types die atIsAllowedbefore any MGet is planned.MatchProfile(signalstore.go:308-334) evaluatesNoneOffirst; a none_of cap-trip propagatesErrCfgUnsafeand surfaces fail-closed even when any_of would pass.signalstore_test.go:245-280pins the reproducer.signalsPass(engine_signals.go:268-300) returns false onfetchErr != nilAND onMatchProfileerr. Bare packages (no profile) still pass vacuously when other packages' fetch failed —TestContext_SignalMGetError_FailsClosed(engine_signals_test.go:186-213) checks that exact shape.Service.Put(pkgconfigstore/pkgconfigstore.go:65-67) gates writes onProfile.Validate. The reader-sidedecodeConfigdoesn't re-validate, but the engine path is load-bearing — everyExpandKeysre-checksIsAllowed. Three-layer claim stands.extractSignalLookupData(engine_signals.go:36-92) projects ArtifactRefs / geo / metro / topic with comma-drop at theappendUniqueseam. Metro is composedsystem:value; topic istax.String() + ":" + topic. Operator-controlledacceptedTaxonomiesplusstrconv.Itoanumeric ID means no,injection vector.cachedReader.MGet(pkgconfigstore/cache.go:66-99) deep-clones viacloneSignalProfile, coveringKeyTypesslice and per-cfg structs.TestCache_HitClonesConfigToIsolateCallersverifies isolation.registry_setup.golifecycle:runSyncerpanic-recovers withdebug.Stack;Shutdownis idempotent across cancel/syncDone/closeRedis;LivenessCheckreadslastSuccessatomically.registry/syncer.go:207firesOnSuccessfulPollafter the persist + cursor-advance branch.TestSyncer_OnSuccessfulPollFirescovers the zero-event drain case.- Conventional-commit story:
feat(context-agent)+ twofix(context-agent)review-nit follow-ups. No wire-path symbol removed or renamed.ContextStoragegains a method (see follow-up below). - The PR description's "known pre-existing failure" on
registry/TestSyncer_AppliesAuthorizationRevokedchecks out — the syncer.go change here is theOnSuccessfulPollhook only, which the failing test doesn't exercise.
Follow-ups (non-blocking — file as issues)
DecodeValuesmismatch path is fail-open. If asignalMGetimpl returnslen(values) != len(keys)without an error,signalstore.DecodeValuesreturns nil (signalstore.go:395-398),fetchSignalsForCandidatesreturns(nil, nil)(engine_signals.go:256), and a none_of-only blocklist profile then sees an empty fetched map and passes vacuously throughMatchProfile. Shipping impls align by construction (InMemory,contextagent/storage.gopass-through), so this is not reachable today — but the PR's own posture is fail-closed beats fail-open. Cheapest fix: infetchSignalsForCandidates, whenkeysis non-empty andDecodeValuesreturns nil, return(nil, ErrCfgUnsafe).decodeConfigskipsProfile.Validate.pkgconfigstore.decodeConfig(pkgconfigstore.go:163-169) doesn't re-validate. The engine-sideExpandKeysstill catches identity types;maxKeysPerCfg+maxKeysPerPlanbound the blast radius. Defense-in-depth, not a DoS amp.- HTTPS pin on
REGISTRY_FEED_URL.registry_config.go validate(L117-138) doesn't require anhttps://scheme whenFeedToken != "". A fat-fingeredhttp://...ships the bearer in plaintext. Operator config but trivial to check at boot. OnSuccessfulPollfires on cursor-save retry path.registry/syncer.go:194-209invokes the hook after acursor.Savefailure whensaveFailures < 3, despite the docstring claim of "fetches AND persists cleanly." Gate onsaveFailures == 0or update the doc.lastSuccessseeded totime.Now()before first poll (registry_setup.go:128). If the first feed call hangs > 5 min,/livereports healthy throughout. Seed to zero and special-case zero in the check, or document the intentional window.Cfg.Validateaccepts comma-bearingSignalID. A misconfigured cfg withSignalID=\"a,b\"validates, persists, and then never matches anything becausesplitCSVdecoded segments can't equal\"a,b\". Reject,inSignalIDat Validate-time for symmetry with the read-path comma drop.ContextStoragegainsSignalMGetwithout a breaking-change marker. Adding a required method to the exportedtargeting.ContextStorageinterface (targeting/context_storage.go:97) breaks any out-of-repo implementer. MUST FIX §6 covers "remove or rename" — adding a required method isn't strictly in scope — but release-please will tag this as a minor bump, which is understated. Worth aBREAKING CHANGE:footer if anyone external is implementingContextStorage, otherwise a one-line "no external implementers yet" call-out somewhere durable.
Minor nits (non-blocking)
appendUniquecomma drop is silent.engine_signals.go:105-108— bump a counter so an operator can detect a key_type whose values are 100% dropped. The comment explains the why; a metric would explain the what.maxKeysPerPlantest comment.signalstore_test.go:391— comment says "17 cfgs > 65536 keys" but the loop generates 32. Math is fine, comment is stale.registry_setup.go:142-143— the empty-bitmap warning fires whenproperties.Count() == 0after Hydrate. With a Redis backend resuming from a saved cursor the index should already be populated, so this branch is really for the memory-backend cold-start case. Worth one line distinguishing the two so an oncall doesn't chase a phantom.
Safe to merge.
The AdCP property registry catalog spec defines property_rid as a
catalog-assigned UUID v7 ("the shared key for TMP matching"), and the
change-feed property.created payload carries it as a string
("019539a0-..."). registry.Property modeled it as uint64, so a
spec-compliant feed event failed to unmarshal (string -> uint64) and
was dropped, and the context-agent's property bitmap could never match
an inbound UUID property_rid — the registry-fed path served empty
offers for every spec-compliant request.
Change Property.PropertyRID to string and the index's RID dimension to
map[string]*Property (LookupByRID, PropertyRID, byRID), guard byRID
inserts on non-empty RID (replacing the uint64==0 sentinel), and update
the syncer's required-field check. The context-agent property-bitmap
adapter collapses to a single LookupByRID(rid) and no longer consults
the property_id slug (matching is on property_rid per spec).
The router's separate string-typed registry (router/registry.go) was
already spec-correct and is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
LGTM. Follow-ups noted below. The privacy partition is the load-bearing claim — verified at all three layers (write Profile.Validate, read Cfg.ExpandKeys + IsAllowed, engine fail-closed via signalsPass on a non-nil signalErr) — and identity key types never reach extractSignalLookupData, so a misconfigured cfg cannot key an MGet on user data even if the gate downstream were ever bypassed.
Things I checked
- Three-layer privacy boundary:
targeting/signalstore/signalstore.go:71-101(allowed set),:223-242(read-timeIsAllowedbefore any lookup),:148-166(Profile.Validaterejects identity at write), andtargeting/engine_signals.go:225-257/:268-300(enginesignalErrfail-closed on plan or fetch failure). No path fromPackageContextConfig.ContextSignalsto a Valkey lookup bypasses all three. none_offail-closed direction is the right shape:ErrCfgUnsafefromExpandKeyspropagates throughMatchProfile(signalstore.go:308-334) andsignalsPassreturns false rather than passing vacuously — the specific case the engine pattern needs.- Cartesian containment: per-cfg cap 4096 at
signalstore.go:239-241, request-wide cap 65536 at:371-373, comma-bearing values dropped atengine_signals.go:106-108so a request value cannot shift the parse and shadow a compound-tuple key.HashURLis base64 (tmproto/urlcanon.go) — no delimiter collision. Topic values are taxonomy-prefixed (engine_signals.go:75-86) so the same numeric topic id under iab:7 vs custom can't collide. Property.PropertyRID uint64→string(commit 810ae12): spec-correct against the catalog UUID-v7 pertmproto/types_gen.go:119andadcp/types_gen.go:7052.router/registry.gowas already string-typed, so the two registries are now aligned. The change-feed eventproperty.createdcarries it as a string ("019539a0-…") and would have failed to unmarshal into the prioruint64, dropping silently —registry/syncer.goconfirms the previous path would have produced the empty-bitmap behavior the PR cites.- Conventional-commit story: no exported wire-path symbol change in
adcp/types.go,tmproto/*,routerpublicRouter*,targetingpublic,tmpclient, orcmd/router/main.goenv-var contracts.fix(registry):is defensible framing for a spec-correctness fix; see Follow-up #1. cmd/context-agent/Dockerfile:7correctly copiesregistry/into the build context to resolve the new localreplacedirectives incmd/context-agent/go.mod:43-45.- Test plan: PR cites the pre-existing
registry/TestSyncer_AppliesAuthorizationRevokedfailure as reproducing on main with this PR'ssyncer.goreverted — verifiable claim, not introduced here. No unchecked manual smoke item in the PR body.
Follow-ups (non-blocking — file as issues)
- `fix(registry):` for a public Go API break of the registry submodule. `Property.PropertyRID uint64→string` breaks any external importer of `github.com/adcontextprotocol/adcp-go/registry`. Spec-correctness framing is defensible — the prior code couldn't unmarshal a real feed event — but downstream Go consumers reading release-please notes get no signal. Worth a `BREAKING CHANGE:` footer or a `fix!:` recut on the next registry-submodule release; at minimum, a release-note line in the registry submodule's CHANGELOG when it next versions.
- Hydrate against pre-PR persisted registry state will fail-fast at boot. `LoadProperties` (`registry/redisstore/store.go:142-146`, `registry/glidestore/store.go:155-167`) will return `json: cannot unmarshal number into Go struct field Property.property_rid of type string` against any Redis populated before this PR. `cmd/context-agent/registry_setup.go:102-104` treats Hydrate failure as fatal. Document a `Clear` / migration step in the upgrade notes so the first deploy on a warm Valkey doesn't refuse to start.
- `Cfg.SignalOwnerID` is still `uint64` (`signalstore.go:108`). The only numeric ID left on what is now a string-typed surface. Encoded into the key via `strconv.FormatUint` (`:191`). If AdCP's signals/curation spec models owner as `agent_url` (string), this is the next type fix in the same line as PropertyRID. Worth a one-line check against the upstream signals task schema.
- Bundle `lastSuccess` is seeded from `time.Now()` before the syncer has polled (`cmd/context-agent/registry_setup.go:127`). A registry feed unreachable from boot shows `/live` green for the full `registrySyncStaleThreshold = 5*time.Minute`. The comment at `:124-126` is self-aware about it, but a startup with an unreachable feed currently looks healthy to a load balancer for the first five minutes. Seed to zero, or document the grace window in the deploy guide.
- Memory backend + non-empty `CursorPath` is a footgun (`cmd/context-agent/registry_setup.go:90-99`). A restart resumes the cursor against an empty index, so everything emitted before the saved cursor stays absent — the staleness probe goes green (feed is polling) while the bitmap is materially incomplete. Reject the combination at config validation, or fail the readiness probe when index size collapses across restart.
- `pkgconfigstore.decodeConfig` does not re-validate the loaded profile. A pre-PR config (>256 cfgs, or a writer that bypassed `Service.Put`) is accepted on read and only backstopped by the request-wide `MaxKeysPerPlan`. Burns CPU before the cap trips. Have `decodeConfig` validate and treat invalid as "no config" — the engine then skips the package.
- `REGISTRY_REDIS_TLS=false` with a non-empty password is silently accepted (`example.standalone.env:90`, `registry_setup.go:67-78`). Operators following the example ship the registry password in cleartext. WARN on startup, or refuse to start when `RedisPassword != "" && !RedisTLS`.
Minor nits (non-blocking)
- `signalstore.Key` silently returns `""` on invariant violation. `targeting/signalstore/signalstore.go:184-207`. An empty key gets seeded into the dedup map and pushed into the MGet batch, where it crowds `maxKeysPerPlan` and produces a `signal::` row no Cfg can match. The invariant is internal — return an error or panic so a future caller that hands in mismatched slices gets a real signal, not a corrupt batch.
- Signal-store nil-guard is dead code in the only real wiring. `targeting/contextagent/storage.go:90-97` fails closed when `s.signals == nil`, but `buildBundle` always passes `rawStore`. Defense-in-depth is fine; consider asserting non-nil at construction so the runtime check is documentation, not a live branch.
- `OnSuccessfulPoll` fires per feed page, not per drain cycle (`registry/syncer.go:207-209`). The docstring matches behavior. But `registrySyncStaleThreshold = 5m` sized for "one missed cycle" assumes steady-state polling; a slow bootstrap drain with >5m between pages will trip liveness. Probably fine; flagging in case the feed gets fat.
Two features in one commit (registry sync + signal targeting) — a future bisect for either lands on the 2,848-line merge. Worth noting only because the title spells it out.
Safe to merge.
PropertyRID just became a string; SignalOwnerID was the last numeric ID on the targeting surface. Making it a string lets the owner identifier be numeric, a UUID, or an agent URL without a future type change, and stays byte-compatible with the existing Valkey keyspace — a numeric owner formatted as decimal digits produces the same signal: key bytes the writer already emits. Key() now writes the owner string verbatim (dropping strconv.FormatUint) and returns "" on an empty owner, and Validate rejects an empty signal_owner_id. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Approving. Substantive PR with the privacy boundary done right: three-layer identity firewall (write-time Validate, read-time ExpandKeys, engine fail-closed), fail-closed posture on every cap trip and storage error, and the property_rid spec fix unblocks the registry-fed bitmap from serving empty offers on every spec-compliant request.
Independent verdicts that drove the call:
ad-tech-protocol-expert: spec-correct on all three claims.property_ridas UUID-v7 string matchestmproto/types_gen.go:119("primary key for TMP matching"); thesignal:*identity firewall matches the TMP context-match "MUST NOT contain user identity" contract;none_ofcap-trip propagation is the right posture.security-reviewer: identity firewall safe at all three layers, fail-closed posture safe under MGet failure /PlanLookupcap trip /ContextConfigsbatch error / nil-store-with-keys, no identity-bearing log fields, no exploitable delimiter shadowing in deployed config.
Things I checked
- Three-layer identity firewall:
Cfg.Validaterejects identity key_types attargeting/signalstore/signalstore.go:128-131;pkgconfigstore.Putvalidates attargeting/pkgconfigstore/pkgconfigstore.go:65-67before anySet;ExpandKeysre-checks atsignalstore.go:234-237; enginesignalsPassfails the package closed attargeting/engine_signals.go:285-297. extractSignalLookupDataatengine_signals.go:36-92never sources identity dimensions — even a hypothetical downstream bypass cannot supply identity values to the MGet.MatchProfileevaluatesNoneOfbeforeAnyOfatsignalstore.go:317-325so a blocklist cap trip surfaces before any_of can mask it (pinned bysignalstore_test.go:245-280).appendUniquedrops comma-bearing request values atengine_signals.go:106-108. The writer uses the sameKey()encoding so the drop is a no-op for any value that could legitimately have been written.KeyTypeURLis intentionally absent fromallowedKeyTypes; raw URL refs project ontoKeyTypeURLHashviatmproto.HashURLatengine_signals.go:48-51— no path builds asignal:*key from a raw URL.Property.PropertyRIDchange matchestmproto/types_gen.go:119. The router's separate string-typed registry (router/registry.go) was already spec-correct and is untouched as the PR description claims.registryBundle.Shutdownis idempotent —cancelis idempotent,syncDonereceivers on a closed channel both unblock,closeRedistolerates double-close, the panic recover captures stack intosyncErr.
Follow-ups (non-blocking — file as issues)
-
Versioning the multi-module breaking changes.
registry.Property.PropertyRID(uint64→string) andContextStorage.SignalMGet(new required method) are public-Go-API breaks.release-please-config.jsononly manages theadcppackage, so neither commit needs!for release-please — butregistryis a published Go module (github.com/adcontextprotocol/adcp-go/registry) and downstream consumers will see a compilation break ongo get -u. Document the pre-1.0 policy forregistry/targeting, or cut explicit module tags. The PR description already flagged this. -
none_ofcfg keying on a dimension the publisher omitted silently doesn't fire.ExpandKeysreturns(nil, nil)for an absent dimension (signalstore.go:239-241), so a topic-keyed blocklist is bypassed when the publisher sends no topics at all. Documented intent, but worth ametrics.ContextEvaluated(ctx, StageSignalMatchAbsentDimension, ...)or debug log so operators can see blocklist evasion when it matters. -
cmd/context-agent/registry_setup.go:75— boot-timeclient.Ping(ctx)has no timeout. Unresponsive Redis stallsbuildRegistryuntil SIGTERM. Wrap the ping withcontext.WithTimeout(ctx, 5*time.Second). -
url_hashartifact-ref values aren't shape-checked atengine_signals.go:122-123. The spec-canonical form is 44-char base64 from Blake3-256. If the spec already constrains the wire shape, moot; otherwise a length/charset guard would close the gap thatappendUnique's,drop doesn't cover.
Minor nits (non-blocking)
-
registry/property_index.go:107— mirror thePutguard onRemove.Putat:79-81only inserts intobyRIDwhenPropertyRID != "";Removedeletes unconditionally. The delete-on-""is a safe no-op today; consistency is cheap. -
targeting/signalstore/signalstore.go:118-134—Cfg.Validatedoesn't rejectSignalOwnerID(orSignalID) containing:or,. Operator-supplied config, write-time governance — not exploitable in deployed config. Defense-in-depth at validation is cheap. -
targeting/signalstore/signalstore.go:376—len(out) > maxKeysPerPlanchecks after the append. Slice transiently holdscap+1entries before erroring.if len(out)+1 > maxKeysPerPlanis tighter.
Approving on the strength of the three-layer identity firewall plus the spec-anchored property_rid fix.
PlanLookup aborted the whole request's key plan on the first cfg that returned ErrCfgUnsafe, so one malformed cfg in one package fail-closed every signal-gated package in the request. PlanLookup now skips an unsafe cfg's keys instead of aborting; the package owning it still fails closed independently at match time, when MatchProfile re-runs ExpandKeys. The only hard abort left is the request-wide key cap, which is a request-level DoS backstop not attributable to a single package. MatchProfile now wraps a cfg error with its any_of/none_of index and signal_owner_id so the engine's fail-closed log pinpoints the offending cfg without a transcript. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Approving. The three-layer privacy boundary on signalstore composes correctly under adversarial review, and property_rid as UUID-v7 string is independently anchored to the schema-generated tmproto/types_gen.go:119 — the uint64 was one-sided drift, not a representational choice.
Things I checked
targeting/signalstore/signalstore.go:228-270— cartesian indices clean; the three caps compose correctly (per-cfg 4096, request-wide 65536, per-profile 256).signalstore.go:317-339+engine_signals.go:268-300— thePlanLookupskip →MatchProfilere-expand →signalsPassfail-closed chain holds. The recent isolate-unsafe-cfg fix (3f61ca2) preserves per-package isolation: skipped at plan time, re-evaluated and rejected at match time whenMatchProfilere-runsExpandKeys.engine_signals.go:36-92—extractSignalLookupDatais the sole producer ofLookupDataand surfaces only context attributes. TheappendUniquecomma-drop (L105-L115) blocks the publisher-controlledcountry=US,eidshadow walk.registry/property_index.go— uint64→string switch consistent across Put/Remove/Hydrate.byRIDinsertion now guards on non-empty RID, replacing theuint64==0sentinel.ad-tech-protocol-expertconfirmed the AdCP property registry catalog spec shipsproperty_ridas UUID-v7 string pertmproto/types_gen.go:119— the schema-generated type carries the inline comment "Property catalog UUID (UUID v7)."cmd/context-agent/registry_setup.go:157-180—runSyncerpanic recovery + atomicsyncErrordering.closeRedisis reached on every error-return path inbuildRegistry.Shutdownis idempotent.- Conventional-commit story: six commits, the
feat:headline is correct. Theregistry/targetingGo-module breaks are not under release-please (release-please-config.jsontracks onlyadcp), so the missing!markers do not trip MUST FIX #6. PR body calls this out explicitly.
Follow-ups (non-blocking — file as issues)
signalstore.Cfg.ExpandKeysdefense-in-depth gap on empty owner / signal id.Cfg.Validaterejects emptySignalOwnerIDandSignalID, butExpandKeys/Matchesdo not re-check.Key()returns "" on empty owner, whichMGetsilently skips — anone_ofcfg with an empty owner that bypassesValidate(operator pokes Valkey directly, future writer code path skipsService.Put) fails OPEN. One-line guard at the top ofExpandKeys.targeting/signalstore/signalstore.go:228.SignalOwnerIDaccepts\":\". No symmetric check for the keyspace delimiter on the owner string. A governance-pipeline writer could landSignalOwnerID = \"7:url_hash:HASH_A:custom\"and shadow another owner's keyspace under split-parse. Reject\":\"and\",\"inCfg.Validate.signalstore.go:118-134.KeyTypeCustomis the privacy boundary's load-bearing convention, not a code-enforced rule. The three-layer story works becauseeid/email/coreidare named in the rejection list.customis the catch-all the writer can populate with anything — by convention identity material stays out, but nothing in this PR enforces it. Architecturally, a privacy boundary enforced by name-match against a fixed list with one catch-all entry next to it is worth a note. Either remove fromAllowedKeyTypesor add a normative writer-side contract test.signalstore.go:56./livemasks a misconfigured feed for 5 minutes after startup.lastSuccessis seeded withtime.Now().Unix()atcmd/context-agent/registry_setup.go:127before any poll has run. A deploy with a badFeedURL/FeedTokenreports healthy through the entireregistrySyncStaleThreshold = 5 * time.Minutewindow. Seed to0(or sentinel) and requirelastSuccess > 0 && age < thresholdin the liveness check.targeting.ContextStorageinterface break needs a CHANGELOG note. Adding a requiredSignalMGetbreaks any external embedder of the publictargetingpackage. Release-please tracks onlyadcp, so not a release-please block — but the existingcontextConfigBatcherextension pattern atengine_signals.go:211-213is the shape this should have followed for backwards compat. PR body already flags it; add a CHANGELOG entry covering thetargetingmodule break.signalstorekeyspace shape is implementation-defined, not AdCP-spec. The package doc reads like a contract; spell out that the contract is with this repo's writer, not with the protocol. The54cff92byte-compat claim onSignalOwnerID(uint64→string) is load-bearing only against the in-repo writer.signalstore.go:4-12.registry/property_index.goPutdoes not detect RID collision across distinctproperty_ids. APutwhose newPropertyRIDalready maps to a differentPropertyIDoverwritesbyRID[RID]without clearing the prior owner'sbyIDentry, leaving the two indexes out of sync. Pre-existing — not introduced here — but with strings the collision surface is larger than the old catalog-assigned uint64 space. Detect and either reject or evict the priorbyIDentry.property_index.go:78-84.
Minor nits (non-blocking)
pkgconfigstore.MGetdocstring promises a per-key log the code does not emit. The comment attargeting/pkgconfigstore/pkgconfigstore.go:150-156says the reader logs the package id on decode error; the code justcontinues with no logger plumbed through. Either wire a logger throughNewReaderor drop the claim from the comment.signalsPassdiscardsfetchErrafter fail-closing the package.targeting/engine_signals.go:281-284. The request-wide error was already recorded viaStoreErrorat the fetch site, but a per-package debug line would help operators chase empty-offer responses to the right cause.signalstore.go:243-246cap check after multiply. Not exploitable at current bounds (4096 × 4096 stays well inside int64), tidier to checktotal > maxKeysPerCfgbefore the nexttotal *= len(vals).
Approving on the strength of the three-layer privacy composition holding against publisher-controlled input plus the property_rid UUID fix being independently anchored to the schema-generated tmproto/types_gen.go:119.
What this does
Brings the production
cmd/context-agentcloser to feature-complete against thecontext-match part of the TMP spec, in two parts, plus a spec-conformance fix to
the
registrypackage that the first part depends on.1. Registry-fed property bitmap (opt-in)
With
REGISTRY_ENABLED=truethe agent runs a background registry sync loop andfeeds the engine's global property bitmap from a live
registry.PropertyIndexinstead of the static
PROPERTY_RIDSenv stopgap (which remains the fallback).memoryorredis/Valkey (viaregistry/redisstore); optional file-backed cursor./livegains aregistry_syncstaleness check, driven by a newSyncerConfig.OnSuccessfulPollbeacon so a quiescent-but-healthy feed still proves liveness.2. Context-attribute signal targeting
PackageContextConfiggains aContextSignalsprofile (any_of/none_of)evaluated against the
signal:{signal_owner_id}:{key_types}:{values}keyspace in asingle deduped MGet per request.
url_hash.pkgconfigstoregains a batchedMGetso candidate configs load in one round-trip.3.⚠️ public-type change
registry.Property.PropertyRID:uint64→string(spec fix)The AdCP property registry catalog spec defines
property_ridas a catalog-assignedUUID v7 ("the shared key for TMP matching"), and the change-feed
property.createdpayload carries it as a string. The
registrypackage modeled it asuint64, so aspec-compliant feed event failed to unmarshal (
string→uint64) and was dropped —the registry-fed bitmap from part 1 could never match an inbound UUID
property_ridand served empty offers for every spec-compliant request.
This changes
Property.PropertyRIDtostringand the index's RID dimension with it(
LookupByRID(string),PropertyRID() string,byRID map[string]*Property). Thecontext-agent bitmap adapter now matches on
property_ridonly (not theproperty_idslug), per spec. The router's separate string-typed registry (
router/registry.go)was already spec-correct and is untouched.
Scope notes
config:pkg:*andsignal:*is out of scope — reader/engine side only.replacedirectives in thecmd/context-agentsubmodule.Testing
go test ./...passes in the root,registry, andcmd/context-agentmodules(
registry/redisstore+glidestorevet clean).Known pre-existing failure
registry/TestSyncer_AppliesAuthorizationRevokedfails on this branch, but italso fails on
mainwith this PR'sregistry/syncer.gochange reverted — notintroduced here.