Skip to content

fix(server): make feature-request reads idempotent (ARN-240) - #396

Draft
rita-aga wants to merge 7 commits into
mainfrom
grok/arn-240-feature-request-get
Draft

fix(server): make feature-request reads idempotent (ARN-240)#396
rita-aga wants to merge 7 commits into
mainfrom
grok/arn-240-feature-request-get

Conversation

@rita-aga

@rita-aga rita-aga commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Grok arena ARN-240. ADR-0175. No merge.

Greptile Summary

This PR fixes a long-standing violation of HTTP idempotency by removing all write-on-read side-effects from GET /observe/evolution/feature-requests and moving materialization exclusively to the sentinel write path. It introduces a SHA-256 content-addressed stable identity for feature requests (tenant + generator version + evidence), an idempotency key on the CreateFeatureRequest dispatch, and a legacy reconciliation pass that migrates old FR-YYYY-* rows to canonical stable IDs while preserving human review state.

  • Pure-read GET: handle_feature_requests is now a bounded query; trajectory loading, upsert, and entity creation are fully removed from that handler.
  • Idempotent sentinel write: materialize_feature_requests computes a deterministic stable_feature_request_id, dispatches CreateFeatureRequest with an idempotency key, and calls reconcile_legacy_feature_requests to merge legacy projections into the canonical row.
  • Store changes: upsert_feature_request (Turso + Postgres) now preserves disposition and developer_notes on conflict; delete_feature_request is added to both stores and the EvolutionStore trait.

Confidence Score: 5/5

Safe to merge; the GET handler is now a pure read, the sentinel write path converges on a single canonical row under concurrent and retry conditions, and upserts preserve human review state on conflict.

The core idempotency invariants are well-tested across concurrent requests, restarts, legacy reconciliation, note deduplication, and multi-tenant disambiguation. Store-layer changes are straightforward and mirrored across both Turso and Postgres. The only discrepancy found is a documentation error in the ADR hash formula with no runtime impact.

docs/adrs/0175-feature-request-get-idempotency.md — hash formula lists frequency but the implementation omits it.

Important Files Changed

Filename Overview
crates/temper-server/src/observe/evolution/operations.rs Core change: removes side-effects from GET, adds stable_feature_request_id hash function, and materialize_feature_requests pipeline. The category string stored in the DB (format!("{:?}")) differs from the category string in the hash (match-arm constant), which is intentional. The hash payload omits frequency even though the ADR formula lists it.
crates/temper-server/src/observe/evolution/operations/reconcile.rs New reconciliation module: migrates legacy FR-YYYY-* projections to canonical stable IDs, merges review state, and deduplicates developer notes. Logic is well-guarded with tenant-disambiguation and evidence-revision matching.
crates/temper-server/src/observe/evolution/insight_generator/gap_analysis.rs Adds deterministic description selection (lex-min error across group entries) and timestamp sorting to make feature-request generation order-independent — necessary for stable hash.
crates/temper-store-turso/src/store/evolution.rs Adds delete_feature_request; upsert ON CONFLICT clause now preserves disposition and developer_notes — correct and necessary for idempotent materialization.
crates/temper-store-postgres/src/platform.rs Mirrors Turso changes: ON CONFLICT clause updated to preserve human review state; delete_feature_request added.
crates/temper-server/src/observe/mod_test.rs Comprehensive new test suite covering pure-read GET, idempotent sentinel materialization, legacy reconciliation, note deduplication, tenant-disambiguation of ambiguous legacy rows, and restart replay — good coverage of the new invariants.
docs/adrs/0175-feature-request-get-idempotency.md ADR is well-written and matches the implementation intent, but the documented hash formula lists frequency as an explicit field while the code omits it.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Client
    participant GET as GET /feature-requests
    participant POST as POST /sentinel/check
    participant Gen as generate_feature_requests
    participant Hash as stable_feature_request_id
    participant IOA as dispatch_system_action_idempotent
    participant Store as MetadataStore
    participant Rec as reconcile_legacy_feature_requests

    Client->>GET: GET /observe/evolution/feature-requests
    GET->>Store: list_feature_requests(filter)
    Store-->>GET: rows[]
    GET-->>Client: 200 OK (pure read, no writes)

    Client->>POST: POST /api/evolution/sentinel/check
    POST->>Gen: generate_feature_requests(trajectory_entries)
    Gen-->>POST: [FeatureRequestRecord, ...]
    POST->>Store: list_feature_requests(None) — snapshot for reconcile
    Store-->>POST: existing_rows[]
    loop per feature_request
        POST->>Hash: stable_feature_request_id(tenant, "v1", request)
        Hash-->>POST: "FR-{sha256}"
        POST->>IOA: "dispatch CreateFeatureRequest (idempotency_key=feature-request:{id})"
        IOA-->>POST: EntityResponse (no-op if already exists)
        POST->>Store: upsert_feature_request(stable_id, …) — preserves disposition/notes
        POST->>Rec: reconcile_legacy_feature_requests(existing_rows, stable_id)
        Rec->>Store: update_feature_request(stable_id, merged_state)
        Rec->>Store: delete_feature_request(legacy_id) x N
    end
    POST-->>Client: "200 OK {feature_request_ids: [...]}"
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Client
    participant GET as GET /feature-requests
    participant POST as POST /sentinel/check
    participant Gen as generate_feature_requests
    participant Hash as stable_feature_request_id
    participant IOA as dispatch_system_action_idempotent
    participant Store as MetadataStore
    participant Rec as reconcile_legacy_feature_requests

    Client->>GET: GET /observe/evolution/feature-requests
    GET->>Store: list_feature_requests(filter)
    Store-->>GET: rows[]
    GET-->>Client: 200 OK (pure read, no writes)

    Client->>POST: POST /api/evolution/sentinel/check
    POST->>Gen: generate_feature_requests(trajectory_entries)
    Gen-->>POST: [FeatureRequestRecord, ...]
    POST->>Store: list_feature_requests(None) — snapshot for reconcile
    Store-->>POST: existing_rows[]
    loop per feature_request
        POST->>Hash: stable_feature_request_id(tenant, "v1", request)
        Hash-->>POST: "FR-{sha256}"
        POST->>IOA: "dispatch CreateFeatureRequest (idempotency_key=feature-request:{id})"
        IOA-->>POST: EntityResponse (no-op if already exists)
        POST->>Store: upsert_feature_request(stable_id, …) — preserves disposition/notes
        POST->>Rec: reconcile_legacy_feature_requests(existing_rows, stable_id)
        Rec->>Store: update_feature_request(stable_id, merged_state)
        Rec->>Store: delete_feature_request(legacy_id) x N
    end
    POST-->>Client: "200 OK {feature_request_ids: [...]}"
Loading

Reviews (2): Last reviewed commit: "fix(server): stable feature-request cate..." | Re-trigger Greptile

@rita-aga

Copy link
Copy Markdown
Collaborator Author

Grok independent review — PR #396 (ARN-240 / ADR-0175)

Scope: Make feature-request GET a pure read; materialize on the evolution write path with stable content-addressed ids; preserve human review on upsert; tenant-safe legacy projection reconciliation.

What was done well

  • Fixes a real HTTP contract bug: GET was a writer (regenerate + upsert projection + next_system_entity_id("FR") entity create). Polling/Observe refresh correctly stopped materializing duplicates.
  • Materialization moved to POST .../sentinel/check with returned feature_request_ids — explicit write path, observable result.
  • Stable id FR- + SHA-256 over tenant + generator_version + category + description + frequency + sorted trajectory_refs; idempotency key feature-request:{stable_id} on system dispatch; generator version bump creates a new identity by design.
  • Generator determinism fixes (min error description, sorted timestamps) support stable identity across reorder replay — covered by unit test.
  • Projection upsert stops clobbering disposition / developer_notes on conflict (Postgres + Turso) while still setting them on insert — correct human-owned fields.
  • Legacy reconcile is conservative: FR-YYYY-hex shape only, evidence revision match, unambiguous single-tenant trajectory ownership; notes merge preserves canonical bytes and dedupes components. Ambiguous multi-tenant rows left alone — good tenancy posture.
  • EvolutionStore extracted; delete_feature_request added for obsolete legacy rows. Tests cover pure GET (incl. concurrent), sentinel idempotency, note merge, tenant isolation, restart retry.

Findings

Important (should fix)

  1. Evidence growth creates new stable identities (orphan accumulation)
    Identity includes frequency and full trajectory_refs. As more matching trajectories arrive (or the loaded window shifts), sentinel rematerialization yields a new FR-<hash> while prior stable rows/entities remain. Reconcile only folds legacy FR-YYYY-<12 hex> rows with identical evidence revision — it does not merge prior stable revisions of the same logical gap.
    Effect: dual-store growth and Observe clutter over time, partially reintroducing the “many identities for one gap” pain the GET bug caused, just on the write path.
    Prefer identity over stable gap keys (tenant + generator_version + category + description prefix / error pattern) and treat frequency/refs as updatable generated fields on that id; or reconcile/supersede prior stable ids when category+description match. At minimum document GC / supersession as a follow-up before leaning on this in production Observe.

  2. serde_json::Value::to_string() as hash preimage
    json!({...}).to_string() is insertion-order stable for this macro shape today, but is a subtle contract. Hashing an explicit canonical encoding (sorted keys, length-prefixed fields, or a small hand-built string) would make the preimage reviewable and less coupled to serde_json formatting quirks.

Suggestions (nice to have)

  1. format!("{:?}", category) in identity and rows couples durable ids to Debug spelling of the enum. A stable string discriminant would survive renames without a forced generator_version bump.

  2. Materialize still dual-writes entity + metadata projection (acknowledged Phase 2 / ARN-218). Fine for this PR; keep GET read-only if the worker path moves later.

  3. Reconcile uses a snapshot of existing_rows loaded once before the generated loop — fine at current scale; concurrent human edits mid-loop are rare. Refresh-after-upsert is optional hardening.

Plan alignment

Matches ADR-0175 sub-decisions 1–5 for the GET purity bug and legacy upgrade path. The largest open product risk is stable-id churn under growing evidence, which the ADR frames as intentional “evidence revision” but does not solve for orphan prior revisions.

Verdict

GET purity, review-preserving upsert, tenant-safe legacy reconcile, and retry/idempotency tests are solid and shippable for the stated ARN-240 bug. Address or explicitly track identity churn / supersession so materialization does not become a slow leak of FR rows.

Verdict: PASS

@rita-aga

Copy link
Copy Markdown
Collaborator Author

@greptile review

@rita-aga

Copy link
Copy Markdown
Collaborator Author

ARENA SHIPPABLE · Grok · 2026-07-14 11:57 PDT

PR: #396
HEAD: e7cf9eb35555 · branch grok/arn-240-feature-request-get
Linear: ARN-240 · ADR: 0175
Merge: nothing (arena rules)

Checklist

Gate Evidence
RED→GREEN on branch history
Independent same-model review Verdict: PASS posted on PR
Greptile requested after PASS
Local tests targeted suite green before push
CI GitHub Actions on head

Summary

feature-request GET idempotency

Comment thread crates/temper-server/src/observe/evolution/operations.rs Outdated
Comment thread crates/temper-server/src/observe/evolution/operations.rs
Comment on lines +173 to +196
let existing_rows = store
.list_feature_requests(None)
.await
.map_err(|error| {
tracing::error!(error = %error, backend = store.backend_name(), "failed to load feature requests for reconciliation");
StatusCode::INTERNAL_SERVER_ERROR
})?;
let mut materialized_ids = Vec::with_capacity(generated.len());

for feature_request in &generated {
let stable_id =
stable_feature_request_id(tenant, FEATURE_REQUEST_GENERATOR_VERSION, feature_request);
let category = format!("{:?}", feature_request.category);
let refs_json = serde_json::to_string(&feature_request.trajectory_refs)
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let disposition = match feature_request.disposition {
FeatureRequestDisposition::Open => "Open",
FeatureRequestDisposition::Acknowledged => "Acknowledged",
FeatureRequestDisposition::Planned => "Planned",
FeatureRequestDisposition::WontFix => "WontFix",
FeatureRequestDisposition::Resolved => "Resolved",
};
let params = serde_json::json!({
"category": category,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 existing_rows snapshot is stale during the multi-feature-request loop

existing_rows is loaded once before the loop, then passed by reference to every reconcile_legacy_feature_requests call inside it. For a first-time stable row insert the canonical_notes path in merged_review_state is always None (the stable row wasn't in the pre-loop snapshot), which is correct but non-obvious. A comment explaining the invariant — that is_same_evidence_revision ensures disjoint legacy-row sets per generated feature request, making the stale snapshot safe — would make the intent explicit.

Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/temper-server/src/observe/evolution/operations.rs
Line: 173-196

Comment:
**`existing_rows` snapshot is stale during the multi-feature-request loop**

`existing_rows` is loaded once before the loop, then passed by reference to every `reconcile_legacy_feature_requests` call inside it. For a first-time stable row insert the `canonical_notes` path in `merged_review_state` is always `None` (the stable row wasn't in the pre-loop snapshot), which is correct but non-obvious. A comment explaining the invariant — that `is_same_evidence_revision` ensures disjoint legacy-row sets per generated feature request, making the stale snapshot safe — would make the intent explicit.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Codex Fix in Cursor

@rita-aga

Copy link
Copy Markdown
Collaborator Author

@greptile review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant