Skip to content

fix(streams): make StreamQueue.record! independent of Rails schema-cache index resolution - #402

Merged
mhenrixon merged 2 commits into
mainfrom
issue-401-stream-queue-record-schema-cache
Aug 14, 2026
Merged

fix(streams): make StreamQueue.record! independent of Rails schema-cache index resolution#402
mhenrixon merged 2 commits into
mainfrom
issue-401-stream-queue-record-schema-cache

Conversation

@mhenrixon

@mhenrixon mhenrixon commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Pgbus::StreamQueue.record! (app/models/pgbus/stream_queue.rb) no longer uses upsert(unique_by: :queue_name) — it writes via raw INSERT INTO … (queue_name) VALUES (…) ON CONFLICT (queue_name) DO NOTHING on the model's connection. The unique index is owned by the gem's own migration, so there is nothing for Rails to resolve, and the pool schema cache's permanently-latched negative data_source_exists? probe can no longer poison registration for the process lifetime.
  • Failure logging is class-aware: transient database errors (ActiveRecord::ActiveRecordError) log at DEBUG per attempt as before; any other StandardError — the bug-signal class the old path's ArgumentError belonged to — logs at WARN once per process, DEBUG thereafter.
  • Integration harness now bootstraps pgbus_stream_queues (mirroring the generator template) so the registry is exercised for real, and a new spec/integration/stream_queue_registry_spec.rb pins the fix.

Closes #401

Acceptance criteria → proof

Criterion Proof
record! succeeds with a deliberately poisoned pool schema cache spec/integration/stream_queue_registry_spec.rb poisons @data_sources["pgbus_stream_queues"] = false exactly as the issue's repro; verified RED against the old implementation (fails) and GREEN with the fix
No schema-cache indexes/data_source_exists? query from record! sql.active_record subscription asserts zero SCHEMA-named events during record! once the table_exists? memo is warm
Resolution-class failure logs at WARN at most once per process Unit specs: ArgumentError → one WARN then DEBUG, across different queue names; StatementInvalid → DEBUG only, never WARN
backfill! / all_names / cache-consistency semantics unchanged Existing specs untouched and green (warm/cold cache, backfill counting, return values)

Test plan

  • bundle exec rspec spec/pgbus/ spec/generators/ — 3866 examples, 0 failures
  • bundle exec rake rubocop — 553 files, no offenses
  • spec/integration/stream_queue_registry_spec.rb — poisoned-cache regression fails on old code, passes on new
  • Full integration suite against local PG

Deviations & judgment calls

  • Issue point 3 (move the registry write off the first-broadcast hot path) not implemented — it's phrased as "consider", the raw-SQL fix already removes all schema-cache/InsertAll traffic from the flush thread, and the write is already once-per-stream-per-process via the @stream_indexes_created memo. Deferral would change ensure_stream_queue semantics and belongs in its own PR if still wanted.
  • Issue point 4 (first-broadcast hang audit) is explicitly a separate follow-up per the issue — out of scope here.
  • Transient-vs-bug error split implemented as ActiveRecord::ActiveRecordError → DEBUG (covers StatementInvalid and connection errors like ConnectionNotEstablished, which are equally transient), any other StandardError → WARN once per process then DEBUG. The issue said "non-StatementInvalid", but that literal reading would WARN on transient connection drops.
  • Used connection.quote/quote_table_name interpolation instead of the issue's sanitize_sql([… ?]) — in Rails 8.1 sanitize_sql_array checks out a pool connection via with_connection, which can't be stubbed in the DB-free unit suite; connection-level quoting is equivalent and testable through the same connection seam the spec file already stubs.

Summary by cubic

Makes Pgbus::StreamQueue.record! write via raw INSERT…ON CONFLICT instead of upsert(unique_by: :queue_name), removing dependence on Rails’ pool schema cache and preventing process-lifetime failures after a false negative probe.

  • No schema-cache index resolution or SCHEMA traffic on the hot path: with a warm table_exists? memo, record! issues a single INSERT…ON CONFLICT DO NOTHING.
  • Error handling: ActiveRecord::ActiveRecordError logs at DEBUG per attempt; any other StandardError logs at WARN once per process (latch serialized via a class-level mutex), then DEBUG.
  • Idempotency, return values, and backfill!/all_names cache semantics are unchanged.
  • Tests: adds an integration spec that poisons the pool schema cache and still registers the stream and asserts zero SCHEMA queries; integration harness bootstraps pgbus_stream_queues.

Written for commit cc3a2c3. Summary will update on new commits.

Review in cubic

…che index resolution

## Summary
record! used upsert(unique_by: :queue_name), which resolves the unique
index through the pool's schema cache. That cache latches a negative
data_source_exists? probe permanently while the table_exists? guard is a
live query — so one wrong first probe poisoned every subsequent record!
in the process ("No unique index found for queue_name", swallowed at
DEBUG) until restart. The registry write is now a raw
INSERT ... ON CONFLICT (queue_name) DO NOTHING on the model connection:
the gem's own migration owns the index, so there is nothing to resolve
and no schema-cache traffic leaves the first-broadcast hot path.

Failure logging is class-aware: ActiveRecordError stays DEBUG per
attempt; any other StandardError (the bug-signal class) logs at WARN
once per process, DEBUG thereafter.

## Test Coverage
- unit: raw INSERT SQL, no upsert machinery, WARN-once vs DEBUG split,
  unchanged return/cache semantics
- integration: poisoned pool schema cache still registers (fails on the
  old implementation), zero SCHEMA sql.active_record events with a warm
  table_exists? memo, ON CONFLICT idempotency; integration harness now
  bootstraps pgbus_stream_queues

## Verification
- [x] bundle exec rake rubocop passes (553 files)
- [x] bundle exec rspec spec/pgbus/ spec/generators/ — 3866 examples, 0 failures
- [x] integration suite — 196 examples, 0 failures

Refs #401

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 5 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread app/models/pgbus/stream_queue.rb Outdated
Comment thread spec/integration/stream_queue_registry_spec.rb
… warm the guard memo explicitly

Review follow-ups (PR #402):
- log_record_failure's check-and-set now runs under a class-level mutex
  so concurrent failures (coalescer flush thread + callers) cannot emit
  two WARNs; logging happens outside the critical section.
- The poisoned-cache regression spec warms and asserts the table_exists?
  memo before poisoning, making the index-resolution target explicit
  rather than a side effect of the live-query guard.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@mhenrixon
mhenrixon merged commit 9941a26 into main Aug 14, 2026
13 checks passed
@mhenrixon
mhenrixon deleted the issue-401-stream-queue-record-schema-cache branch August 14, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant