Skip to content

DB pool/DSN unconfigured: silent SQLITE_BUSY (no busy_timeout, no SetMaxOpenConns) #7

Description

@fahimfaisaal

Summary

The *sql.DB is created with no pool or busy-timeout configuration. With WAL and concurrent writers this yields silent SQLITE_BUSY failures surfaced as false/no-op returns.

Root cause

queues.go:18-33:

db, err := sql.Open("sqlite3", dbPath)
db.Exec("PRAGMA journal_mode=WAL;")
  • SetMaxOpenConns is not set, so database/sql opens multiple connections and lets multiple goroutines attempt concurrent writes.
  • No busy_timeout (DSN _busy_timeout or PRAGMA busy_timeout), so a blocked writer errors immediately instead of waiting.

SQLite allows only one writer at a time; WAL only relaxes reader/writer blocking, not writer/writer.

Impact

Medium. Write operations (Enqueue, dequeue upgrades, Acknowledge) intermittently fail and return false with no error signal, especially under the varmq worker pool this adapter targets.

Possible fix

Cheapest robust option — serialize writers and add a wait:

db, err := sql.Open("sqlite3", dbPath+"?_busy_timeout=5000&_journal_mode=WAL&_txlock=immediate")
if err != nil { ... }
db.SetMaxOpenConns(1) // single writer; SQLite is not a concurrent-write store

SetMaxOpenConns(1) removes writer contention entirely at the cost of write throughput (acceptable for a SQLite-backed queue). If read concurrency matters, keep a higher limit but rely on busy_timeout + BEGIN IMMEDIATE. Setting PRAGMAs via the DSN also guarantees they apply to every pooled connection, not just the first.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions