Windows: concurrent WAL writes across pooled connections corrupt the database

[franchb]

Summary

On Windows, using a single *sql.DB with more than one open connection
(SetMaxOpenConns(N>1)) against a WAL database reliably corrupts the
database file under a concurrent write workload. The writers use ordinary
BEGIN DEFERRED transactions (so they serialize on the WAL write lock via
busy_timeout) — no BEGIN CONCURRENT, nothing exotic.

The same program is clean on Linux and clean as native C SQLite (see
Observations), which points at the Go Windows VFS rather than SQLite itself.

Environment

• OS: Windows Server 2022 (build 10.0.20348), amd64, 4 vCPU
• Go: go1.26.4
• github.com/ncruces/go-sqlite3 v0.35.1
• github.com/ncruces/go-sqlite3-wasm/v3 v3.1.35302 (indirect)

Reproducer

A single *sql.DB, SetMaxOpenConns(64), WAL, _txlock=deferred, 64 goroutines
each doing parent+child inserts in a transaction, with an occasional
wal_checkpoint(TRUNCATE); then close, cold-reopen, PRAGMA integrity_check.

go.mod:

module walrepro

go 1.26

require github.com/ncruces/go-sqlite3 v0.35.1

main.go:

package main

import (
        "context"
        "crypto/rand"
        "database/sql"
        "fmt"
        "os"
        "path/filepath"
        "sync"

        "github.com/ncruces/go-sqlite3/driver"
)

const (
        workers   = 64
        iters     = 2000
        blobBytes = 8192
        ckptEvery = 25
)

const schema = `
CREATE TABLE parent(id INTEGER PRIMARY KEY AUTOINCREMENT, hash TEXT NOT NULL UNIQUE);
CREATE TABLE child(id INTEGER PRIMARY KEY AUTOINCREMENT,
    parent_id INTEGER NOT NULL REFERENCES parent(id) ON DELETE CASCADE, data BLOB NOT NULL);
CREATE INDEX child_parent_idx ON child(parent_id);
`

func dsn(path string) string {
        return "file:" + path +
                "?_pragma=busy_timeout(10000)&_pragma=journal_mode(wal)&_pragma=synchronous(normal)&_txlock=deferred"
}

func main() {
        dir, err := os.MkdirTemp("", "walrepro")
        must(err)
        defer os.RemoveAll(dir)
        path := filepath.Join(dir, "repro.db")

        db, err := driver.Open(dsn(path))
        must(err)
        db.SetMaxOpenConns(workers)
        _, err = db.Exec(schema)
        must(err)

        blob := make([]byte, blobBytes)
        _, _ = rand.Read(blob)

        var wg sync.WaitGroup
        errCh := make(chan error, workers)
        for w := 0; w < workers; w++ {
                wg.Add(1)
                go func(gid int) {
                        defer wg.Done()
                        for i := 0; i < iters; i++ {
                                if err := doTx(db, gid, i, blob); err != nil {
                                        errCh <- fmt.Errorf("worker %d iter %d: %w", gid, i, err)
                                        return
                                }
                                if i%ckptEvery == 0 {
                                        if _, err := db.Exec("PRAGMA wal_checkpoint(TRUNCATE)"); err != nil {
                                                errCh <- fmt.Errorf("checkpoint: %w", err)
                                                return
                                        }
                                }
                        }
                }(w)
        }
        wg.Wait()
        close(errCh)
        workErr := <-errCh // first error, if any (nil when none)

        must(db.Close())

        db2, err := driver.Open(dsn(path))
        if err != nil {
                fmt.Printf("RESULT=CORRUPT reopen: %v (workErr=%v)\n", err, workErr)
                os.Exit(1)
        }
        defer db2.Close()
        db2.SetMaxOpenConns(1)
        var first string
        if err := db2.QueryRow("PRAGMA integrity_check").Scan(&first); err != nil {
                fmt.Printf("RESULT=CORRUPT integrity_check: %v (workErr=%v)\n", err, workErr)
                os.Exit(1)
        }
        if first != "ok" {
                fmt.Printf("RESULT=CORRUPT integrity_check=%q (workErr=%v)\n", first, workErr)
                os.Exit(1)
        }
        if workErr != nil {
                fmt.Printf("RESULT=CORRUPT workErr=%v\n", workErr)
                os.Exit(1)
        }
        fmt.Println("RESULT=CLEAN")
}

func doTx(db *sql.DB, gid, i int, blob []byte) error {
        ctx := context.Background()
        tx, err := db.BeginTx(ctx, nil)
        if err != nil {
                return err
        }
        var pid int64
        if err := tx.QueryRowContext(ctx,
                "INSERT INTO parent(hash) VALUES(?) RETURNING id",
                fmt.Sprintf("%d-%d", gid, i)).Scan(&pid); err != nil {
                _ = tx.Rollback()
                return err
        }
        if _, err := tx.ExecContext(ctx,
                "INSERT INTO child(parent_id, data) VALUES(?, ?)", pid, blob); err != nil {
                _ = tx.Rollback()
                return err
        }
        return tx.Commit()
}

func must(err error) {
        if err != nil {
                panic(err)
        }
}

Run:

go mod tidy
go run .

(A clean run writes ~1 GiB to the temp dir before cleanup; point TMPDIR/%TEMP%
at a volume with room.)

Expected vs actual

• Expected: RESULT=CLEAN (this is a normal multi-connection WAL usage pattern).
• Actual on Windows: RESULT=CORRUPT on every run (3/3 here). The symptom
  varies run to run, all consistent with on-disk corruption:
  • sqlite3: file is not a database
  • sqlite3: database disk image is malformed
  • mid-workload SQLITE_PROTOCOL ("locking protocol")
  • integrity_check reports b-tree/index damage, e.g.:

    wrong # of entries in index child_parent_idx
    Tree 5 page 18419 cell 2: overflow list length is 1 but should be 2
    Tree 5 page 18402: btreeInitPage() returns error code 11
    Tree 2 page 16435 cell 138: Rowid 8912 out of order
    Page 7556: never used
    ...
    

• Linux: RESULT=CLEAN on the same code (control).

Observations / narrowing

• Requires multiple concurrent connections. The trigger is SetMaxOpenConns(>1)
  writing to the same WAL DB; reducing the pool/worker count lowers the rate.
• Not WAL2 / BEGIN CONCURRENT. Plain journal_mode(wal) + BEGIN DEFERRED.
• Not the checkpoint. It also reproduces with the interleaved
  wal_checkpoint(TRUNCATE) removed.
• Timing-sensitive. Higher core counts / more workers raise the reproduction rate.
• Native C SQLite is clean. As a control I compiled the same SQLite version
  (3.53.2) as native C — using SQLite's own os_win.c VFS — and ran an equivalent
  64-thread BEGIN DEFERRED + plain-WAL + checkpoint workload on the same machine.
  It completes with integrity_check = ok every time. Combined with the Linux
  result, this suggests the defect is in the Go Windows VFS layer rather than
  SQLite core.

Suspected area

Given the above, the wal-index (-shm) coordination and/or file locking in
vfs/os_windows.go (LockFileEx/UnlockFileEx) and vfs/shm_windows.go
(memory-mapped -shm) look like the likely culprits — the shared cross-connection
state that native os_win.c handles but the Go port appears to mishandle under
concurrency on Windows. This may be related to earlier Windows-WAL work
(e.g. #200, #252, #253, #254).

Happy to test patches, try build tags, or provide additional traces / captured
corrupt databases

[ncruces]

OK, looking into this.

I don't have ready access to a Windows system, but (unfortunately) I can apparently repro with sqlite3_dotlk.

If it can't be fixed, it's possible the entire approach is hopeless.

But I need to figure out a way to iterate faster ... this test takes far too long.

[franchb]

Interesting that it repros with sqlite3_dotlk - that at least means real Windows isn't required to iterate on it.
On my side: I've set up a local Windows VM (Incus on Fedora), so I can now test on actual Windows with fast turnaround. I'm going to point an automated agent loop (Claude Code) at the reproducer to try to

• (a) shrink it into a faster/minimal failing case
• (b) narrow down where the corruption is introduced (shm coordination vs. LockFileEx paths).

I'll post anything useful I find.

In the meantime, happy to run any patches or build tags against the VM — turnaround should be quick now.

[ncruces]

OK, maybe I need to partially walk back my previous comment: I don't think SQLITE_PROTOCOL or SQLITE_BUSY automatically mean the database is corrupted.

Which is not to say this isn't something that needs to improve. But "data corruption" is worse than "denial of service."

And so far, that was all I was able to reproduce. So I definitely need to test Windows.

[ncruces]

And I don't mean to downplay the significance of the bug. From your report it seems corruption is definitely the result on Windows.

But Windows uses a ... weird mix of the scheme I made up for dotlk, with a specific implementation of memory mapping and file locks.

If the building blocks are sound, this raises the chances that it might be possible to fix things.

Also, for your specific use case, you may temporarily be able to use sqlite3_dotlk on Windows, at it supposedly works the same everywhere.

[ncruces]

For sqlite3_dotlk the problem also completely goes away with db.SetMaxOpenConns(runtime.GOMAXPROCS(0)), which I guess makes sense. This apparently holds even for a machine with 48 vCPUs.

So that's something I should document: the protocol for faking shared memory by copying memory around breaks down if you have enough write contention that readers are constantly preempted by writers. And even before it breaks down, performance drops dramatically.

I'm also pretty sure it similarly drops like a rock if the performance of copying memory drops (e.g. making it remote).

Still no clues about why you see "data corruption" on Windows. But it does mean that, if I'm able to reduce the problem to SQLITE_PROTOCOL or SQLITE_BUSY, I am happy.

The "data corruption" is the problem, the "denial-of-service" may just be something we have to live with, and that we should minimize by reducing contention. I hope you can agree with that.

[ncruces]

Can you test winfix, specifically f98c05b?

It's a wild guess, but maybe the locks added to shm_windows.go‎ help.

As I wrote above, I think SQLITE_PROTOCOL and SQLITE_BUSY are a non issue, especially if they disappear with db.SetMaxOpenConns(runtime.GOMAXPROCS(0)).

I've modified your test to stop at the first "actual" error, and log+retry for those two:

package main

import (
        "context"
        "crypto/rand"
        "database/sql"
        "errors"
        "fmt"
        "log"
        "os"
        "path/filepath"

        "golang.org/x/sync/errgroup"

        "github.com/ncruces/go-sqlite3"
        "github.com/ncruces/go-sqlite3/driver"
)

const (
        workers   = 64
        iters     = 2000
        blobBytes = 8192
        ckptEvery = 25
)

const schema = `
CREATE TABLE parent(id INTEGER PRIMARY KEY AUTOINCREMENT, hash TEXT NOT NULL UNIQUE);
CREATE TABLE child(id INTEGER PRIMARY KEY AUTOINCREMENT,
    parent_id INTEGER NOT NULL REFERENCES parent(id) ON DELETE CASCADE, data BLOB NOT NULL);
CREATE INDEX child_parent_idx ON child(parent_id);
`

func dsn(path string) string {
        return "file:" + path +
                "?_pragma=busy_timeout(10000)&_pragma=journal_mode(wal)&_pragma=synchronous(normal)&_txlock=deferred"
}

func main() {
        dir, err := os.MkdirTemp("", "walrepro")
        must(err)
        defer os.RemoveAll(dir)
        path := filepath.Join(dir, "repro.db")

        db, err := driver.Open(dsn(path))
        must(err)
        db.SetMaxOpenConns(workers)
        _, err = db.Exec(schema)
        must(err)

        blob := make([]byte, blobBytes)
        _, _ = rand.Read(blob)

        wg, ctx := errgroup.WithContext(context.Background())
        for gid := range workers {
                wg.Go(func() error {
                        for i := range iters {
                                if err := doTx(ctx, db, gid, i, blob); err != nil {
                                        code, _ := errors.AsType[sqlite3.ExtendedErrorCode](err)
                                        if errors.Is(err, sqlite3.PROTOCOL) || errors.Is(err, sqlite3.BUSY) {
                                                log.Printf("worker %d iter %d: %v (%d)", gid, i, err, code)
                                        } else {
                                                return fmt.Errorf("worker %d iter %d: %w", gid, i, err)
                                        }
                                }
                                if i%ckptEvery == 0 {
                                        if _, err := db.Exec("PRAGMA wal_checkpoint(TRUNCATE)"); err != nil {
                                                return fmt.Errorf("checkpoint: %w", err)
                                        }
                                }
                        }
                        return nil
                })
        }
        workErr := wg.Wait()

        must(db.Close())

        db2, err := driver.Open(dsn(path))
        if err != nil {
                fmt.Printf("RESULT=CORRUPT reopen: %v (workErr=%v)\n", err, workErr)
                os.Exit(1)
        }
        defer db2.Close()
        db2.SetMaxOpenConns(1)
        var first string
        if err := db2.QueryRow("PRAGMA integrity_check").Scan(&first); err != nil {
                fmt.Printf("RESULT=CORRUPT integrity_check: %v (workErr=%v)\n", err, workErr)
                os.Exit(1)
        }
        if first != "ok" {
                fmt.Printf("RESULT=CORRUPT integrity_check=%q (workErr=%v)\n", first, workErr)
                os.Exit(1)
        }
        if workErr != nil {
                fmt.Printf("RESULT=CORRUPT workErr=%v\n", workErr)
                os.Exit(1)
        }
        fmt.Println("RESULT=CLEAN")
}

func doTx(ctx context.Context, db *sql.DB, gid, i int, blob []byte) error {
        tx, err := db.BeginTx(ctx, nil)
        if err != nil {
                return err
        }
        var pid int64
        if err := tx.QueryRowContext(ctx,
                "INSERT INTO parent(hash) VALUES(?) RETURNING id",
                fmt.Sprintf("%d-%d", gid, i)).Scan(&pid); err != nil {
                _ = tx.Rollback()
                return err
        }
        if _, err := tx.ExecContext(ctx,
                "INSERT INTO child(parent_id, data) VALUES(?, ?)", pid, blob); err != nil {
                _ = tx.Rollback()
                return err
        }
        return tx.Commit()
}

func must(err error) {
        if err != nil {
                panic(err)
        }
}

[ncruces]

Tried it on a Windows laptop, couldn't repro the corruption.

I mean, only SQLITE_PROTOCOL or SQLITE_BUSY, and even those go away with db.SetMaxOpenConns(runtime.GOMAXPROCS(0)).

Not sure what else to do.

[ncruces]

Hi @franchb, any updates? I've rebased winfix, but I'm not confident 5441ed3 is a fix (it shouldn't be worse, though, that's for sure).

I'd like to release SQLite 3.53.3 and I'm not sure what to do.

Cutting another release with a known corruption issue is bad, but I haven't been able to reproduce corruption, only "denial of service" (i.e.: too much concurrency locks you out, but retrying eventually works).

[franchb]

Hi @ncruces — yes, update: I opened #405 with what I believe is a real fix. But first, the part that I think unblocks your release decision, because it's the same thing that had me confused for a while.

Why you can only reproduce "denial of service" and not corruption

There are really two different symptoms of the same broken mechanism, and they don't show up in the same places:

1. The "denial of service" — SQLITE_PROTOCOL / database is locked under high concurrency, where retrying eventually works. This reproduces everywhere the copy scheme is used, including sqlite3_dotlk on Linux/macOS. This is what you're seeing.

2. The actual on-disk corruption — this I've only ever seen on real Windows. On sqlite3_dotlk (Linux/macOS) I ran the stress test dozens of times: every run hit the DoS errors, but PRAGMA integrity_check was always ok — the database stayed intact. On real Windows Server 2022, the same test corrupted the database on ~every run: rowid out of order, btreeInitPage() returns error code 11, database disk image is malformed, sometimes a completely destroyed header.

So the trap is: if you test on anything other than Windows, the bug looks like "just a concurrency/DoS problem." On Windows it's silent data loss. Since you've mentioned you don't have Windows hardware, I think that's exactly why you can only reproduce the DoS — you're testing where only symptom #1 shows.

Offer: I have a reliable Windows repro and a Windows Server 2022 box. If it helps, I'm very happy to run any candidate fix (including winfix) through it and give you a definitive corrupt/clean verdict, so you're not flying blind on the one platform where it matters. Just point me at a branch.

On winfix / 5441ed3

Your instinct ("not confident it's a fix, but not worse") matches what I found. 5441ed3 takes the per-connection mutex (s.Lock()) around the shm operations and adds a fence. The problem is that s.Lock() is per-connection, and the corruption comes from different connections' copy loops interleaving through the shared -shm file — the per-connection lock doesn't order those against each other.

I actually tried the stronger version of this: a process-global mutex around all connections' shmAcquire/shmRelease/shmBarrier. It still corrupted 5/5. So it isn't the locking around the copy that's wrong — it's the copy-at-lock-boundary model itself. wal.c reads and writes the wal-index between xShmLock calls and relies on seeing other connections' updates live (the dirty-header re-read + retry in walIndexTryHdr, the aReadMark re-checks in walTryBeginRead, and the checkpointer capping mxSafeFrame on a read-mark it observed after its exclusive probe lost to a live reader). A copy that only refreshes at lock boundaries hands out stale views on exactly those paths, and a checkpointer acting on one backfills stale pages / truncates not-yet-backfilled frames.

What #405 does

Stop copying on Windows. On 1803+ / Server 2019+ it maps the -shm regions directly into the wasm linear memory with address-space placeholders (VirtualAlloc2 + MapViewOfFile3) — the same idea as the MAP_FIXED mapping the unix build already uses. SQLite then works on genuinely shared, always-current memory, xShmBarrier becomes a plain fence, and the private/shadow copies disappear. That removes both symptoms at once — no corruption and no SQLITE_PROTOCOL storms. Validated on Server 2022 (clean 10/10 where stock corrupts on ~every run), and cross-process interop with a native sqlite3.exe reading the same WAL DB is intact (integrity_check ok). The copy scheme stays as a pre-1803 fallback.

On releasing 3.53.3

For what it's worth: the corruption is pre-existing — it reproduces on every release back to when Windows shared memory was introduced, so 3.53.3 wouldn't be a regression, it'd just carry the same known issue. So you're not making anything worse by cutting it. But it is real data loss on Windows WAL under concurrency, so my vote would be to get a fix in rather than ship another release with it — and I'd love for you to sanity-check #405 (or a variant you prefer) rather than take my word for it. If it's easier, I can also verify winfix on Windows first so you at least know where it stands.

Happy to adjust #405 however you like — structure, the capability gate, whatever fits your taste. Thanks for being so responsive on this.

[ncruces]

OK, first of all thanks for looking into this, and your contribution.

I think I can't wait for it to cut a release, so I'd like to ask if you prefer I disable WAL altogether in the new release, or just note the corruption issue on the release notes?

I'm really glad you got placeholders to work. I remember investigating it, and thinking it was too complicated to implement without access to real hardware. I will look into it, and I'm sure I'll merge some version of it… but it will take me some time.

And the reason for that is that I don't understand the problem very well. Even if your solution is better (which I'm pretty sure it is), I'd like to understand the problem and it's implications for dotlk (as that … hack … is a big part of this project's motto: works everywhere Go does). I want to take this opportunity to learn.

It seems to me that, if dotlk works (does it, on Windows?) my solution for Windows should also work. It bugs me if it doesn't, means something about Windows memory mapping doesn't work as I expect. It also bugs me that I ran this like 20 times on a beafy i9 loaner laptop, and never hit it.

About your solution, at a high level, my two concerns are:

• does this make performance of non-WAL significantly worse? like is the overhead of memory management now worse overall? correctness trumps performance, I'd just like to know.
• does it even make sense then to keep a path for older Windows? I'd bias towards no: I also don't support old Linux, and you can fallback to dotlk if that works.

And thanks again @franchb, this is really welcome!

[franchb]

Thanks — and no rush at all on merging; taking time to understand it is exactly right. Let me answer the release question first, then the part you actually want, which is understanding why.

For 3.53.3 now: I'd release-note it, not disable WAL

My reasoning: this isn't a regression in 3.53.3 — it's been there since Windows shared memory was introduced, so cutting the release doesn't make anything worse. And disabling WAL punishes everyone who uses it safely (readers + a single serialized writer, which is the overwhelmingly common shape) to protect against a pattern — concurrent writers across pooled connections — that has a clean workaround. So my vote is a note plus the workaround. Something like:

Known issue (Windows, WAL): concurrent writes from multiple connections to the same WAL database can corrupt it on Windows. Until this is fixed, on Windows either serialize writers onto a single connection (e.g. db.SetMaxOpenConns(1) for the writing pool, or a dedicated write connection) or use EXCLUSIVE locking mode / rollback journal. Readers are unaffected. Tracked in #404.

If you'd rather be conservative, disabling WAL is defensible too — I just think it's a heavy hammer for this.

Why Windows corrupts but dotlk doesn't — and does dotlk "work"?

Here's the thing that I think will bug you more, not less: the copy scheme in shm_copy.go is the same code for dotlk and Windows. So the staleness defect is present in both. What differs is only the sharing topology:

• dotlk: every in-process connection shares one shared buffer, deduped by path through the global vfsShmList. One buffer, one process — cross-process WAL isn't supported at all.
• Windows: no dedup — each connection independently MapViewOfFiles the -shm file (s.regions/s.shared are per-vfsShm). So you have N private copy pipelines shuttling between wasm-local buffers and a file-backed region, instead of one shared buffer.

That topology difference is the whole story. The bug is: wal.c reads and writes the wal-index between xShmLock calls and needs to see other connections' updates live — the dirty-header re-read in walIndexTryHdr, the aReadMark re-checks in walTryBeginRead, and the checkpointer capping mxSafeFrame on a read-mark it observed after its exclusive probe lost to a live reader. Copy-at-lock-boundary is too coarse for those, so a connection acts on a stale view; under concurrent writers a checkpointer then backfills stale pages / truncates not-yet-backfilled frames.

On dotlk that same staleness exists, but the single shared buffer plus the ferocious SQLITE_PROTOCOL/BUSY storms (which back writers off and keep checkpoints from completing under load) keep the window from ever opening in practice. So the uncomfortable conclusion is: dotlk isn't correct here, it's lucky. "dotlk works" doesn't validate the copy scheme — it just hasn't lost the race yet. That's also why your 20 runs on the i9 came up clean: the corruption needs a specific checkpoint-vs-reader interleaving, and casual concurrency usually doesn't hit it. My repro is tuned to hit it deterministically (64 connections, BEGIN DEFERRED writers, aggressive TRUNCATE checkpoints, cold-reopen + integrity_check).

And to your instinct that "if dotlk works, my Windows solution should too" — they're the same mechanism, so if you could observe dotlk long enough under the right interleaving I believe it would corrupt too. It's not that Windows memory mapping misbehaves; it's that the copy discipline is too coarse for what wal.c assumes, and Windows' per-connection file mappings + real preemption just make it lose every time.

I can turn that belief into data — see below.

Your perf concern (does #405 make non-WAL worse?)

Fair question, and correctness-trumps-perf is the right framing. The design answer: the placeholder machinery only changes (1) how the wasm heap is reserved — VirtualAlloc2 placeholders instead of a plain reserve, one-time at open; and (2) commit granularity on growth — 64K instead of 4K, negligibly coarser. The split/coalesce path runs only when a -shm region is mapped or unmapped, i.e. only for WAL. A non-WAL database never maps a -shm, so it never touches any of the new code — same hot path as stock.

But I don't want to hand-wave it. I'll run the repo's own benchmarks (speedtest1, driver loop, and the parallel/WAL ones) on real Windows, stock vs #405, and post the benchstat comparison here so you have numbers instead of my assurance.

Keep the old-Windows fallback?

I agree with your bias — drop it. Requiring 1803+ / Server 2019+ removes shm_copy.go from the Windows path entirely (the one piece I said can't be made fully correct), shrinks the diff a lot, and matches how you already treat old Linux. On pre-1803 I'd fail shm creation cleanly so WAL falls back to no-shm EXCLUSIVE mode (as unsupported platforms already do) rather than silently using a broken copy path — but tell me which behavior you prefer there and I'll match it. I'm happy to revise #405 to drop the fallback on your say-so.

What I'll come back with

Since I have the hardware and you don't, let me just get you the data:

1. winfix on real Windows — a definitive corrupt/clean verdict on 5441ed3.
2. sqlite3_dotlk forced on Windows — to test the "same defect, just lucky" claim directly.
3. Benchmarks stock vs vfs: fix Windows WAL corruption by mapping the wal-index into wasm memory #405 (WAL and non-WAL) with benchstat.

I'll post all three here. Thanks again — this is a fun one.

[franchb]

Ran the three experiments on Windows Server 2022 (i5-14600K, GOMAXPROCS=4). Results below — and one of them corrected a claim I made above, which I'm glad I checked instead of just asserting.

Setup for all of these: a single *sql.DB, SetMaxOpenConns(64), BEGIN DEFERRED writers doing insert-into-parent+child, wal_checkpoint(TRUNCATE) every 25 iters, then cold-reopen + PRAGMA integrity_check. Same base commit (d081bc7) for every binary, so the only variable is the shm implementation. SQLITE_BUSY is tolerated (normal backpressure); SQLITE_PROTOCOL and any integrity failure are not.

1. winfix (5441ed3): still corrupts, 10/10

runs	outcome
8/10	genuine on-disk corruption — integrity_check b-tree failures (Rowid … out of order, overflow list length…, Page … never used), or database disk image is malformed / file is not a database on reopen
2/10	SQLITE_PROTOCOL (locking protocol)
0/10	clean

So the per-connection s.Lock() doesn't move the needle — same as a process-global mutex did in my earlier testing. Expected, but now you have the verdict you couldn't get without the hardware.

2. sqlite3_dotlk on Windows — this corrected me

I claimed above that dotlk "would corrupt too if you observed it long enough." My 10-run test says otherwise, so I'm walking that back. Forcing -tags sqlite3_dotlk on Windows:

runs	outcome
9/10	SQLITE_PROTOCOL (locking protocol) — the DoS
0/10	on-disk corruption
1/10	fully clean

Every failure was the DoS; not one run produced on-disk corruption. So the two defects separate cleanly:

• Defect 1 (the SQLITE_PROTOCOL storm) is shared by dotlk and the Windows copy path — dotlk actually hits it harder (9/10).
• Defect 2 (on-disk corruption) showed up only on the Windows copy path, never on dotlk in these runs.

That means the corruption tracks the Windows topology specifically — each connection independently MapViewOfFiles the -shm, so you have N private copy pipelines — not "the copy scheme" in the abstract. dotlk's single process-wide shared buffer (deduped by path) apparently narrows the corruption window to nothing I could hit in 10 runs, even while the DoS fires constantly. My "dotlk is just lucky" line was half right: it's lucky on corruption, but it's not lucky on the DoS — it's squarely subject to that.

Practical upshot for the release/fallback question: dotlk is not a usable WAL path on Windows either. It answers your "does dotlk work on Windows?" — it builds and runs, but under concurrent writers it fails 9/10, just in the retryable-lockout way rather than silent data loss. So "fall back to dotlk on old Windows" wouldn't give correct WAL; it'd give the storm. Worth knowing when you decide what to do below 1803.

3. Performance: stock vs #405

Repo benchmarks, same VM, -benchmem -count=6, compared with benchstat:

benchmark	workload	stock	#405	Δ time/op
driver/Benchmark_loop	in-memory, non-WAL (1M-row recursive CTE)	474 ms	466 ms	~ (p=0.39)
Benchmark_parallel	on-disk, journal_mode=truncate, concurrent (non-WAL)	315 µs	327 µs	~ (p=0.66)
Benchmark_wal	on-disk, WAL, concurrent	1.32 ms	1.47 ms	~ (p=0.18)

Non-WAL is unchanged — both the in-memory and the on-disk non-WAL benchmarks are statistically flat, which matches the design (nothing on a non-WAL path touches the placeholder code; a database that never maps a -shm runs the stock code exactly). That's the answer to your main question: the memory-management change doesn't cost non-WAL anything.

WAL time is within noise (p=0.18), but I'll be honest: the point estimate ticked up ~11% with high variance (±24%) on a noisy shared VM, so I don't trust it either way — happy to run a longer sweep if you want a tight bound. One nice side effect: WAL Go-heap allocations dropped ~half (60.4 kB → 28.1 kB/op), because the direct map removes the per-connection copy/shadow buffers the old path churned.

(speedtest1 I couldn't include — it needs a wasm built with the main_speedtest1 export, which the stock module doesn't ship, so it nil-derefs identically on both. No signal either way.)

Where that leaves things

• Release: I'd still release-note it (workaround: serialize writers / SetMaxOpenConns(1) on the writer). dotlk isn't a safe alternative on Windows.
• vfs: fix Windows WAL corruption by mapping the wal-index into wasm memory #405: happy to revise it to drop the pre-1803 copy fallback on your say-so — given (2), below 1803 I'd fail shm creation so WAL falls back to no-shm EXCLUSIVE mode rather than silently using either broken path.

All the binaries were built from the same base; happy to share the reproducer or run anything else you want checked on real hardware.

[franchb]

Follow-up: I turned the reproducer into a regression test and, more to the point, got it to fail on GitHub's own windows-latest runner — so this no longer depends on my hardware. It's now in #405.

It reproduces in your CI, reliably

The test (vfs/tests/wal_stress_test.go) drives concurrent WAL writers, aggressively checkpoints, cold-reopens and runs integrity_check, repeating until it either catches corruption or hits a time budget. On windows-latest (which reports NumCPU=4):

build	windows-latest	detail
stock main	❌ fail	reproduced in round 1 — database disk image is malformed (round 1, 40s)
#405 (fix)	✅ pass	budget clean (PASS, ~10 min)
either, ubuntu	✅ pass	Linux direct-maps the -shm, so no bug — a nice built-in control

Proof runs (public, on my fork):

• fix, green: https://github.com/franchb/go-sqlite3/actions/runs/28793171338
• stock, red: https://github.com/franchb/go-sqlite3/actions/runs/28793181822
• (stock is red on every run I did — 4/4: file is not a database ×2 and database disk image is malformed ×2)

So the test goes red only on an unfixed Windows build and green everywhere else — which is exactly what you'd want a regression gate to do, and it means you can watch it flip red→green yourself.

One thing worth flagging about reproducing it in CI

My first CI attempt was green on stock — a false negative — and the reason is instructive. GitHub's Windows runners have a slow temp disk, so each round ran ~10× slower than on my box; a light per-round workload didn't accumulate enough checkpoint cycles to open the window. Matching the workload I use locally (64 writers × 2000 iters, 8 KB rows, TRUNCATE checkpoint every 25) reproduces in the first round, ~30 s. Under-provisioning the workload hides the bug — which is probably also why it took a while for anyone to hit corruption rather than just the SQLITE_PROTOCOL storm.

Packaging (your call)

The test is opt-in — gated behind SQLITE3_TEST_WAL_STRESS, several minutes of heavy I/O — so it does not run in your normal matrix and costs you nothing by default. #405 also adds a small workflow_dispatch workflow that flips the gate and runs just this test on Windows + Linux, so it's one click to run on demand. Whether you want it always-on, nightly, behind a label, or purely manual is entirely up to you — I kept it out of the hot path so the decision is yours. Happy to wire it however you prefer, or drop the workflow and keep just the test.