Web: Remote error: Invalid argument(s): length cannot be negative from database worker on Safari

[xvrh]

We have this error in one of our computer. The database seems corrupted and it won't open anymore.

Remote error: Invalid argument(s): length cannot be negative
The minified stack symbolicated against a source-map:

package:sqlite3_web/src/channel.dart 202:7              ProtocolChannel.sendRequest
dart-sdk/_internal/js_runtime/async_patch.dart 339:19   _wrapJsFunctionForAsync
dart-sdk/_internal/js_runtime/async_patch.dart 365:23   _wrapJsFunctionForAsync.<fn>
dart-sdk/_internal/js_runtime/async_patch.dart 313:7    _awaitOnObject.<fn>
dart-sdk/async/zone.dart 962:46                         _RootZone.runUnary
dart-sdk/async/future_impl.dart 186:29                  _FutureListener.handleValue
dart-sdk/async/future_impl.dart 977:13                  _Future._propagateToListeners
dart-sdk/async/future_impl.dart 711:7                   _Future._complete
dart-sdk/async/future_impl.dart 113:5                   _SyncCompleter.complete
package:sqlite3_web/src/channel.dart 131:20             ProtocolChannel._handleIncoming.<fn>
package:sqlite3_web/src/protocol/helper.g.dart 810:14   dispatchMessage
package:sqlite3_web/src/channel.dart 128:5              ProtocolChannel._handleIncoming
package:sqlite3_web/src/channel.dart 98:11              ProtocolChannel.<fn>
package:web/src/helpers/events/streams.dart 165:45      _EventStreamSubscription.<fn>
dart-sdk/_internal/js_runtime/js_allow_interop_patch.dart 367:27  _callDartFunctionFast1

Versions: sqlite3_web 0.8.1, sqlite3 3.3.2, powersync 2.2.0. Observed on Safari only so far.

If this is useful for you, we can try to extract the IndexedDB bytes from the browser to understand the corruption.

[simolus3]

It's worth noting that we're able to use OPFS on Safari without special headers too know (but we'd keep using IndexedDB if such database already exists).

Is this reproducible with the same steps shared in powersync-ja/powersync.dart#428?

[xvrh]

@simolus3 I think this is a separate problem than the other ones.
The user had a database from a previous session, then after an update he is now un-able to open the app.
The update was new client side lib (powersync) + updated server side sync_rules.

[xvrh]

I gave the AI a chance to investigate fully. It did the extraction of the corrupted database and applied some analysis.
To take with a grain of salt obviously.

AI analysis

Some more information: we dumped the raw IndexedDB stores of the affected database on that machine (Safari 26.5, macOS). The failure is a property of the stored data — every open fails, including from a fresh tab — and the dump shows exactly why.

What the IndexedDB store contains

The database (files + blocks object stores, as created by IndexedDbFileSystem):

• files store: one entry, /database, recorded length = 917,504 bytes (= 224 × 4096).
• blocks store: 230 rows. All 224 expected blocks covering [0, length) are present — no holes, every value is a 4096-byte ArrayBuffer.
• 6 extra rows at or past the recorded length, at offsets 917504, 921600, 925696, 929792, 933888, 937984.

So everything below the recorded file length is intact; the only anomaly is leftover block rows past the end of the file.

Why those rows make the database unopenable

readFully (in lib/src/wasm/vfs/indexed_db.dart, called for every file from _readFiles() when the file system loads) iterates a cursor over all blocks rows of the file, including rows past the recorded length:

final result = Uint8List(file.length);
// cursor over all blocks of this file…
final length = min(_blockSize, file.length - rowOffset);
result.setAll(rowOffset, data.asUint8List(0, length));

• The row at offset == 917504 (== length): min(4096, 0) = 0 → asUint8List(0, 0) → harmless.
• The row at offset == 921600 (length + 4096): min(4096, -4096) = -4096 → data.asUint8List(0, -4096). On JavaScriptCore the underlying new Uint8Array(buffer, 0, -4096) throws RangeError: length cannot be negative, which dart2js rethrows as the ArgumentError from the stack trace above (Chrome would phrase the same error Invalid typed array length).

Since this happens while the VFS loads its files, the database can never be opened again. We verified the diagnosis by deleting the 6 extra rows with a devtools script: the database opens fine afterwards, no data lost.

read() has the same unguarded file.length - rowOffset in its rowOffset < offset branch (only the other branch checks lengthToCopy < 0).

The crash state is easy to synthesize without Safari: take any working IndexedDbFileSystem database and, using plain IndexedDB, put a 4096-byte ArrayBuffer at key [fileId, recordedLength + 4096]. The next open throws.

Where the row at exactly length comes from (reproducible in current code)

truncate():

final lastBlock = (length ~/ _blockSize) * _blockSize;
// Delete all higher blocks
await blocks.delete(_rangeOverFile(fileId, startOffset: lastBlock + 1));

SQLite only truncates to page-aligned sizes (multiples of 4 KiB), so length % _blockSize == 0 in practice, which makes lastBlock == length: the delete range starts at length + 1 and the block at exactly length is never deleted. Every shrinking truncate leaves one stale row behind. Repro: write a 12,288-byte file (blocks at 0/4096/8192), xTruncate to 8192, inspect the blocks store — the row at offset 8192 is still there.

That row alone is harmless (the min → 0 case), but it normalizes "rows past EOF" as a state, and anything one block further turns into the permanent crash above. For the fix, startOffset: length looks correct for both cases: when length is unaligned the first existing key ≥ length is lastBlock + 4096 (the right first block to delete), and when aligned it includes the block at length.

What we can't explain yet

How the rows further past EOF (921600…937984) survived: _write and truncate both update blocks and the file length within a single IndexedDB transaction, so per spec one shouldn't be observable without the other. We've only seen this on Safari (long-lived profile, database predates OPFS support, so it stays on IndexedDB per the stickiness). Possibly a WebKit IndexedDB durability issue — no proof.

Suggested hardening

• readFully / read: skip rows with rowOffset >= file.length (ideally delete them — they're unreachable anyway).
• truncate: include the block at length in the delete range when length is block-aligned.

We kept the full dump (every block key/size/type + file metadata, JSON) and a copy of the affected store, so we can test a fix against the real data if useful.

[simolus3]

I think that open bug is definitely something worth fixing, ignoring blocks past the file size sounds correct (fixed in 4d492e0).

But as the LLM said, the truncate bug is an off-by-one error causing blocks at the file length (so we use length: 0 when reading, which doesn't cause crashes). So I'm also not sure why this happened, but being defensive about it when reading files should be an obvious solution.

Do you have a way to compile your own workers for PowerSync to try this out? I can provide one otherwise.

[xvrh]

We don't compile the worker, if we can just download it we will try it.

[simolus3]

This worker is the latest release with the IndexedDB patches applied.

[xvrh]

@simolus3 I tried to deploy a new version of the app with your worker. Hit this error, I don't know if it's related or not:

[SEVERE] app: Failed to load user (SqliteException(1): The powersync extension is not loaded correctly. Details: SqliteException(11): while preparing statement, malformed database schema (sqlite_autoindex_ps_data__organization_invites_1) - invalid rootpage, database disk image is malformed (code 11)
  Causing statement: SELECT powersync_rs_version() as version)

[simolus3]

I can see that being caused by the same root cause (causing the reported file size and the actual amount of blocks to go out of sync). I don't see how the changes would have made that worse though, it might just be another symptom.