[Onyx] Investigate `QuotaExceededError` storage error

[fabioh8010]

Coming from #87844.

Summary

Let's investigate the following Onyx storage error: QuotaExceededError (bare, no detail), which accounts for 51,900 (1.5%) of the 3,364,645 analysed errors during a time span of 1 month. Full details about the investigation and methodology are in #87844.

Platforms	Top Methods
Desktop Chrome (51K), Desktop Edge (915)	setWithRetry (51K)

Retries Exhausted	Evictions	No Eviction	Quota Checks	>= 95% Full	Mean % Full	Quota Check Failed
259	51,322	307	48,022	1,990	4.2%	1,196

• The only error with significant >= 95% full cases (1,990) and the highest mean storage usage (4.2%)
• 1,196 failed to even get database size — navigator.storage.estimate() itself threw an error
• Chromium — generic quota exceeded with real storage pressure in some cases

Action Items

• Investigate with Sentry, VictoriaLogs and Fullstory to understand user scenarios — these are the users with the highest actual storage pressure. What does their Onyx store look like? Are there specific keys consuming disproportionate space?
• 307 requests hit the "no evictable keys" dead end — review whether more Onyx keys could be added to the evictableKeys list
• Investigate why 1,196 requests failed to get database size — navigator.storage.estimate() throwing errors may indicate browsers in private/incognito mode or restrictive privacy settings

---

[sosek108]

I'll be investigating this

[sosek108]

We're seeing ~52k QuotaExceededError events/month on web when Onyx writes to IndexedDB, and 100% of the affected population is on Chromium (Chrome + Edge). The brief framed this as a storage-pressure problem ("heavy users hitting the origin quota cap"), but log analysis killed that story: across 23k VictoriaLogs lines over 7 days, the maximum observed IDB usage was 6.26% of origin quota — every affected user has gigabytes of headroom.

The current best hypothesis is that Chromium surfaces JS-heap allocation failures during IDB writes as a bare QuotaExceededError with no message. Three reproducer users in Sentry were all at 87–97% JS heap when the error fired, on screens that issue large bulk writes (/search, /home, /transition, /workspaces). MDN and web.dev confirm QuotaExceededError is an umbrella class covering disk-quota, per-transaction sub-quota, and memory-constraint failures — Chromium throws the bare error name without a message by design, so the empty message is normal, not a logging bug. Example Sentry to support this hypothesis. (FullStory)

Proposed next steps:

• tag every Onyx write-error log with memoryUsagePercentage to confirm the correlation in bulk,
• continue investigation

[sosek108]

What a ride this investigation is.

Update

Out of 28k events 80% of them are from 4 users (where one of them is our test account) so this is user specific:

These users are reporting high Storage usage (Storage Quota Check -- bytesUsed: logs):

I've been able to reproduce this issue by setting low storage quota:

• Open https://new.expensify.com → DevTools → Application → Storage → check "Simulate custom storage quota" → set to e.g. 10 MB

When storage is overused browser throws QuotaExceededError without any specific explanation. Our logs Storage Quota Check -- bytesUsed: ${bytesUsed} bytesRemaining: ${bytesRemaining}. Original error: ${error} are misleading because they don't show correct bytesRemaining. Browser reports full 10G availability, but assigned storage can be much much lower (eg. 150M).

Observation

when user has low storage assigned (weaker laptop, small disk availability) storage may be totally used and we get this error. I don't think we can do anything about this.
Maybe we can see why users has so much storage used at all?

I've open staging app on my second computer (old M1 with 9G disk space availability) and Chrome assinged 147 MB storage.
60M of it was used by CACHE before I've even logged in!. After switch to Applause QA perf account I've used 128MB out of 147MB.

Action steps

1. In storage IDBKeyValProvider modify getDatabaseSize to return also usageDetails props which tells us how is the storage used.

• indexedDB -> onyx
• cache or fileSystem -> may not be onyx related

2. I will open the application for few hours and check how is the ussage there. CUrrently: 121 MB used by cache and 7M used by indexedDb.

[sosek108]

Follow up:

after 4h of total inactivity there is 132M used and 124 by cache and 7,5M by IndexedDB. So inactivity does not add much, but it's still 3M out of thin air

[sosek108]

Immediately after application is open Workbox Precache is caching ~314 elements and this weights around 125M. I did not notice this value to rise, but prod user sessions may be very long. And cache more things such as receipt images.

The final root cause of this error is: user's application uses all of available space. This is either for cache, IndexedDB or both. This is hard to reproduce on dev as Precache is disabled for dev environment and we use quite capable machines I guess. For low end machines or machines with low disk space availability storage allowed for Exfy is very low. And we cannot do much about it.

After I've logged in to applausetester@ account I've used 137M in no time. But it was hard to go to cap 147 as database clearing was working.

Impact
There are plenty of log messages on ViktoriaLog because there is retry method after QuotaExceededError is very aggresive but after taking closer look only few (like less than 10) users are really affected by this.
Sentry data backups this claim as QuotaExceededErrors are much much less occuring in sentry.

Proposed next steps

1. we can extend Storage Quota Check -- bytesUsed: to log also what is using the bytes - cache, DB or other. This will give us more debug information for future issues.

[sosek108]

If we decide to push these logs here is PR: Expensify/react-native-onyx#795

cc: @mountiny @fabioh8010