[BUG] Unstable ISR cache after migration to 1.19.2 (and newer)

[grabmateusz]

Describe the bug

OpenNext randomly decides to treat cache as STALE and revalidates the route. This generates cascade of problems - revalidation is triggered too often, during revalidation we call downstream systems, this generates much bigger traffic on downstream systems than previous versions, those systems have their own pricing, etc.

Steps to reproduce

I am not able to reproduce the problem in isolation at the moment - very likely huge number of sub-requests we generate during revalidation is also a factor. But what we observe is well visible if we run a cron that presents status + 4 crucial headers. Here is the output:

[2026-06-12 08:08:09] status=200 | cache-control=s-maxage=1136, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:08:25] status=200 | cache-control=s-maxage=1120, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:08:40] status=200 | cache-control=s-maxage=1105, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:08:56] status=200 | cache-control=s-maxage=1090, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:09:11] status=200 | cache-control=s-maxage=1074, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:09:26] status=200 | cache-control=s-maxage=1059, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:09:42] status=200 | cache-control=s-maxage=1043, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:09:57] status=200 | cache-control=s-maxage=1028, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:10:13] status=200 | cache-control=s-maxage=1012, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:10:28] status=200 | cache-control=s-maxage=997, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:10:44] status=200 | cache-control=s-maxage=981, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:10:59] status=200 | cache-control=s-maxage=966, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:11:15] status=200 | cache-control=s-maxage=951, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:11:30] status=200 | cache-control=s-maxage=935, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:11:46] status=200 | cache-control=s-maxage=920, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:12:01] status=200 | cache-control=s-maxage=904, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:12:17] status=200 | cache-control=s-maxage=889, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:12:32] status=200 | cache-control=s-maxage=873, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:12:48] status=200 | cache-control=s-maxage=858, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:13:03] status=200 | cache-control=s-maxage=842, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:13:19] status=200 | cache-control=s-maxage=827, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:13:34] status=200 | cache-control=s-maxage=1783, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:13:49] status=200 | cache-control=s-maxage=1767, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:14:05] status=200 | cache-control=s-maxage=1752, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:14:21] status=200 | cache-control=s-maxage=5, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:14:36] status=200 | cache-control=s-maxage=1, stale-while-revalidate=2592000 | x-opennext-cache=STALE | cf-placement=local-WAW
[2026-06-12 08:14:51] status=200 | cache-control=s-maxage=1782, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:15:07] status=200 | cache-control=s-maxage=1767, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:15:22] status=200 | cache-control=s-maxage=1751, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:15:38] status=200 | cache-control=s-maxage=1736, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:15:53] status=200 | cache-control=s-maxage=1720, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:16:09] status=200 | cache-control=s-maxage=1705, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW
[2026-06-12 08:16:25] status=200 | cache-control=s-maxage=1689, stale-while-revalidate=2592000 | x-opennext-cache=HIT | cf-placement=local-WAW

During this time frame there was no revalidateTag being called explicitly from the app, while (as you can see) route was 2 times revalidated, even if it should return cached version for next 13 minutes.
While watching production logs we can see we have revalidate requests (HEAD HTTP method requests) on each route every minute, sometimes even more often, while we do not call any HEAD HTTP request from our application.
There are occurrences of rpc default.revalidate in the logs, but with the information there we can't find correlation between particular default.revalidate call and HEAD HTTP method revalidation request.

Expected behavior

OpenNext should not trigger periodic revalidation of the route, if TTL is not reached / exceeded and no explicit revalidation is called

@opennextjs/cloudflare version

1.19.10

Wrangler version

4.77.0

next info output

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 25.3.0: Wed Jan 28 20:53:15 PST 2026; root:xnu-12377.81.4~5/RELEASE_ARM64_T6000
  Available memory (MB): 32768
  Available CPU cores: 10
Binaries:
  Node: 22.21.1
  npm: 10.9.4
  Yarn: 1.22.21
  pnpm: N/A
Relevant Packages:
  next: 16.2.6 // There is a newer version (16.2.9) available, upgrade recommended! 
  eslint-config-next: N/A
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.9.3
Next.js Config:
  output: N/A
⚠ There is a newer version (16.2.9) available, upgrade recommended! 
   Please try the latest canary version (`npm install next@canary`) to confirm the issue still exists before creating a new issue.
   Read more - https://nextjs.org/docs/messages/opening-an-issue

Additional context

This seems to be regression of SWR implementation on OpenNext - before updating to 1.19.2 we've had much lower outbound traffic.

[MattieTK]

Hey @grabmateusz, do you have a minimal reproduction of this? At a minimum would it be possible for you to provide an anonymised version of your opennext.config.ts file as this will help understand how your caching is set up?

Thanks

[grabmateusz]

Hi @MattieTK, our open-next.config.ts:

import { defineCloudflareConfig } from "@opennextjs/cloudflare";
import r2IncrementalCache from "@opennextjs/cloudflare/overrides/incremental-cache/r2-incremental-cache";
import { withRegionalCache } from "@opennextjs/cloudflare/overrides/incremental-cache/regional-cache";
import doQueue from "@opennextjs/cloudflare/overrides/queue/do-queue";
import d1NextTagCache from "@opennextjs/cloudflare/overrides/tag-cache/d1-next-tag-cache";

export default defineCloudflareConfig({
  incrementalCache: withRegionalCache(r2IncrementalCache, {
    mode: "long-lived",
    shouldLazilyUpdateOnCacheHit: true,
  }),
  queue: doQueue,
  tagCache: d1NextTagCache,
  enableCacheInterception: true,
});

As I've written we do not have standalone reproduction scenario, for now it seems that after OpenNext upgrade our DO Queue is overwhelmed and we're going with mitigation where new open-next.config.ts looks like this:

import { defineCloudflareConfig } from "@opennextjs/cloudflare";
import r2IncrementalCache from "@opennextjs/cloudflare/overrides/incremental-cache/r2-incremental-cache";
import { withRegionalCache } from "@opennextjs/cloudflare/overrides/incremental-cache/regional-cache";
import doQueue from "@opennextjs/cloudflare/overrides/queue/do-queue";
import queueCache from "@opennextjs/cloudflare/overrides/queue/queue-cache";
import d1NextTagCache from "@opennextjs/cloudflare/overrides/tag-cache/d1-next-tag-cache";

export default defineCloudflareConfig({
  incrementalCache: withRegionalCache(r2IncrementalCache, {
    mode: "long-lived",
    shouldLazilyUpdateOnCacheHit: true,
  }),
  queue: queueCache(doQueue, {
    regionalCacheTtlSec: 5,
    waitForQueueAck: false,
  }),
  tagCache: d1NextTagCache,
  enableCacheInterception: true,
});

• we tweak default DO_QUEUE vars:

"NEXT_CACHE_DO_QUEUE_MAX_REVALIDATION": "2",        // default ~5 parallel per instance
"NEXT_CACHE_DO_QUEUE_REVALIDATION_TIMEOUT_MS": "30000",
"NEXT_CACHE_DO_QUEUE_MAX_RETRIES": "2",
"NEXT_CACHE_DO_QUEUE_RETRY_INTERVAL_MS": "10000"

as per https://opennext.js.org/cloudflare/caching#queue.
However those are mitigations after the upgrade, we do not see increased inbound traffic on the workers itself while comparing to previous versions.

[james-elicx]

I was working through an idea with Codex that it might be related to the long-lived regional cache, and lazy updates somehow overwriting what's currently in the cache with an older value.

To be honest, I'm not completely sold on that being the fix, but it might potentially help mitigate some of the impact (or there's always the chance it won't).

There's a prerelease for that theory in #1283 (comment) if you would be open to trying it out.

[conico974]

@grabmateusz Just to confirm it started on 1.19.2 and no other newer versions fix it right ?
Also are you using SWR kind of revalidateTag (i.e. not with expire: 0) ?

I think @james-elicx theory might be the right one, but I don't think his fix is enough. The problem might be because of the nature of the regional cache itself and the lazy update.
When a page is STALE, the queue ensure that only 1 revalidation occurs, which means only 1 region gets the updated cache entry.
All other regions where a page has become STALE will persist that STALE version again that will be checked for staleness against the tag cache at a later time (and it will be considered stale then)
I think the shouldLazilyUpdateOnCacheHit might be inherently incompatible with SWR kind of revalidateTag

[grabmateusz]

@conico974 yes, we're having TTL on the routes either 5 or 30 minutes + we have revalidation endpoint, triggered as a webhook in other system, which calls:
revalidateTag("<some-tag>", "max");

[grabmateusz]

@conico974 apologies for not answering explicitly. You've written:

Just to confirm it started on 1.19.2 and no other newer versions fix it right ?

It started on 1.19.2, we've on 1.19.10 and issue is still happening.

[conico974]

@grabmateusz Before 1.19.2, there were no support for SWR kind of revalidateTag anyway.
As I said, I think you cannot use the shouldLazilyUpdateOnCacheHit with SWR kind of revalidateTag. We should probably just deprecate that option.

Could you remove that option and try to use purge cache instead?
We may need to make some other change to the regional cache, but this should help already reducing the load.

[grabmateusz]

@conico974 interesting input, thank you.

As I said, I think you cannot use the shouldLazilyUpdateOnCacheHit with SWR kind of revalidateTag. We should probably just deprecate that option.

@vicb @MattieTK could you comment?

[grabmateusz]

Hi @conico974 , you've written:

Could you remove that option and try to use purge cache instead? We may need to make some other change to the regional cache, but this should help already reducing the load.

Removal of the option does nothing, as "true" is default value, so I assume you want me to switch to explicit "false". With explicit "false" I had once infinite staleness of the route, but then I've not been able to reproduce the issue on non-prod environment, anyway this staleness shows me that there is still essentially wrong with the setup.
I will do some session with Claude to deep dive into the topic later this week.

[grabmateusz]

Going forward with the issue,
we've had a scenario in which some of the routes go to "inifinite staleness" mode and we're not even able to recover from this state.
We have a route, where in the response headers we have:

cf-placement: remote-LHR
cf-ray: <ray>-WAW
x-opennext-cache: STALE

And there is no way to recover from this situation, x-opennext-cache: STALE seems to be infinite. Even if we call an endpoint that calls:

    revalidatePath("/[locale]/[[...slug]]", "page");

This seems to be executed on the edge that received the traffic (so in my case WAW one) and not affect origin (LHR in this case).

[conico974]

I see I think I have a solution for this. The idea will be to never save stale data into the regional cache, this will impact performance while the data is stale, but I don't think there is a better alternative for this case.
I have a PR in aws (it's missing some part) opennextjs/opennextjs-aws#1193, I'll work on it tonight or this week-end