Skip to content

debug(mobile): no-issue: add sync debug logs and disable server overrides#10140

Open
jaskfla wants to merge 3 commits into
release/2.54from
debug/no-issue/mobile-sync-debug-logs-2.54
Open

debug(mobile): no-issue: add sync debug logs and disable server overrides#10140
jaskfla wants to merge 3 commits into
release/2.54from
debug/no-issue/mobile-sync-debug-logs-2.54

Conversation

@jaskfla

@jaskfla jaskfla commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Changes

Temporary debug PR for investigating initial sync failures on mobile.

  • Adds SyncDebugLog service that collects sync lifecycle and insert-failure details during sync runs
  • Surfaces debug logs in the sync error display UI (scrollable, below the error message)
  • Logs sync start params, pull initiation, per-table save counts, and detailed row data on insert failures
  • Temporarily comments out serverOverrides.json usage so dev builds use the normal meta server list_

Auto-Deploy

  • Deploy
Options
  • Artillery load test
  • Seed from closest snapshot
  • Generate fake data
  • More data (20Gi)
  • No facility servers (central-only)
  • No sync (facility tasks scaled to zero)
  • AMD64 architecture (default is arm64)
  • Skip mobile build
  • Always build mobile
  • Stay up for 8 hours
  • Stay up for 24 hours
  • Stay up (no TTL)
  • Build images only (don't deploy)
  • Pause this deploy

Tests

  • Run E2E Tests

Review Hero

  • Run Review Hero
  • Auto-fix review suggestions Wait for Review Hero to finish, resolve any comments you disagree with or want to fix manually, then check this to auto-fix the rest.
  • Auto-fix CI failures Check this to auto-fix lint errors, test failures, and other CI issues.
  • Auto-merge upstream Check this to merge the base branch into this PR, with AI conflict resolution if needed.

Remember to...

  • ...write or update tests
  • ...add UI screenshots and testing notes to the Linear issue
  • ...add any manual upgrade steps to the Linear issue
  • ...update the config reference, settings reference, or any relevant runbook(s)
  • ...call out additions or changes to config files for the deployment team to take note of

…ides

Add in-app sync debug logging to help diagnose initial sync failures,
and temporarily disable server overrides so dev builds can use the normal
server selector.

Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit d3dec22. Configure here.

const defaultMetaServer = 'https://meta.tamanu.app';
const metaServer = metaServerOverride || defaultMetaServer;
const metaServer = defaultMetaServer;
// const metaServer = metaServerOverride || defaultMetaServer;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build server overrides ignored

High Severity

The PR disables reading serverOverrides.json for every build, not only dev. App Center still writes that file from SERVER_OVERRIDES, but fetchServers always uses the default meta server and public server list, so pinned central servers and custom meta URLs no longer apply.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d3dec22. Configure here.

Comment thread packages/mobile/App/services/sync/utils/executePreparedQuery.ts Outdated
Comment thread packages/mobile/App/services/sync/SyncDebugLog.ts
Comment thread packages/mobile/App/services/sync/utils/saveIncomingChanges.ts
Comment thread packages/mobile/App/services/sync/utils/saveIncomingChanges.ts
@review-hero

review-hero Bot commented Jun 24, 2026

Copy link
Copy Markdown

🦸 Review Hero Summary
11 agents reviewed this PR | 1 failed | 2 critical | 3 suggestions | 1 nitpick | Filtering: consensus 3 voters, 4 below threshold

Below consensus threshold (4 unique issues not confirmed by majority)
Location Agent Severity Comment
packages/mobile/App/services/sync/index.ts:3 Design & Architecture suggestion SyncDebugLog is re-exported through the module's public index, making it part of the sync service's public API. This is a debug utility that's likely temporary — exporting it through the barrel mea...
packages/mobile/App/services/sync/MobileSyncManager.ts:376 Performance suggestion tablesForFullResync is passed directly as a log field in the Pull initiated entry. On initial sync this can be the full list of all sync-able tables (potentially 50-100+ items). Because `SyncDe...
packages/mobile/App/services/sync/SyncDebugLog.ts:19 Performance suggestion Array.shift() is O(n) — it re-indexes the entire array on every call. For a ring buffer, prefer a fixed-size circular buffer or simply splice/unshift less often. At MAX_LOGS=100 this is negligible ...
packages/mobile/App/services/sync/SyncDebugLog.ts:27 Performance suggestion formatLogs() calls JSON.stringify(entry.data) for every stored entry when constructing the error display string. Entries can contain large objects: tablesForFullResync (an array of table name...

Nitpicks

File Line Agent Comment
packages/mobile/App/services/sync/SyncDebugLog.ts 25 Design & Architecture getLogs() returns a defensive copy of the log array but is never called anywhere in this PR. The only consumer is SyncErrorDisplay which calls formatLogs(). Unused public method adds surface area to the API with no current benefit.
Local fix prompt (copy to your coding agent)

Fix these issues identified on the pull request. One commit per issue fixed.


packages/mobile/App/services/sync/utils/executePreparedQuery.ts:61: Logging data: row passes the full row object to SyncDebugLog, which then emits it via console.log. Sync rows include Patient, Encounter, and other clinical records containing patient-identifiable information. This violates the project rule 'Patient data must never be logged at INFO level or above'. Remove the data: row field — recordId and columns: Object.keys(row) already provide enough context to diagnose the failure without exposing the values.


packages/mobile/App/services/sync/SyncDebugLog.ts:22: Every call to SyncDebugLog.log() unconditionally emits to console.log, including the full data payload. Because callers pass full row objects (see executePreparedQuery.ts line 57-63), patient-identifiable data ends up in device console logs. Console output is accessible via adb logcat and crash reporting tools. Either remove the console.log entirely (the in-memory log is sufficient for the debug use-case) or guard it behind a dev/debug flag and ensure patient row data is never passed through.


packages/mobile/App/ui/components/ServerSelectorField/ServerSelector.tsx:8: Three blocks of production code are commented out with 'TEMPORARY' markers rather than removed or feature-flagged. Commented-out code is a project antipattern — it rots as the surrounding code evolves and signals unfinished work to reviewers. If this is genuinely temporary, use a compile-time constant or environment check to toggle the behaviour so the dead code path is explicit and removable in one place.


packages/mobile/App/services/sync/SyncDebugLog.ts:25: getLogs() returns a defensive copy of the log array but is never called anywhere in this PR. The only consumer is SyncErrorDisplay which calls formatLogs(). Unused public method adds surface area to the API with no current benefit.


packages/mobile/App/services/sync/utils/saveIncomingChanges.ts:48: SyncDebugLog.log calls console.log on every invocation (see SyncDebugLog.ts line 21). saveChangesForModel is called once per model per batch — with ~50 models and many pages of records this means hundreds of synchronous console.log calls per sync. In React Native, console.log is expensive because it serializes arguments and crosses the JS-native bridge. Consider removing the console.log from the hot-path log() method and only printing to console in formatLogs or on error, or gating console output behind a debug flag.


packages/mobile/App/services/sync/utils/saveIncomingChanges.ts:98: The saveChangesFromMemory log at line ~106 allocates a new array via .map() on modelChanges on every batch call to build the models summary. Because this runs once per batch (potentially dozens of times per sync) and the result is only ever used for logging, consider deferring or removing this allocation. If the log is needed, compute it lazily only when a debug flag is set, or pass a count rather than mapping the full array.

tableName,
recordId: row.id,
columns: Object.keys(row),
data: row,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Bugs & Correctness] critical

Logging data: row on insert failure writes the full row payload (which can include patient clinical data) to both the in-memory SyncDebugLog and directly to console.log (via SyncDebugLog.log). In React Native, console.log output is visible via ADB logcat and system log viewers. The project rules explicitly state patient data must never be logged at INFO level or above. Remove data: row from the log entry; the existing recordId, columns (just column names), and error.message are sufficient for diagnosis without exposing record content.

console.log(`[SyncDebug] ${message}`, data ?? '');
}

getLogs(): SyncDebugLogEntry[] {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Design & Architecture] suggestion

getLogs() is defined but never called anywhere in this PR. SyncErrorDisplay calls formatLogs() directly. Either remove getLogs() (formatting is a display concern and belongs in the component anyway) or invert it: have the component call getLogs() and format the entries itself, and drop formatLogs(). Right now both methods exist with overlapping purposes, and one is dead code.

@@ -1,5 +1,6 @@
export * from './MobileSyncManager';
export * from './CentralServerConnection';
export * from './SyncDebugLog';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Design & Architecture] suggestion

SyncDebugLog is a debug utility that shouldn't be part of the sync service's public API. Exporting it from index.ts means any consumer of the sync module can depend on it, turning an internal debugging aid into a stable interface. If SyncErrorDisplay needs it, import directly from the source file ('~/services/sync/SyncDebugLog') rather than promoting it to public API.

import { StyledText, StyledView } from '../../styled/common';
import { theme } from '../../styled/theme';
import * as overrides from '/root/serverOverrides.json';
// TEMPORARY: server overrides commented out to allow normal server select in dev builds

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Design & Architecture] suggestion

The server overrides import and usage are commented out with "TEMPORARY" markers rather than removed or placed behind a proper flag. Commented-out code is opaque about intent — it's unclear when or whether it will be re-enabled, and the import * as overrides from '/root/serverOverrides.json' line (even commented) signals a path that only exists on specific devices. If this is being disabled for the release, delete the dead code; if it needs to return, use a build-time flag or a config value instead of re-introducing file-system-path imports.

if (this.logs.length > MAX_LOGS) {
this.logs.shift();
}
console.log(`[SyncDebug] ${message}`, data ?? '');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Performance] suggestion

Every call to SyncDebugLog.log() unconditionally invokes console.log. This method is called in hot sync paths — saveChangesForModel fires once per model per batch during both initial and incremental sync. In React Native, console.log is synchronous and non-trivial (it serialises arguments and bridges to the JS thread). On an initial sync with many models and batches, this compounds. Consider gating on a debug flag (__DEV__ or an explicit env var) so it does not run in production/release builds.

await repository.insert(row);
} catch (error: any) {
throw new Error(`Insert failed with '${error.message}', recordId: ${row.id}`);
SyncDebugLog.log(`Insert failed for ${tableName}`, {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Performance] suggestion

The insert-failure log stores the entire row object (data: row) in the in-memory SyncDebugLog buffer. Healthcare sync records can be large (nested JSON fields, blobs, etc.). If a batch fails completely and triggers row-by-row fallback, up to chunkSize log entries are queued — each holding a full record object. Combined with the MAX_LOGS=100 cap using Array.shift() (O(n) on every overflow), this can hold large live objects in memory that the GC cannot collect for the lifetime of the sync session. Consider logging only recordId and error rather than the complete row for the per-row failure case.

@review-hero

review-hero Bot commented Jun 24, 2026

Copy link
Copy Markdown

🦸 Review Hero Summary
9 agents reviewed this PR | 1 critical | 5 suggestions | 1 nitpick | Filtering: consensus 3 voters, 6 below threshold

Below consensus threshold (6 unique issues not confirmed by majority)
Location Agent Severity Comment
packages/mobile/App/services/sync/MobileSyncManager.ts:205 Bugs & Correctness suggestion SyncDebugLog.clear() is called unconditionally at the start of performSync(), but the guard for this.isSyncing returns early before reaching this line (looking at the broader method structure...
packages/mobile/App/services/sync/SyncDebugLog.ts:19 Performance nitpick Using Array.shift() to evict the oldest entry when the buffer is full is O(n) — it moves every remaining element down by one index. For MAX_LOGS=100 this is negligible, but a ring-buffer or a simpl...
packages/mobile/App/services/sync/SyncDebugLog.ts:31 Bugs & Correctness critical JSON.stringify(entry.data) in formatLogs() will throw a TypeError if any logged data contains a circular reference. TypeORM entity objects (like the row passed in executePreparedQuery) can ca...
packages/mobile/App/services/sync/utils/executePreparedQuery.ts:52 Bugs & Correctness suggestion The row-by-row fallback uses Promise.all, so all rows in the chunk are attempted in parallel. If the first failing row throws, the other repository.insert calls are still in-flight — their resu...
packages/mobile/App/ui/components/SyncErrorDisplay.tsx:22 Design & Architecture suggestion SyncErrorDisplay now mixes user-facing error display with raw developer debug logs rendered inline. These are different responsibilities — one is UI for end users, the other is a debugging tool for...
packages/mobile/App/ui/components/SyncErrorDisplay.tsx:38 Bugs & Correctness suggestion SyncDebugLog.formatLogs() is called inside the SYNC_ERROR event handler. Because SyncDebugLog is a module-level singleton and clear() is only called at the start of the next sync (in `syn...

Nitpicks

File Line Agent Comment
packages/mobile/App/services/sync/utils/saveIncomingChanges.ts 101 Performance saveChangesFromMemory is the inner loop of every batch during initial sync (called from saveChangesFromSnapshot via chunk iteration). The .map() that builds the models summary for the log entry allocates a new array on every batch. This is minor, but since this function is explicitly memory...
Local fix prompt (copy to your coding agent)

Fix these issues identified on the pull request. One commit per issue fixed.


packages/mobile/App/services/sync/utils/executePreparedQuery.ts:61: Logging data: row on insert failure writes the full row payload (which can include patient clinical data) to both the in-memory SyncDebugLog and directly to console.log (via SyncDebugLog.log). In React Native, console.log output is visible via ADB logcat and system log viewers. The project rules explicitly state patient data must never be logged at INFO level or above. Remove data: row from the log entry; the existing recordId, columns (just column names), and error.message are sufficient for diagnosis without exposing record content.


packages/mobile/App/services/sync/SyncDebugLog.ts:25: getLogs() is defined but never called anywhere in this PR. SyncErrorDisplay calls formatLogs() directly. Either remove getLogs() (formatting is a display concern and belongs in the component anyway) or invert it: have the component call getLogs() and format the entries itself, and drop formatLogs(). Right now both methods exist with overlapping purposes, and one is dead code.


packages/mobile/App/services/sync/index.ts:3: SyncDebugLog is a debug utility that shouldn't be part of the sync service's public API. Exporting it from index.ts means any consumer of the sync module can depend on it, turning an internal debugging aid into a stable interface. If SyncErrorDisplay needs it, import directly from the source file ('~/services/sync/SyncDebugLog') rather than promoting it to public API.


packages/mobile/App/ui/components/ServerSelectorField/ServerSelector.tsx:8: The server overrides import and usage are commented out with "TEMPORARY" markers rather than removed or placed behind a proper flag. Commented-out code is opaque about intent — it's unclear when or whether it will be re-enabled, and the import * as overrides from '/root/serverOverrides.json' line (even commented) signals a path that only exists on specific devices. If this is being disabled for the release, delete the dead code; if it needs to return, use a build-time flag or a config value instead of re-introducing file-system-path imports.


packages/mobile/App/services/sync/SyncDebugLog.ts:22: Every call to SyncDebugLog.log() unconditionally invokes console.log. This method is called in hot sync paths — saveChangesForModel fires once per model per batch during both initial and incremental sync. In React Native, console.log is synchronous and non-trivial (it serialises arguments and bridges to the JS thread). On an initial sync with many models and batches, this compounds. Consider gating on a debug flag (__DEV__ or an explicit env var) so it does not run in production/release builds.


packages/mobile/App/services/sync/utils/executePreparedQuery.ts:57: The insert-failure log stores the entire row object (data: row) in the in-memory SyncDebugLog buffer. Healthcare sync records can be large (nested JSON fields, blobs, etc.). If a batch fails completely and triggers row-by-row fallback, up to chunkSize log entries are queued — each holding a full record object. Combined with the MAX_LOGS=100 cap using Array.shift() (O(n) on every overflow), this can hold large live objects in memory that the GC cannot collect for the lifetime of the sync session. Consider logging only recordId and error rather than the complete row for the per-row failure case.


packages/mobile/App/services/sync/utils/saveIncomingChanges.ts:101: saveChangesFromMemory is the inner loop of every batch during initial sync (called from saveChangesFromSnapshot via chunk iteration). The .map() that builds the models summary for the log entry allocates a new array on every batch. This is minor, but since this function is explicitly memory-sensitive (there is a forceGC() call in prepareChangesForModels for this reason), the extra allocation — and keeping modelChanges live until the log entry is consumed — is a small but avoidable pressure. The log could be moved outside of the hot path or the map deferred until the log is actually needed.

@github-actions

github-actions Bot commented Jun 24, 2026

Copy link
Copy Markdown

Android builds 📱

@edmofro

edmofro commented Jun 24, 2026

Copy link
Copy Markdown
Member

You know it seems not a bad idea to have some version of this service persist permanently, but instead of displaying the logs through the app it sends them up to central (or canopy?) any time it hits a sync error. Acknowledging scope creep here, just consider it when you are secondary next week.
cc- @passcod re. the option for mobile endpoints to report to Canopy
cc- @rohan-bes in case there are potential self-healing benefits (e.g. the same pattern as the FK failures on facility server)

Cherry-pick fix from #10008 into the debug branch. When batch insert
fails, retry per-row with raw SQL instead of repository.insert so
@RelationId-backed FK columns are preserved. Keeps SyncDebugLog logging
from the debug work.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

🍹 up on tamanu-on-k8s/bes/tamanu-on-k8s/debug-no-issue-mobile-sync-debug-logs-2-54

Pulumi report
   Updating (debug-no-issue-mobile-sync-debug-logs-2-54)

View Live: https://app.pulumi.com/bes/tamanu-on-k8s/debug-no-issue-mobile-sync-debug-logs-2-54/updates/3

Downloading plugin random-4.19.0: starting
Downloading plugin random-4.19.0: done
Installing plugin random-4.19.0: starting
Installing plugin random-4.19.0: done

@ Updating....
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running 
@ Updating....
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read pulumi:pulumi:StackReference bes/k8s-core/tamanu-internal-main
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read pulumi:pulumi:StackReference bes/k8s-core/tamanu-internal-main
@ Updating....
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read pulumi:pulumi:StackReference bes/core/tamanu-internal
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read kubernetes:core/v1:Namespace tamanu-debug-no-issue-mobile-sync-debug-logs-2-54
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read pulumi:pulumi:StackReference bes/core/tamanu-internal
@ Updating.....
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read kubernetes:core/v1:Namespace tamanu-debug-no-issue-mobile-sync-debug-logs-2-54
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Waiting for central-db...
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Waiting for facility-1-db...
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Waiting for facility-2-db...
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read kubernetes:core/v1:ConfigMap actual-provisioning
~  kubernetes:apps/v1:Deployment patient-portal-web updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment facility-1-web updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment central-web updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment facility-2-web updating (0s) [diff: ~spec]
@ Updating....
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running read kubernetes:core/v1:ConfigMap actual-provisioning
++ kubernetes:batch/v1:Job central-migrator creating replacement (0s) [diff: ~spec]
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Secret facility-2-db-superuser not found or not ready: Error: HTTP-Code: 404
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Message: Unknown API Status Code!
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Body: "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"secrets \\\"facility-2-db-superuser\\\" not found\",\"reason\":\"NotFound\",\"details\":{\"name\":\"facility-2-db-superuser\",\"kind\":\"secrets\"},\"code\":404}
"
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Headers: {"audit-id":"203fb214-0911-4b23-8d93-96beb9f48595","cache-control":"no-cache, private","connection":"close","content-length":"220","content-type":"application/json","date":"Thu, 25 Jun 2026 00:42:06 GMT","x-kubernetes-pf-flowschema-uid":"3fb296fc-e46b-45d1-9306-057e37ddd229","x-kubernetes-pf-prioritylevel-uid":"feccf24d-a074-4fa8-aa6f-db82477fc2f5"}
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Secret central-db-superuser not found or not ready: Error: HTTP-Code: 404
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Message: Unknown API Status Code!
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Body: "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"secrets \\\"central-db-superuser\\\" not found\",\"reason\":\"NotFound\",\"details\":{\"name\":\"central-db-superuser\",\"kind\":\"secrets\"},\"code\":404}
"
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Headers: {"audit-id":"58ce67fb-b202-4454-b54d-387f875bd184","cache-control":"no-cache, private","connection":"close","content-length":"214","content-type":"application/json","date":"Thu, 25 Jun 2026 00:42:06 GMT","x-kubernetes-pf-flowschema-uid":"3fb296fc-e46b-45d1-9306-057e37ddd229","x-kubernetes-pf-prioritylevel-uid":"feccf24d-a074-4fa8-aa6f-db82477fc2f5"}
++ kubernetes:batch/v1:Job central-migrator creating replacement (0s) [diff: ~spec]; 
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Secret facility-1-db-superuser not found or not ready: Error: HTTP-Code: 404
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Message: Unknown API Status Code!
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Body: "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"secrets \\\"facility-1-db-superuser\\\" not found\",\"reason\":\"NotFound\",\"details\":{\"name\":\"facility-1-db-superuser\",\"kind\":\"secrets\"},\"code\":404}
"
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54 running Headers: {"audit-id":"2aa4dace-8a8d-46a5-ae63-360b8b165d36","cache-control":"no-cache, private","connection":"close","content-length":"220","content-type":"application/json","date":"Thu, 25 Jun 2026 00:42:07 GMT","x-kubernetes-pf-flowschema-uid":"3fb296fc-e46b-45d1-9306-057e37ddd229","x-kubernetes-pf-prioritylevel-uid":"feccf24d-a074-4fa8-aa6f-db82477fc2f5"}
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (0s) [diff: ~spec]
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (0s) [diff: ~spec]
@ Updating....
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (0s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (0s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job central-migrator creating replacement (1s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-migrator-e08a116a" to start
++ kubernetes:batch/v1:Job central-migrator creating replacement (1s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-migrator-e08a116a" to succeed (Active: 1 | Succeeded: 0 | Failed: 0)
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (0s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-migrator-0ffeb977" to start
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (0s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-migrator-0ffeb977" to succeed (Active: 1 | Succeeded: 0 | Failed: 0)
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (0s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-migrator-4fd13d85" to start
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (0s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-migrator-4fd13d85" to succeed (Active: 1 | Succeeded: 0 | Failed: 0)
@ Updating....
~  kubernetes:apps/v1:Deployment facility-1-web updating (2s) [diff: ~spec]; Waiting for app ReplicaSet to be available (0/1 Pods available)
~  kubernetes:apps/v1:Deployment central-web updating (2s) [diff: ~spec]; Waiting for app ReplicaSet to be available (0/1 Pods available)
~  kubernetes:apps/v1:Deployment patient-portal-web updating (2s) [diff: ~spec]; Waiting for app ReplicaSet to be available (0/1 Pods available)
~  kubernetes:apps/v1:Deployment facility-2-web updating (2s) [diff: ~spec]; Waiting for app ReplicaSet to be available (0/1 Pods available)
@ Updating.............
~  kubernetes:apps/v1:Deployment central-web updating (12s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-web-e7fd7963-69d5c87d95-tzqdk]: containers with unready status: [http]
~  kubernetes:apps/v1:Deployment patient-portal-web updating (12s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/patient-portal-web-98819443-949bb5bc4-r6rdt]: containers with unready status: [http]
@ Updating............
~  kubernetes:apps/v1:Deployment patient-portal-web updating (21s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment patient-portal-web updating (21s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment patient-portal-web updated (21s) [diff: ~spec]; 
@ Updating....
~  kubernetes:apps/v1:Deployment facility-1-web updating (22s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-web-8a71c5f0-588c4d7f7c-8jwgx]: containers with unready status: [http]
~  kubernetes:apps/v1:Deployment facility-2-web updating (22s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-web-29b6f9e1-8556647858-ldl52]: containers with unready status: [http]
~  kubernetes:apps/v1:Deployment central-web updating (23s) [diff: ~spec]; Waiting for app ReplicaSet to be available (1/2 Pods available)
@ Updating....
~  kubernetes:apps/v1:Deployment facility-1-web updating (23s) [diff: ~spec]; Waiting for app ReplicaSet to be available (1/2 Pods available)
~  kubernetes:apps/v1:Deployment facility-2-web updating (23s) [diff: ~spec]; Waiting for app ReplicaSet to be available (1/2 Pods available)
@ Updating...........
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (30s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-migrator-0ffeb977-4q8bv]: Container "migrator" completed with exit code 0
++ kubernetes:batch/v1:Job central-migrator creating replacement (31s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-migrator-e08a116a-bv4zr]: Container "migrator" completed with exit code 0
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (30s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-migrator-4fd13d85-7jfq8]: Container "migrator" completed with exit code 0
@ Updating....
~  kubernetes:apps/v1:Deployment facility-1-web updating (32s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-web-8a71c5f0-588c4d7f7c-znllw]: containers with unready status: [http]
~  kubernetes:apps/v1:Deployment central-web updating (32s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-web-e7fd7963-69d5c87d95-d4dl8]: containers with unready status: [http]
~  kubernetes:apps/v1:Deployment facility-2-web updating (32s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-web-29b6f9e1-8556647858-ct66x]: containers with unready status: [http]
@ Updating....
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (32s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-migrator-0ffeb977" to succeed (Active: 0 | Succeeded: 0 | Failed: 0)
++ kubernetes:batch/v1:Job central-migrator creating replacement (33s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-migrator-e08a116a" to succeed (Active: 0 | Succeeded: 0 | Failed: 0)
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (32s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-2-migrator-0ffeb977" to succeed (Active: 0 | Succeeded: 1 | Failed: 0)
++ kubernetes:batch/v1:Job facility-2-migrator creating replacement (32s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job facility-2-migrator created replacement (32s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job facility-2-migrator replacing (0s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job facility-2-migrator replaced (0.00s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (32s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-migrator-4fd13d85" to succeed (Active: 0 | Succeeded: 0 | Failed: 0)
++ kubernetes:batch/v1:Job central-migrator creating replacement (33s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-migrator-e08a116a" to succeed (Active: 0 | Succeeded: 1 | Failed: 0)
++ kubernetes:batch/v1:Job central-migrator creating replacement (33s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job central-migrator created replacement (33s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (32s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/facility-1-migrator-4fd13d85" to succeed (Active: 0 | Succeeded: 1 | Failed: 0)
++ kubernetes:batch/v1:Job facility-1-migrator creating replacement (32s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-sync updating (0s) [diff: ~spec]
++ kubernetes:batch/v1:Job facility-1-migrator created replacement (32s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-tasks updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment facility-2-api updating (0s) [diff: ~spec]
+- kubernetes:batch/v1:Job facility-1-migrator replacing (0s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job central-migrator replacing (0s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job facility-1-migrator replaced (0.00s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job central-migrator replaced (0.01s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job central-provisioner creating replacement (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment facility-1-tasks updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment facility-1-api updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment facility-1-sync updating (0s) [diff: ~spec]
@ Updating....
++ kubernetes:batch/v1:Job central-provisioner creating replacement (0s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job central-provisioner creating replacement (0s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-provisioner-3cc2ba69" to start
++ kubernetes:batch/v1:Job central-provisioner creating replacement (0s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-provisioner-3cc2ba69" to succeed (Active: 1 | Succeeded: 0 | Failed: 0)
~  kubernetes:apps/v1:Deployment facility-2-sync updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-2-sync updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-sync updated (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-tasks updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-2-tasks updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-tasks updated (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-api updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-2-api updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-api updated (1s) [diff: ~spec]; 
@ Updating....
~  kubernetes:apps/v1:Deployment facility-1-tasks updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-1-tasks updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-1-tasks updated (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-1-sync updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-1-sync updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-1-api updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-1-api updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-1-sync updated (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-1-api updated (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-web updating (35s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-2-web updating (35s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-2-web updated (35s) [diff: ~spec]; 
@ Updating....
~  kubernetes:apps/v1:Deployment central-web updating (36s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment central-web updating (36s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-web updated (36s) [diff: ~spec]; 
@ Updating.....
~  kubernetes:apps/v1:Deployment facility-1-web updating (39s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment facility-1-web updating (39s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment facility-1-web updated (39s) [diff: ~spec]; 
@ Updating.......
++ kubernetes:batch/v1:Job central-provisioner creating replacement (8s) [diff: ~spec]; warning: [Pod tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-provisioner-3cc2ba69-wqqf9]: Container "provisioner" completed with exit code 0
@ Updating....
++ kubernetes:batch/v1:Job central-provisioner creating replacement (9s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-provisioner-3cc2ba69" to succeed (Active: 0 | Succeeded: 0 | Failed: 0)
++ kubernetes:batch/v1:Job central-provisioner creating replacement (9s) [diff: ~spec]; Waiting for Job "tamanu-debug-no-issue-mobile-sync-debug-logs-2-54/central-provisioner-3cc2ba69" to succeed (Active: 0 | Succeeded: 1 | Failed: 0)
++ kubernetes:batch/v1:Job central-provisioner creating replacement (9s) [diff: ~spec]; 
++ kubernetes:batch/v1:Job central-provisioner created replacement (9s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job central-provisioner replacing (0s) [diff: ~spec]; 
+- kubernetes:batch/v1:Job central-provisioner replaced (0.00s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-fhir-refresh updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment central-api updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment central-tasks updating (0s) [diff: ~spec]
~  kubernetes:apps/v1:Deployment central-fhir-resolver updating (0s) [diff: ~spec]
@ Updating....
~  kubernetes:apps/v1:Deployment central-fhir-refresh updating (0s) [diff: ~spec]; Waiting for app ReplicaSet to be available (0/1 Pods available)
~  kubernetes:apps/v1:Deployment central-api updating (0s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment central-api updating (0s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-api updated (0.99s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-fhir-resolver updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment central-fhir-resolver updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-fhir-resolver updated (1s) [diff: ~spec]; 
@ Updating....
~  kubernetes:apps/v1:Deployment central-tasks updating (1s) [diff: ~spec]; warning: Replicas scaled to 0 for Deployment "central-tasks-d5ae1327"
~  kubernetes:apps/v1:Deployment central-tasks updating (1s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment central-tasks updating (1s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-tasks updated (1s) [diff: ~spec]; 
@ Updating.....
~  kubernetes:apps/v1:Deployment central-fhir-refresh updating (4s) [diff: ~spec]; Deployment initialization complete
~  kubernetes:apps/v1:Deployment central-fhir-refresh updating (4s) [diff: ~spec]; 
~  kubernetes:apps/v1:Deployment central-fhir-refresh updated (4s) [diff: ~spec]; 
@ Updating....
-- kubernetes:batch/v1:Job central-provisioner deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job central-provisioner deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job central-provisioner deleted original (0.40s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job facility-1-migrator deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job facility-2-migrator deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job central-migrator deleting original (0s) [diff: ~spec]; 
@ Updating....
-- kubernetes:batch/v1:Job facility-1-migrator deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job facility-1-migrator deleted original (0.55s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job facility-2-migrator deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job facility-2-migrator deleted original (0.70s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job central-migrator deleting original (0s) [diff: ~spec]; 
-- kubernetes:batch/v1:Job central-migrator deleted original (0.72s) [diff: ~spec]; 
   pulumi:pulumi:Stack tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54  15 messages
Diagnostics:
 pulumi:pulumi:Stack (tamanu-on-k8s-debug-no-issue-mobile-sync-debug-logs-2-54):
   Secret facility-2-db-superuser not found or not ready: Error: HTTP-Code: 404
   Message: Unknown API Status Code!
   Body: "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"secrets \\\"facility-2-db-superuser\\\" not found\",\"reason\":\"NotFound\",\"details\":{\"name\":\"facility-2-db-superuser\",\"kind\":\"secrets\"},\"code\":404}
"
   Headers: {"audit-id":"203fb214-0911-4b23-8d93-96beb9f48595","cache-control":"no-cache, private","connection":"close","content-length":"220","content-type":"application/json","date":"Thu, 25 Jun 2026 00:42:06 GMT","x-kubernetes-pf-flowschema-uid":"3fb296fc-e46b-45d1-9306-057e37ddd229","x-kubernetes-pf-prioritylevel-uid":"feccf24d-a074-4fa8-aa6f-db82477fc2f5"}
   Secret central-db-superuser not found or not ready: Error: HTTP-Code: 404
   Message: Unknown API Status Code!
   Body: "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"secrets \\\"central-db-superuser\\\" not found\",\"reason\":\"NotFound\",\"details\":{\"name\":\"central-db-superuser\",\"kind\":\"secrets\"},\"code\":404}
"
   Headers: {"audit-id":"58ce67fb-b202-4454-b54d-387f875bd184","cache-control":"no-cache, private","connection":"close","content-length":"214","content-type":"application/json","date":"Thu, 25 Jun 2026 00:42:06 GMT","x-kubernetes-pf-flowschema-uid":"3fb296fc-e46b-45d1-9306-057e37ddd229","x-kubernetes-pf-prioritylevel-uid":"feccf24d-a074-4fa8-aa6f-db82477fc2f5"}
   Secret facility-1-db-superuser not found or not ready: Error: HTTP-Code: 404
   Message: Unknown API Status Code!
   Body: "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"secrets \\\"facility-1-db-superuser\\\" not found\",\"reason\":\"NotFound\",\"details\":{\"name\":\"facility-1-db-superuser\",\"kind\":\"secrets\"},\"code\":404}
"
   Headers: {"audit-id":"2aa4dace-8a8d-46a5-ae63-360b8b165d36","cache-control":"no-cache, private","connection":"close","content-length":"220","content-type":"application/json","date":"Thu, 25 Jun 2026 00:42:07 GMT","x-kubernetes-pf-flowschema-uid":"3fb296fc-e46b-45d1-9306-057e37ddd229","x-kubernetes-pf-prioritylevel-uid":"feccf24d-a074-4fa8-aa6f-db82477fc2f5"}

   Waiting for central-db...
   Waiting for facility-1-db...
   Waiting for facility-2-db...

Outputs:
   urls: {
       Central      : "https://central.debug-no-issue-mobile-sync-debug-logs-2-54.cd.tamanu.app"
       Facility- 1  : "https://facility-1.debug-no-issue-mobile-sync-debug-logs-2-54.cd.tamanu.app"
       Facility- 2  : "https://facility-2.debug-no-issue-mobile-sync-debug-logs-2-54.cd.tamanu.app"
       PatientPortal: "https://portal.debug-no-issue-mobile-sync-debug-logs-2-54.cd.tamanu.app"
   }

Resources:
   ~ 14 updated
   +-4 replaced
   18 changes. 70 unchanged

Duration: 56s

   

Mock SyncDebugLog and add getTableName to the test model stub so debug
logging in saveChangesForModel does not break existing unit tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

passcod commented Jun 25, 2026

Copy link
Copy Markdown
Member

Not saying it's impossible, but everything that talks to canopy has a trust relationship with it, and that's just not the case with mobile. So it implies adding on a third separate authentication mode to canopy; going via central is probably easier at this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants