Skip to content

677 heartbeat reporting#678

Merged
alexanderkirtzel merged 9 commits into
mainfrom
677-heartbeat-reporting
Jun 18, 2026
Merged

677 heartbeat reporting#678
alexanderkirtzel merged 9 commits into
mainfrom
677-heartbeat-reporting

Conversation

@alexanderkirtzel

@alexanderkirtzel alexanderkirtzel commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

Release Notes

  • New Features
    • Added async/respond-first configuration for response-producing server sources (2xx now reflects acceptance when enabled).
    • Added per-destination circuit breakers plus out-of-band reportError handling with /ready degrading to 503 when error thresholds are exceeded.
    • Added durable error persistence and heartbeat “flush soon” behavior, plus richer telemetry frames and run-scoped source.trace/source.count.
  • Bug Fixes
    • Improved secret redaction in persisted errors and heartbeat payloads.
    • Improved BigQuery Storage resilience: connection errors are contained, writers self-heal, and failures route to DLQ.
  • Documentation
    • Updated event-model and destination error-handling guidance.

@alexanderkirtzel alexanderkirtzel linked an issue Jun 16, 2026 that may be closed by this pull request
@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR adds shared step-isolation contracts, collector error routing and telemetry framing, CLI degraded readiness and error persistence, BigQuery writer recovery, Express async response control, observe API schema updates, and secure ID/session length changes.

Changes

Step isolation across shared runtime and platform code

Layer / File(s) Summary
Shared contracts and config fields
packages/core/src/types/..., packages/core/src/schemas/..., .changeset/..., skills/..., website/docs/..., packages/server/destinations/gcp/src/pubsub/examples/step.ts
reportError, breaker state/config, collector status maps, source async, run-scoped trace/count, and longer ID contracts are added to shared types, schemas, docs, examples, and release notes.
Collector breaker engine and reportError routing
packages/collector/src/breaker.ts, packages/collector/src/report-error.ts, packages/collector/src/destination.ts, packages/collector/src/source.ts, packages/collector/src/store.ts, packages/collector/src/transformer.ts, packages/collector/src/on.ts, packages/collector/src/index.ts, packages/collector/src/collector.ts, packages/collector/src/push.ts, packages/collector/src/__tests__/*, .changeset/collector-inevent-telemetry.md
Breaker state management, DLQ/error accounting, reportError wiring, and collector frame payload updates are implemented and covered by tests.
Collector inEvent/outEvent telemetry framing
packages/collector/src/push.ts, packages/collector/src/transformer.ts, packages/collector/src/destination.ts, packages/collector/src/__tests__/*, .changeset/collector-inevent-telemetry.md
Collector step emissions now carry inbound events and destination response metadata in the observed frame payloads, with matching telemetry observer tests.
CLI process guards, degraded readiness, and shutdown orchestration
packages/cli/src/runtime/health-server.ts, packages/cli/src/commands/run/pipeline.ts, packages/cli/src/commands/run/error-sink.ts, packages/cli/src/commands/run/index.ts, packages/cli/src/runtime/log-ring.ts, packages/cli/src/runtime/heartbeat.ts, packages/cli/src/__tests__/*, packages/cli/src/commands/run/__tests__/*
The health server gains degraded readiness, the pipeline registers guards earlier and wires out-of-band degradation tracking, durable JSONL error seeding/sinking is added, and heartbeat flushing now supports debounce and interval reporting.
BigQuery stream error containment and writer self-heal
packages/server/destinations/gcp/src/bigquery/*, packages/server/destinations/gcp/src/bigquery/__mocks__/*, packages/server/destinations/gcp/src/bigquery/__tests__/*, .changeset/step-isolation-gcp.md
BigQuery writer lifecycle tracks stream connections and connection errors, reopens broken writers, and updates mocks/tests for listener attachment and recovery behavior.
Express source respond-first config.async behavior
packages/server/sources/express/src/index.ts, packages/server/sources/express/src/schemas/settings.ts, packages/server/sources/express/src/types.ts, packages/server/sources/express/src/__tests__/*, .changeset/source-config-async.md
Express response timing moves to config.async, schema/docs/tests update to the new config shape, and POST/GET ordering tests reflect the changed acknowledge-before-delivery behavior.
Observe API schema/types and flow validation tests
packages/cli/openapi/spec.json, packages/cli/src/types/api.gen.d.ts, packages/cli/src/commands/validate/__tests__/flow-routes.test.ts
The observe-session request gains optional force, generated client types are updated, and validation coverage is added for required flow config.platform.
Secure ID generation and session ID length updates
packages/core/src/getId.ts, packages/core/src/getSpanId.ts, packages/core/src/getTraceId.ts, packages/core/src/hexId.ts, packages/core/src/index.ts, packages/core/src/__tests__/*, packages/web/sources/session/src/lib/sessionStorage.ts, packages/web/sources/session/src/lib/sessionWindow.ts, packages/web/sources/session/src/__tests__/*, .changeset/getid-secure-random.md, .changeset/run-scoped-trace.md
getId, getSpanId, and getTraceId now use crypto-backed sampling, and session IDs are lengthened and reflected in tests and docs.

Sequence Diagram(s)

sequenceDiagram
  participant runPipeline
  participant registerProcessGuards
  participant createOutOfBandErrorTracker
  participant HealthServer
  participant ErrorRing

  runPipeline->>registerProcessGuards: register uncaughtException / unhandledRejection
  registerProcessGuards-->>runPipeline: ProcessGuardHandle
  runPipeline->>createOutOfBandErrorTracker: configure threshold/window
  runPipeline->>registerProcessGuards: setOnOutOfBandError(record)
  ErrorRing-->>runPipeline: listener(isNew=true)
  runPipeline->>HealthServer: setDegraded(reason)
Loading
sequenceDiagram
  participant StreamConnection
  participant destinationBigQuery
  participant settings
  participant ensureWriter
  participant push

  StreamConnection-->>destinationBigQuery: onConnectionError(err)
  destinationBigQuery->>settings: writerBroken=true, lastStreamError=err
  push->>settings: check writerBroken
  push->>ensureWriter: reopenWriter()
  ensureWriter->>settings: swap handles, clear broken state
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • elbwalker/walkerOS#657: This PR also changes collector telemetry emission around step frames and event payloads, which overlaps with the inEvent/outEvent updates here.

Poem

🐇 I hop through errors, soft and bright,
With breakers blinking in the night.
flushSoon puffs up a tiny drum,
And BigQuery says, “I’ll rise, not succumb.”
Express twirls async like a kite,
While trace and count keep paths in sight.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 677-heartbeat-reporting

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/collector/src/report-error.ts`:
- Around line 159-162: The logger event key uses camelCase format
('reportError') but coding guidelines require space-separated format ('entity
action'). Change the first argument in the logger.error call from 'reportError'
to 'report error' to maintain consistent event naming across the codebase.

In `@packages/server/sources/express/src/types.ts`:
- Around line 31-36: The comment block describing the Ack contract in the
types.ts file contains stale wording about GET tracking pixel behavior. Update
the comment to accurately reflect that GET requests also respect the
config.async flag and will wait for delivery to settle when config.async is set
to false, rather than stating that GET always responds first regardless of the
flag. This should align with the actual implementation behavior found in
index.ts.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6dd5279f-31cf-4c54-a47d-83a42fc7dc7e

📥 Commits

Reviewing files that changed from the base of the PR and between 5ffd9d1 and 0a8a08b.

📒 Files selected for processing (61)
  • .changeset/source-config-async.md
  • .changeset/step-isolation-cli.md
  • .changeset/step-isolation-collector.md
  • .changeset/step-isolation-core.md
  • .changeset/step-isolation-gcp.md
  • packages/cli/src/__tests__/unit/commands/run-pipeline-frozen.test.ts
  • packages/cli/src/__tests__/unit/commands/run-pipeline-telemetry.test.ts
  • packages/cli/src/__tests__/unit/commands/run-pipeline.test.ts
  • packages/cli/src/__tests__/unit/runtime/health-server.test.ts
  • packages/cli/src/commands/run/__tests__/error-redaction.test.ts
  • packages/cli/src/commands/run/__tests__/error-sink.test.ts
  • packages/cli/src/commands/run/__tests__/pipeline.test.ts
  • packages/cli/src/commands/run/error-sink.ts
  • packages/cli/src/commands/run/index.ts
  • packages/cli/src/commands/run/pipeline.ts
  • packages/cli/src/runtime/__tests__/heartbeat.test.ts
  • packages/cli/src/runtime/__tests__/log-ring.test.ts
  • packages/cli/src/runtime/__tests__/runner.test.ts
  • packages/cli/src/runtime/health-server.ts
  • packages/cli/src/runtime/heartbeat.ts
  • packages/cli/src/runtime/log-ring.ts
  • packages/collector/src/__tests__/breaker.test.ts
  • packages/collector/src/__tests__/observerEmit.test.ts
  • packages/collector/src/__tests__/queue-bounds.test.ts
  • packages/collector/src/__tests__/report-error.test.ts
  • packages/collector/src/__tests__/store-cache-wrapper.observer.test.ts
  • packages/collector/src/__tests__/store.test.ts
  • packages/collector/src/__tests__/transformer-branch.test.ts
  • packages/collector/src/breaker.ts
  • packages/collector/src/collector.ts
  • packages/collector/src/destination.ts
  • packages/collector/src/index.ts
  • packages/collector/src/on.ts
  • packages/collector/src/report-error.ts
  • packages/collector/src/source.ts
  • packages/collector/src/store.ts
  • packages/collector/src/transformer.ts
  • packages/core/src/__tests__/emitStep.test.ts
  • packages/core/src/schemas/destination.ts
  • packages/core/src/schemas/source.ts
  • packages/core/src/types/collector.ts
  • packages/core/src/types/context.ts
  • packages/core/src/types/destination.ts
  • packages/core/src/types/index.ts
  • packages/core/src/types/source.ts
  • packages/server/destinations/gcp/src/bigquery/__mocks__/@google-cloud/bigquery-storage.ts
  • packages/server/destinations/gcp/src/bigquery/__tests__/bigquery-storage-mock.d.ts
  • packages/server/destinations/gcp/src/bigquery/__tests__/index.test.ts
  • packages/server/destinations/gcp/src/bigquery/__tests__/stepExamples.test.ts
  • packages/server/destinations/gcp/src/bigquery/__tests__/writer.test.ts
  • packages/server/destinations/gcp/src/bigquery/index.ts
  • packages/server/destinations/gcp/src/bigquery/push.ts
  • packages/server/destinations/gcp/src/bigquery/pushBatch.ts
  • packages/server/destinations/gcp/src/bigquery/types/index.ts
  • packages/server/destinations/gcp/src/bigquery/writer.ts
  • packages/server/sources/express/src/__tests__/cache-roundtrip.test.ts
  • packages/server/sources/express/src/__tests__/index.test.ts
  • packages/server/sources/express/src/index.ts
  • packages/server/sources/express/src/schemas/settings.ts
  • packages/server/sources/express/src/types.ts
  • skills/walkeros-understanding-destinations/SKILL.md
💤 Files with no reviewable changes (1)
  • packages/server/sources/express/src/schemas/settings.ts

Comment thread packages/collector/src/report-error.ts Outdated
Comment thread packages/server/sources/express/src/types.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/collector/src/push.ts (1)

2-10: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Prevent observer mutation from changing inbound collector.push payloads.

Line 238 stores the original inbound event by reference, and innerPush runs right after. Observer mutation can change what gets processed.

Suggested fix
 import {
+  clone,
   createIngest,
   emitStep,
   FatalError,
@@
-    inState.inEvent = event;
+    inState.inEvent = clone(event);

Also applies to: 238-239

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/collector/src/push.ts` around lines 2 - 10, At lines 238-239 in the
push.ts file, the original inbound event is being stored by reference, which
allows observer mutations to modify the stored value before innerPush runs.
Create a deep copy of the inbound event when storing it (rather than storing by
reference) to ensure that any mutations made by observers do not affect the
value that gets processed by the subsequent innerPush call. This will preserve
the original event state for proper processing.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/cli/src/commands/validate/__tests__/flow-routes.test.ts`:
- Around line 458-480: The positive-control test case for validateFlow is
incomplete and can pass even if unrelated validation errors are introduced. In
addition to the existing assertion that checks for platform-related errors using
result.errors.some(), add a new assertion to verify that result.valid is true.
This ensures the entire validation passes, not just that platform-specific
errors are absent, making the test more robust against future changes.

In `@packages/collector/src/destination.ts`:
- Around line 1454-1455: The code passes `processed.event` by reference directly
to `inState.inEvent` before calling `emitStep(collector, inState)`, allowing
observers to mutate the event object and affect downstream behavior. Create a
deep copy or clone of `processed.event` before assigning it to `inState.inEvent`
at line 1454, and apply the same fix to the similar assignments at lines
1485-1488 where event objects are assigned to state objects before being emitted
via `emitStep`.

---

Outside diff comments:
In `@packages/collector/src/push.ts`:
- Around line 2-10: At lines 238-239 in the push.ts file, the original inbound
event is being stored by reference, which allows observer mutations to modify
the stored value before innerPush runs. Create a deep copy of the inbound event
when storing it (rather than storing by reference) to ensure that any mutations
made by observers do not affect the value that gets processed by the subsequent
innerPush call. This will preserve the original event state for proper
processing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e2b37919-ca8e-4769-a708-697962843bc2

📥 Commits

Reviewing files that changed from the base of the PR and between 0a8a08b and d8aebd1.

📒 Files selected for processing (10)
  • .changeset/collector-inevent-telemetry.md
  • packages/cli/openapi/spec.json
  • packages/cli/src/commands/validate/__tests__/flow-routes.test.ts
  • packages/cli/src/types/api.gen.d.ts
  • packages/collector/src/__tests__/destination.observer.test.ts
  • packages/collector/src/__tests__/push.observer.test.ts
  • packages/collector/src/__tests__/transformer.observer.test.ts
  • packages/collector/src/destination.ts
  • packages/collector/src/push.ts
  • packages/collector/src/transformer.ts
✅ Files skipped from review due to trivial changes (2)
  • .changeset/collector-inevent-telemetry.md
  • packages/cli/openapi/spec.json
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/collector/src/transformer.ts

Comment on lines +458 to +480
it('accepts a flow whose config declares a valid platform', () => {
// Positive control: an otherwise identical flow with `config.platform`
// present produces no platform-related schema error.
const flow = {
version: 4,
flows: {
default: {
config: { platform: 'web' as const, url: 'https://example.com' },
sources: {
browser: {
package: '@walkeros/web-source-browser',
},
},
},
},
};

const result = validateFlow(flow);

expect(
result.errors.some((e) => /platform/i.test(e.message + ' ' + e.path)),
).toBe(false);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Strengthen the positive-control assertion in this test.

At Line 458, this case should also assert result.valid is true; otherwise it can still pass if unrelated validation errors are introduced later.

Suggested patch
   const result = validateFlow(flow);

+  expect(result.valid).toBe(true);
   expect(
     result.errors.some((e) => /platform/i.test(e.message + ' ' + e.path)),
   ).toBe(false);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
it('accepts a flow whose config declares a valid platform', () => {
// Positive control: an otherwise identical flow with `config.platform`
// present produces no platform-related schema error.
const flow = {
version: 4,
flows: {
default: {
config: { platform: 'web' as const, url: 'https://example.com' },
sources: {
browser: {
package: '@walkeros/web-source-browser',
},
},
},
},
};
const result = validateFlow(flow);
expect(
result.errors.some((e) => /platform/i.test(e.message + ' ' + e.path)),
).toBe(false);
});
it('accepts a flow whose config declares a valid platform', () => {
// Positive control: an otherwise identical flow with `config.platform`
// present produces no platform-related schema error.
const flow = {
version: 4,
flows: {
default: {
config: { platform: 'web' as const, url: 'https://example.com' },
sources: {
browser: {
package: '`@walkeros/web-source-browser`',
},
},
},
},
};
const result = validateFlow(flow);
expect(result.valid).toBe(true);
expect(
result.errors.some((e) => /platform/i.test(e.message + ' ' + e.path)),
).toBe(false);
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/validate/__tests__/flow-routes.test.ts` around
lines 458 - 480, The positive-control test case for validateFlow is incomplete
and can pass even if unrelated validation errors are introduced. In addition to
the existing assertion that checks for platform-related errors using
result.errors.some(), add a new assertion to verify that result.valid is true.
This ensures the entire validation passes, not just that platform-specific
errors are absent, making the test more robust against future changes.

Comment on lines +1454 to 1455
inState.inEvent = processed.event;
emitStep(collector, inState);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Avoid exposing mutable event references in emitted FlowState.

Line 1454 and Lines 1485-1488 pass processed.event by reference into observer frames. An observer can mutate that object and affect downstream behavior (especially before push on the in frame).

Suggested fix
-    inState.inEvent = processed.event;
+    inState.inEvent = clone(processed.event);
@@
-      outState.outEvent = processed.event;
+      outState.outEvent = clone(processed.event);

Also applies to: 1485-1488

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/collector/src/destination.ts` around lines 1454 - 1455, The code
passes `processed.event` by reference directly to `inState.inEvent` before
calling `emitStep(collector, inState)`, allowing observers to mutate the event
object and affect downstream behavior. Create a deep copy or clone of
`processed.event` before assigning it to `inState.inEvent` at line 1454, and
apply the same fix to the similar assignments at lines 1485-1488 where event
objects are assigned to state objects before being emitted via `emitStep`.

@alexanderkirtzel alexanderkirtzel linked an issue Jun 17, 2026 that may be closed by this pull request

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/src/getId.ts`:
- Around line 26-27: The buffer allocation in the getId function at the line
with `new Uint8Array(Math.ceil(need * 1.3) + 4)` can exceed the Web Crypto API's
per-call limit of 65,536 bytes, causing crypto.getRandomValues() to throw a
QuotaExceededError that is silently caught by secureBytes, resulting in a
fallback to Math.random. Cap the buffer size calculation using Math.min() to
ensure it never exceeds 65,536 bytes, preventing the silent RNG downgrade while
maintaining the current usage patterns.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 71cf67dd-a164-47bd-a6fe-aa77dc3b808d

📥 Commits

Reviewing files that changed from the base of the PR and between d8aebd1 and 31c6858.

📒 Files selected for processing (7)
  • .changeset/getid-secure-random.md
  • packages/core/src/__tests__/getId.test.ts
  • packages/core/src/getId.ts
  • packages/web/sources/session/src/__tests__/sessionStorage.test.ts
  • packages/web/sources/session/src/__tests__/stepExamples.test.ts
  • packages/web/sources/session/src/lib/sessionStorage.ts
  • packages/web/sources/session/src/lib/sessionWindow.ts
✅ Files skipped from review due to trivial changes (1)
  • .changeset/getid-secure-random.md

Comment on lines +26 to +27
const buffer = new Uint8Array(Math.ceil(need * 1.3) + 4);
if (!secureBytes(buffer)) break; // No secure source, drop to fallback below

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

What is the maximum byte length allowed per call to Web Crypto crypto.getRandomValues, and what exception is thrown when that limit is exceeded?

💡 Result:

The maximum byte length allowed per call to the Web Crypto API's crypto.getRandomValues method is 65,536 bytes [1][2][3]. When a TypedArray with a byte length exceeding this limit is passed to the method, it throws a QuotaExceededError DOMException [1][4][3].

Citations:


🏁 Script executed:

#!/bin/bash
# Find obvious large constant callsites that could trigger oversized secure buffers.
rg -nP --type=ts --type=tsx '\bgetId\s*\(\s*([5-9]\d{4,}|\d{6,})\b'

Repository: elbwalker/walkerOS

Length of output: 90


🏁 Script executed:

#!/bin/bash
# Locate and read the getId.ts file
find . -type f -name "getId.ts" 2>/dev/null

Repository: elbwalker/walkerOS

Length of output: 91


🏁 Script executed:

#!/bin/bash
# Search for the getId function and surrounding context in TypeScript files
rg -n "export function getId" --type ts -A 35

Repository: elbwalker/walkerOS

Length of output: 2396


🏁 Script executed:

#!/bin/bash
# Find the secureBytes function to understand exception handling
rg -n "secureBytes" --type ts -B 3 -A 10

Repository: elbwalker/walkerOS

Length of output: 1758


🏁 Script executed:

#!/bin/bash
# Search for calls to getId with potentially large arguments
rg -n "getId\s*\(" --type ts --type js -B 2 -A 2

Repository: elbwalker/walkerOS

Length of output: 10443


Cap getRandomValues request size to prevent silent RNG downgrade.

At Line 26, the buffer allocation can exceed the Web Crypto API's per-call limit of 65,536 bytes. When that happens, crypto.getRandomValues() throws a QuotaExceededError, which is caught by secureBytes at Line 27. This silent exception handling causes a fallback to Math.random, degrading entropy despite CSPRNG availability.

While current usage in the codebase calls getId with small lengths (max 16 bytes), this is a latent vulnerability that should be fixed proactively to prevent future issues.

Suggested fix
 const defaultCharset = '0123456789abcdefghijklmnopqrstuvwxyz';
+const MAX_RANDOM_VALUES_BYTES = 65536;
 
 export function getId(length = 6, charset: string = defaultCharset): string {
@@
     while (str.length < length) {
       const need = length - str.length;
-      const buffer = new Uint8Array(Math.ceil(need * 1.3) + 4);
+      const bufferSize = Math.min(
+        MAX_RANDOM_VALUES_BYTES,
+        Math.ceil(need * 1.3) + 4,
+      );
+      const buffer = new Uint8Array(bufferSize);
       if (!secureBytes(buffer)) break; // No secure source, drop to fallback below
       for (let i = 0; i < buffer.length && str.length < length; i++) {
         if (buffer[i] < limit) str += charset[buffer[i] % n];
       }
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const buffer = new Uint8Array(Math.ceil(need * 1.3) + 4);
if (!secureBytes(buffer)) break; // No secure source, drop to fallback below
const defaultCharset = '0123456789abcdefghijklmnopqrstuvwxyz';
const MAX_RANDOM_VALUES_BYTES = 65536;
export function getId(length = 6, charset: string = defaultCharset): string {
let str = '';
while (str.length < length) {
const need = length - str.length;
const bufferSize = Math.min(
MAX_RANDOM_VALUES_BYTES,
Math.ceil(need * 1.3) + 4,
);
const buffer = new Uint8Array(bufferSize);
if (!secureBytes(buffer)) break; // No secure source, drop to fallback below
for (let i = 0; i < buffer.length && str.length < length; i++) {
if (buffer[i] < limit) str += charset[buffer[i] % n];
}
}
// ... rest of function
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/getId.ts` around lines 26 - 27, The buffer allocation in
the getId function at the line with `new Uint8Array(Math.ceil(need * 1.3) + 4)`
can exceed the Web Crypto API's per-call limit of 65,536 bytes, causing
crypto.getRandomValues() to throw a QuotaExceededError that is silently caught
by secureBytes, resulting in a fallback to Math.random. Cap the buffer size
calculation using Math.min() to ensure it never exceeds 65,536 bytes, preventing
the silent RNG downgrade while maintaining the current usage patterns.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/collector/src/__tests__/handle.test.ts`:
- Around line 370-382: The deep equality assertion comparing viaEnrich and
viaCompose is flaky because both objects independently read the current time,
causing timestamp and timing fields to drift between them. In addition to
normalizing source.count to 0 (which is already done), you must also normalize
the time-derived fields (timestamp and timing) in both viaEnrich and viaCompose
to fixed values before performing the deep equality comparison to ensure
deterministic test behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d701361c-bbca-4777-8dea-28d69b473801

📥 Commits

Reviewing files that changed from the base of the PR and between 31c6858 and d1b41ca.

📒 Files selected for processing (30)
  • .changeset/run-scoped-trace.md
  • packages/cli/openapi/spec.json
  • packages/cli/src/types/api.gen.d.ts
  • packages/collector/src/__tests__/boundary-error.test.ts
  • packages/collector/src/__tests__/handle-createEvent.test.ts
  • packages/collector/src/__tests__/handle.test.ts
  • packages/collector/src/__tests__/inline-code.test.ts
  • packages/collector/src/__tests__/observerEmit.test.ts
  • packages/collector/src/__tests__/queue-bounds.test.ts
  • packages/collector/src/__tests__/run-trace.test.ts
  • packages/collector/src/__tests__/store-cache-wrapper.observer.test.ts
  • packages/collector/src/__tests__/store.test.ts
  • packages/collector/src/__tests__/transformer-branch.test.ts
  • packages/collector/src/__tests__/transformer-init-error.test.ts
  • packages/collector/src/__tests__/transformer.test.ts
  • packages/collector/src/collector.ts
  • packages/collector/src/handle.ts
  • packages/core/src/__tests__/emitStep.test.ts
  • packages/core/src/__tests__/getSpanId.test.ts
  • packages/core/src/__tests__/getTraceId.test.ts
  • packages/core/src/getSpanId.ts
  • packages/core/src/getTraceId.ts
  • packages/core/src/hexId.ts
  • packages/core/src/index.ts
  • packages/core/src/schemas/walkeros.ts
  • packages/core/src/types/collector.ts
  • packages/core/src/types/walkeros.ts
  • packages/server/destinations/gcp/src/pubsub/examples/step.ts
  • skills/walkeros-understanding-events/SKILL.md
  • website/docs/getting-started/event-model.mdx
✅ Files skipped from review due to trivial changes (11)
  • packages/core/src/hexId.ts
  • packages/core/src/getSpanId.ts
  • packages/collector/src/tests/boundary-error.test.ts
  • packages/collector/src/tests/transformer-init-error.test.ts
  • packages/collector/src/tests/transformer.test.ts
  • packages/core/src/tests/emitStep.test.ts
  • .changeset/run-scoped-trace.md
  • packages/core/src/schemas/walkeros.ts
  • website/docs/getting-started/event-model.mdx
  • packages/core/src/types/walkeros.ts
  • packages/cli/src/types/api.gen.d.ts
🚧 Files skipped from review as they are similar to previous changes (6)
  • packages/collector/src/collector.ts
  • packages/collector/src/tests/store.test.ts
  • packages/collector/src/tests/observerEmit.test.ts
  • packages/collector/src/tests/store-cache-wrapper.observer.test.ts
  • packages/collector/src/tests/queue-bounds.test.ts
  • packages/core/src/types/collector.ts

Comment on lines +370 to 382
const viaEnrich = enrichEvent(c, partial);
const viaCompose = createEvent(c, prepareEvent(c, partial));
// Both paths produce the same event. `source.count` is a per-run sequence
// that legitimately advances on each createEvent call, so compare it
// independently and assert deep equality on everything else.
expect(viaEnrich.source.count).toBe(1);
expect(viaCompose.source.count).toBe(2);
expect({ ...viaEnrich, source: { ...viaEnrich.source, count: 0 } }).toEqual(
{
...viaCompose,
source: { ...viaCompose.source, count: 0 },
},
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Stabilize this equality test by fixing time-derived fields.

viaEnrich and viaCompose each read current time independently, so timestamp/timing can drift and make this deep-equality assertion flaky even after normalizing source.count.

Proposed fix
     const partial: WalkerOS.DeepPartialEvent = {
       name: 'page view',
       id: 'fixed',
+      timestamp: 1_700_000_000_000,
+      timing: 12.34,
     };
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/collector/src/__tests__/handle.test.ts` around lines 370 - 382, The
deep equality assertion comparing viaEnrich and viaCompose is flaky because both
objects independently read the current time, causing timestamp and timing fields
to drift between them. In addition to normalizing source.count to 0 (which is
already done), you must also normalize the time-derived fields (timestamp and
timing) in both viaEnrich and viaCompose to fixed values before performing the
deep equality comparison to ensure deterministic test behavior.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Pre-release published (next)

Packages

Install: npm i @walkeros/core@next

🐳 Docker images published

  • walkeros/flow:4.2.1-next-1781715165983 (:next)
  • walkeros/cli:4.2.1-next-1781715165983 (:next)

Docker: docker pull walkeros/flow:next

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Stable release published

Packages

Install: npm i @walkeros/core@latest

🐳 Docker images published

  • walkeros/flow:4.2.1 (:latest)
  • walkeros/cli:4.2.1 (:latest)

Docker: docker pull walkeros/flow:latest

@alexanderkirtzel alexanderkirtzel merged commit 89360bf into main Jun 18, 2026
1 check passed
@alexanderkirtzel alexanderkirtzel deleted the 677-heartbeat-reporting branch June 18, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade to a collision-free deviceId for millions of devices heartbeat reporting

1 participant