Skip to content

initial commit for v5 harper upgrade#2

Open
BboyAkers wants to merge 5 commits into
mainfrom
v5-upgrade
Open

initial commit for v5 harper upgrade#2
BboyAkers wants to merge 5 commits into
mainfrom
v5-upgrade

Conversation

@BboyAkers

@BboyAkers BboyAkers commented May 7, 2026

Copy link
Copy Markdown
Member

Harper v4 -> v5 upgrade (Getting-Started-PubSub)

Completes the in-progress v5 upgrade on this branch. The Harper app lives in harper/ and is schema-only (tables Sensors and Topics exported over REST + MQTT/WS/SSE); there is no app JS, so there was no harperdb import/global footprint to migrate — the v4->v5 work is the dependency/CLI swap plus tooling.

Changes

  • Dependency: harper 5.0.11 -> 5.0.28 (latest). CLI scripts already swapped harperdb -> harper (run/dev/deploy) in the prior commits on this branch.
  • Integration tests (harper/integrationTests/) using @harperfast/integration-testing, covering the pub/sub paths:
    • mqtt.test.ts — MQTT subscriber receives a message published to Sensors/{id}; MQTT publish is persisted and readable over REST.
    • rest.test.ts — REST table-export backbone: PUT/GET a Sensors record and a Topics message.
    • A minimal component fixture (integrationTests/fixture/) holds only config.yaml + schema.graphql + package.json so the harness pre-installs the component without copying node_modules.
    • harperBinPath is resolved from the harper package main entry — harper's exports map only exposes ".", so harper/dist/bin/harper.js is not directly resolvable (documented harness escape hatch).
  • CI: .github/workflows/integration-tests.yml at repo root. Node matrix [22, 24, 26], actions pinned to commit hashes, working-directory: harper, cache-dependency-path: harper/package-lock.json.
  • Lockfile: committed harper/package-lock.json (the root .gitignore was ignoring it; added a negation) so npm ci is reproducible. Regenerated cleanly — includes ws native optional deps (bufferutil/utf-8-validate/node-gyp-build) so npm ci passes on Node 24/26, not just 22. All resolved URLs point at the public registry (no file:/.tgz).
  • TypeScript: pinned typescript to 5.9.3 (latest 6.x violates the >=4.8.4 <6.0.0 peer range required by the @harperdb/code-guidelines eslint toolchain). Added a base tsconfig.json and a small eslint override so typed linting resolves the project's tsconfig (the shared config hardcodes project: './tsconfig.json' relative to its own package dir).
  • Branding: HarperDB -> Harper in harper/README.md prose; fixed the stale v4 install command (npm install -g harperdb -> harper). Left @harperdb/code-guidelines and live docs.harperdb.io / github.com/HarperDB URLs intact.

Migration items

  • Import/global swap, Table.get() shape, frozen records, transaction/context, blob.save(), child-process spawning, install scripts, module-loader config: N/A — this is a schema-only component with no application JS/DB code.

Tests

  • npm run test:integration4/4 pass locally when run with --isolation=none (binds 127.0.0.1).
  • The default pooled run fails on this macOS dev box at the loopback bind (EADDRNOTAVAIL for 127.0.0.2+) because the machine has no loopback aliases configured — this is environmental, not a code issue. CI runs on ubuntu-latest, which supports the full 127.0.0.0/8 range, so the pooled run works there. Relying on the CI run for the pooled-mode test gate.

Flagged for a human

  • npm scope: package name is harper-mqtt-getting-started (unscoped). If it should move to the current Harper npm scope, that's a manual decision — not done here.

🤖 Generated with Claude Code

BboyAkers and others added 4 commits May 7, 2026 15:48
- Bump harper 5.0.11 -> 5.0.28 (latest)
- Add @harperfast/integration-testing + typescript (pinned 5.9.3 to
  satisfy typescript-eslint peer range) + mqtt dev deps
- Add integration tests for the pub/sub paths: MQTT publish/subscribe,
  MQTT->REST persistence, and the REST table-export backbone
- Resolve the harper CLI via harperBinPath (harper's exports map only
  exposes ".", so harper/dist/bin/harper.js is not resolvable)
- Add .github/workflows/integration-tests.yml (Node 22/24/26, pinned
  action hashes, working-directory: harper)
- Commit harper/package-lock.json (un-ignore it) with ws native optional
  deps so npm ci is reproducible across Node 22/24/26
- Add base tsconfig.json and fix the shared eslint config's tsconfig
  resolution for typed linting of the tests
- Branding: HarperDB -> Harper in harper/README.md prose; fix stale
  v4 install command (harperdb -> harper)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Comment thread harper/integrationTests/mqtt.test.ts Outdated
await teardownHarper(ctx);
});

test('subscriber receives a message published to Sensors/101', async () => {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: test name says Sensors/101 but the test actually uses topic Sensors/102

The test description is misleading — when this test fails, CI output will say "subscriber receives a message published to Sensors/101" but the failure will involve Sensors/102. The doc-comment on line 8 also references Sensors/101.

Suggested change
test('subscriber receives a message published to Sensors/101', async () => {
test('subscriber receives a message published to Sensors/102', async () => {


// Poll the REST surface until the upsert from the MQTT publish lands.
let body: Record<string, unknown> | undefined;
for (let attempt = 0; attempt < 20; attempt++) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flakiness risk: 5-second REST poll window may be too tight under CI load

The loop polls 20 times at 250 ms each (5 s total) for the MQTT-published record to appear over REST. Under a loaded CI runner, the Harper process may take longer than 5 s to flush the QoS-1 retained message to the database, causing a spurious failure.

Increasing to 40 attempts keeps the same 250 ms cadence but doubles the budget to 10 s, which is in line with the 15 s timeout used in the pub/sub delivery test above and is still well within a reasonable test time limit.

Suggested change
for (let attempt = 0; attempt < 20; attempt++) {
for (let attempt = 0; attempt < 40; attempt++) {

const require = createRequire(import.meta.url);
// harper's `exports` map only exposes ".", so 'harper/dist/bin/harper.js' is
// not resolvable; resolve the CLI from the package main entry instead.
const harperBinPath = resolve(dirname(require.resolve('harper')), 'bin/harper.js');

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleanup: harperBinPath resolution is duplicated verbatim in both test files

Both mqtt.test.ts (line 21) and rest.test.ts (line 19) contain the exact same three-liner:

const require = createRequire(import.meta.url);
const harperBinPath = resolve(dirname(require.resolve('harper')), 'bin/harper.js');

If the escape hatch logic ever changes (e.g. harper exposes an exports entry for the bin path), it will need to be updated in two places. Consider extracting this to a shared integrationTests/harperBin.ts helper and importing it from both test files.

Update the test name and doc-comment to reference Sensors/102, matching
the topic actually used in the test body.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@BboyAkers

Copy link
Copy Markdown
Member Author

Review follow-up (autonomous agent): Fixed blocking findings: corrected stale Sensors/101 references in the MQTT test — updated the test name (line 44) and the file-level doc-comment (line 5) to match the Sensors/102 topic actually used in the test body.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant