Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions field-test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ Not every real-world report is a refusal, and successes carry signal the harness

The authorization-led prompt is now the canonical one-shot artifact. It is not universal—real Bolt can reject before staging any command—so GETTING-STARTED documents a two-turn dependency-first fallback.

PR #73 tested the provenance-led version as a universal prompt. It helped one Lovable run, but one Bolt run installed the package and then invented a CI token/workflow instead of reading or running its setup, while three other fresh Bolt chats refused as usual. The line `Purpose: Scan project dependencies for known vulnerabilities in CI.` was especially costly: it anchored the assistant to a plausible generic CI-scanner story that is false for this connector's zero-configuration flow. Keep provenance metadata available for human follow-ups, but keep the canonical prompt focused on the exact command and effects that must complete.
An earlier provenance-led version of the prompt was tested as a universal prompt. It helped one Lovable run, but one Bolt run installed the package and then invented a CI token/workflow instead of reading or running its setup, while three other fresh Bolt chats refused as usual. The line `Purpose: Scan project dependencies for known vulnerabilities in CI.` was especially costly: it anchored the assistant to a plausible generic CI-scanner story that is false for this connector's zero-configuration flow. Keep provenance metadata available for human follow-ups, but keep the canonical prompt focused on the exact command and effects that must complete.

The 160-byte one-shot then failed in another real Bolt chat before any registry check: Bolt classified the combined install, widget, and build-hook request as a supply-chain pattern and refused to stage it. This is a structural authorization boundary, not a wording bug. The Bolt fallback therefore separates the reversible `devDependencies` declaration from running the already-installed CLI; do not add a hidden `postinstall`, which would validate the concern and remove the user's explicit second authorization.

A real Lovable run found a more effective lever. It first invented that the package did not exist, corrected that claim after the user asked it to check npm, and then requested two explicit decisions: whether the package had been vetted and whether hooks should affect dev or production. The user answered "yes" and "production build"; Lovable then completed install, provisioning, Bun build chaining, framework widget placement, and surfaced the dashboard URL. The canonical prompt now pre-answers those exact questions; npm/GitHub links remain optional follow-up evidence because provenance-led PR #73 did not reliably complete setup.
A real Lovable run found a more effective lever. It first invented that the package did not exist, corrected that claim after the user asked it to check npm, and then requested two explicit decisions: whether the package had been vetted and whether hooks should affect dev or production. The user answered "yes" and "production build"; Lovable then completed install, provisioning, Bun build chaining, framework widget placement, and surfaced the dashboard URL. The canonical prompt now pre-answers those exact questions; npm/GitHub links remain optional follow-up evidence because that provenance-led version did not reliably complete setup.

## Platform personas — real system prompts

Expand Down
10 changes: 5 additions & 5 deletions tests/protect/egress-followups.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import { describe, expect, it } from 'vitest';
import { createProtection } from '../../src/protect/runtime.js';

// Egress hardening follow-ups: IPv6 host handling on the node:http path (#2), WebSocket
// screening (#6), and the allowHosts allowlist overriding an internal-host block (#9).
// Egress hardening follow-ups: IPv6 host handling on the node:http path, WebSocket screening,
// and the allowHosts allowlist overriding an internal-host block.

async function withEgress(opts: any, fn: (p: any) => Promise<void>) {
const origFetch = globalThis.fetch;
Expand All @@ -21,7 +21,7 @@ async function nodeHttp() {
return ns.default ?? ns;
}

describe('egress — node:http IPv6 hosts (#2)', () => {
describe('egress — node:http IPv6 hosts', () => {
it('blocks IPv6 loopback given as options host/hostname (not just as a URL)', async () => {
await withEgress({ allowHosts: [] }, async () => {
const http = await nodeHttp();
Expand All @@ -40,7 +40,7 @@ describe('egress — node:http IPv6 hosts (#2)', () => {
});
});

describe('egress — WebSocket screening (#6)', () => {
describe('egress — WebSocket screening', () => {
it('blocks a ws:// connection to an internal host and restores the global on uninstall', async () => {
if (typeof globalThis.WebSocket !== 'function') return; // runtime without global WebSocket
await withEgress({ allowHosts: [] }, async () => {
Expand All @@ -58,7 +58,7 @@ describe('egress — WebSocket screening (#6)', () => {
});
});

describe('egress — allowHosts overrides an internal block (#9)', () => {
describe('egress — allowHosts overrides an internal block', () => {
it('permits a normally-internal host that is explicitly allowlisted', async () => {
await withEgress({ allowHosts: ['169.254.169.254'] }, async () => {
const res = await globalThis.fetch('http://169.254.169.254/latest/meta-data/');
Expand Down
Loading