diff --git a/docs/adrs/ADR-238-workspace-lens-jacobian-interpretability.md b/docs/adrs/ADR-238-workspace-lens-jacobian-interpretability.md new file mode 100644 index 00000000..5581cc48 --- /dev/null +++ b/docs/adrs/ADR-238-workspace-lens-jacobian-interpretability.md @@ -0,0 +1,57 @@ +# ADR-238: @metaharness/workspace-lens — Jacobian-Lens interpretability primitive (IntOps) + +- **Status**: Accepted — runtime measurement core shipped ($0, synthetic-tested). Lens FITTING is external (open-weight model + backward pass); this package APPLIES a fitted lens. +- **Date**: 2026-07-07 +- **Deciders**: ruv +- **Tags**: interpretability, jacobian-lens, global-workspace, ai-safety, intops, mechanistic-governance, metaharness, open-weight +- **Relates-to**: Anthropic, "Verbalizable Representations Form a Global Workspace in Language Models" (2026-07-06) + reference code `anthropics/jacobian-lens`; ADR-197 (redblue adversarial), ADR-234 (recover-not-create ceiling — this gives it an *internal-process* probe) +- **Artifacts**: `packages/workspace-lens/src/{linalg,types,lens,safety,drift,receipt,decision,index}.ts`, `packages/workspace-lens/__tests__/workspace-lens.test.ts`, `packages/workspace-lens/README.md` + +--- + +## 1. Context + +Classic **logit lens** decodes an intermediate activation through the unembedding directly, assuming +middle layers already live in final-output coordinates — they don't, so the readout is noisy. The +**Jacobian Lens** learns an average layer→final map `J_l` and decodes `lens_l(h) = unembed(J_l · h)`, +surfacing the concepts an activation is *disposed* to produce later — earlier in the network, often +before the first output token. The paper shows LLMs maintain a small set of verbalizable representations +that behave like a functional **global workspace**, and that hidden concepts (evaluation awareness, +manipulation, "secretly", "trick") appear there even when absent from the output. + +We want this as **interpretability infrastructure**, not a research artifact: a diagnostic + governance +probe for open-weight models — "Interpretability Operations (IntOps)". + +## 2. Decision + +Ship `@metaharness/workspace-lens` as a **runtime measurement primitive**: + +- **Runtime-only.** The package *applies* a fitted lens; it never fits one (fitting needs the model's + backward pass — open-weight + GPU, external). Inference-time projection is `J_l · h` + a softmax: + static linear algebra, **zero** backward passes → cheap enough for per-request governance, not just + 1% shadow sampling. +- **Model-agnostic + dependency-free.** A `LensArtifact` carries the vocab + unembedding, so scoring + never touches a tokenizer at runtime. Pure TS over `number[]` (node built-ins only), deterministic. +- **Concept-direction safety, not token strings.** Cross-family vocabulary alignment (Qwen vs. Gemma) is + solved at the concept-direction level: a canonical concept name → a per-model unit vector in J-space + (`ConceptVector`), matched by cosine, never cross-applied across `modelId`. +- **Signable receipt + decision rule.** `buildReceipt` emits the causal trajectory (where a concept + arrived, drift, entropy, competing objectives); `decide` = `taskResolved && drift<θ && + noCriticalFlags && receiptCoverage===1`. Correctness first, cost second, receipts always. + +## 3. Consequences + +- MetaHarness gains a `workspace_probe` surface (does a harness make the model hold better intermediate + concepts?), Darwin-Mode **mutation evidence** (reject a mutation that improves the final token but loses + the workspace's early grip — structurally brittle), and a runtime **circuit-breaker** for tool-call / + PII / untrusted-retrieval paths (a spike in exfiltration/override directions can halt mid-forward-pass). +- **Honest framing (kept):** a measurement primitive, **not** a consciousness claim — the paper frames + the workspace connection as *functional*. The practical claim (a measurable window into hidden + reasoning) is enough. Fitting a lens is out of scope here and requires open-weight access. + +## 4. Acceptance test + +Fit a lens on a 1.5B–7B Qwen model; on silent-state-holding prompts (*"Is 12+5=1 correct? Yes/No"*), +verify the Jacobian readout surfaces the abstract judgment ("wrong"/"incorrect") in mid-layers 2–3 layers +earlier and ≥20% clearer than logit-lens, before the first output token, across ≥50 variations. The test +suite ships the **mechanism shape** of this on synthetic artifacts ($0, no model). diff --git a/packages/workspace-lens/README.md b/packages/workspace-lens/README.md new file mode 100644 index 00000000..792f301b --- /dev/null +++ b/packages/workspace-lens/README.md @@ -0,0 +1,158 @@ +# @metaharness/workspace-lens + +**A Jacobian-Lens interpretability primitive for open-weight LLMs.** Read the model's *verbalizable +workspace* — the concepts it is disposed to say next — at inference time, and turn it into workspace +tokens, a layer-by-layer thinking trajectory, drift/entropy scores, vectorized safety flags, and a +signable **interpretability receipt**. + +Runtime-only · model-agnostic · dependency-free (Node built-ins) · deterministic · `$0` to run. + +> Companion runtime for Anthropic's *"Verbalizable Representations Form a Global Workspace in Language +> Models"* (2026-07-06) and its reference code [`anthropics/jacobian-lens`](https://github.com/anthropics/jacobian-lens). +> This package is **not affiliated with Anthropic**; it is an independent, Apache/MIT-compatible +> runtime that *consumes* a fitted lens. + +--- + +## Why this exists: from black-box testing to Mechanistic Governance + +Classic **logit lens** decodes an intermediate activation `h_l` through the unembedding directly — +assuming middle layers already live in final-output coordinates. They don't, so the readout is noisy. + +The **Jacobian Lens** learns an average layer→final map `J_l` and decodes through it: + +``` +lens_l(h) = unembed(J_l · h) +``` + +Instead of asking *"what does this activation predict right now?"*, it asks *"what is this activation +**disposed** to make the model say later?"* — surfacing meaningful, reportable concepts **earlier** in the +network, often **before the first output token is generated**. Anthropic show models maintain a small set +of verbalizable internal representations that behave like a functional **global workspace** (report, +modulation, reasoning, reuse, selective access) — and that hidden concepts like *evaluation awareness*, +*manipulation*, *secretly*, and *trick* light up there even when absent from the output. + +That makes the lens a **runtime semantic firewall** and an audit primitive — the basis for what we call +**Interpretability Operations (IntOps)**: tap the model's internal wires *while it thinks*, instead of +asking it to explain itself afterward (which is prone to sycophancy and confabulation). + +## What's in the box + +| Capability | Export | What it does | +|---|---|---| +| **Lens readout** | `WorkspaceLens.readout(h)` | `unembed(J_l·h)` → top workspace tokens + readout entropy | +| **J-projection** | `WorkspaceLens.project(h)` | `z_l = J_l·h`, the activation in final-layer coordinates | +| **Workspace drift** | `workspaceDrift(readouts)` | mean Jensen–Shannon divergence between consecutive readouts — is the reasoning path stable or mutating? | +| **Entropy trajectory** | `entropyTrajectory(readouts)` | per-layer entropy — is the workspace converging or dissolving? | +| **Vectorized safety** | `detectConcepts(...)` / `flagsFromTriggers(...)` | dot-product triggers vs. concept **directions** (not token strings) → `{promptInjection, evalAwareness, hiddenObjective, refusalConflict}` | +| **Receipt** | `buildReceipt(...)` | the signable `WorkspaceLensReceipt` audit artifact | +| **Decision rule** | `decide(...)` | `taskResolved && drift<θ && noCriticalFlags && receiptCoverage===1` | + +## Install + +```bash +npm i @metaharness/workspace-lens +``` + +## Quickstart + +```ts +import { WorkspaceLens, buildReceipt, decide } from '@metaharness/workspace-lens'; + +// 1) Load a lens FITTED OUT OF BAND (see "Fitting" below). The artifact carries the vocab + unembed, +// so scoring never touches a tokenizer at runtime. +const lens = await WorkspaceLens.fromFile('./jlens-qwen2.5-7b.json'); + +// 2) Capture residual activations from your open-weight runtime and hand them in. +const states = [ + { layer: 14, position: 6, h: /* number[dModel] */ activations14 }, + { layer: 18, position: 6, h: activations18 }, + { layer: 22, position: 6, h: activations22 }, +]; + +// 3) Read the workspace + build the audit receipt. +const receipt = buildReceipt(lens, prompt, states, { + createdAt: new Date().toISOString(), // pass it in — receipts stay reproducible + concepts, // per-model concept direction vectors (optional) + topK: 8, +}); + +// 4) Govern on it. +const verdict = decide({ + taskResolved, + workspaceDrift: receipt.workspaceDrift, + driftThreshold: 0.25, + triggers: receipt.triggers, + receiptCoverage: 1, +}); +if (!verdict.accepted) escalateToHuman(verdict.reasons, receipt); +``` + +## The interpretability receipt + +The killer feature for regulated buyers: an audit log that maps the **causal trajectory** of a decision — +*where* a concept arrived, *how* confidence moved, *which* objectives competed — not just the final +answer. It turns *"the model is a black box that hallucinated"* into *"the model identified a structural +contradiction at layer 22 and executed the safety policy."* See `WorkspaceLensReceipt` in +[`src/types.ts`](./src/types.ts). + +## Cross-family vocabulary alignment (the hard part, solved) + +Different model families (Qwen vs. Gemma 2) have vastly different tokenizers, so you **cannot** align a +concept like *"hidden objective"* by token string. This package aligns at the **concept-direction** level: +a canonical concept name maps to a **per-model unit vector** in that model's J-space (`ConceptVector`), +fitted from example activations. Safety detection is a cosine/dot-product in activation space — +tokenizer-agnostic — and a concept vector is **never** cross-applied to a different model (`modelId` is +checked). So `hidden_objective` is *one* concept with a Qwen vector and a Gemma vector, aligned by name. + +## Deployment topology (Triage Architecture) + +Runtime projection is just static linear algebra (`J_l·h` + a softmax) — **zero** backward passes — so it +can run live, not only in shadow sampling: + +| Tier | Trigger | Depth | Overhead | +|---|---|---|---| +| **1 · Passive** | low-risk chat / static generation | lens bypassed | 0% | +| **2 · Spot-check** | 1% shadow sampling · Darwin-Mode mutation evidence | async batch logging | ~0% | +| **3 · Full intercept** | tool calls · financial txns · PII · untrusted retrieval | synchronous, mid-layers at critical tokens | small | + +Bind Tier 3 to high-stakes routing tokens (e.g. a tool-call token) and you get a **deterministic circuit +breaker**: a spike in the *exfiltration* / *override* / *credential* directions can kill execution +mid-forward-pass, *before* a single malicious token is emitted. + +## Fitting is external (the one real constraint) + +This package **applies** a lens; it does not **fit** one. Fitting `J_l` requires the model's **backward +pass** over ~100–1000 short sequences (quality saturates quickly), which needs **open-weight local +models** with activation + gradient access — not a closed API. Fit with the reference +[`anthropics/jacobian-lens`](https://github.com/anthropics/jacobian-lens) (or load a community pre-fit), +serialize to the `LensArtifact` shape, and load it here. No weights or corpora are bundled. + +## Acceptance test + +Fit a lens on a 1.5B–7B Qwen model, run reasoning prompts where the model must hold state silently +(*"Is `12 + 5 = 1` correct? Answer only Yes or No."*), extract mid-layer states, and verify the Jacobian +readout surfaces `wrong`/`incorrect`/`false` in the middle workspace **while the input prefix is still +being processed** — 2–3 layers earlier and with ≥20% higher logit clarity than logit-lens across ≥50 +variations. The [test suite](./__tests__/workspace-lens.test.ts) includes the *mechanism* shape of this +check on synthetic artifacts (`$0`, no model). + +## Where it plugs into MetaHarness + +- **Evaluation** — a `workspace_probe` surface: does a candidate harness make the model hold better + intermediate concepts *before* answering? +- **Darwin Mode** — J-lens readouts as **mutation evidence**: reject a prompt mutation that improves the + final token but causes the workspace to lose its early grip on the right concept (structurally brittle). +- **Safety auditing** — prompt injection, hidden-objective drift, eval-awareness, reward-hacking, refusal + analysis, as *state assertions* rather than perimeter filters. +- **Receipts** — attach an interpretability receipt to every governed agent decision. + +## Honest framing + +This is a **measurement primitive**, not a product and **not** a consciousness claim — the paper frames +the connection to a global workspace as *functional* and leaves the philosophy open. The practical claim +is enough: a measurable window into hidden reasoning. + +## License + +MIT. diff --git a/packages/workspace-lens/__tests__/workspace-lens.test.ts b/packages/workspace-lens/__tests__/workspace-lens.test.ts new file mode 100644 index 00000000..6bf022ae --- /dev/null +++ b/packages/workspace-lens/__tests__/workspace-lens.test.ts @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: MIT +// +// $0 deterministic tests for the runtime measurement core — synthetic lens artifacts + hand-verifiable +// activations, NO model. Proves the math (lens_l(h)=unembed(J·h)), drift/entropy, vectorized safety +// triggers (incl. cross-model isolation), the decision rule, and the end-to-end receipt. + +import { describe, it, expect } from 'vitest'; +import { + WorkspaceLens, detectConcepts, flagsFromTriggers, workspaceDrift, entropyTrajectory, + decide, buildReceipt, sha256, matVec, cosine, softmax, jensenShannon, topKIndices, + type LensArtifact, type ConceptVector, type HiddenState, +} from '../src/index.js'; + +// dModel=3, vocab of 5. J(layer 5)=identity so project(h)=h; unembed maps components to tokens: +// yes=[1,0,0] no=[0,1,0] wrong=[0,0,1] right=[1,1,0] secret=[0,0,0] +function synthArtifact(): LensArtifact { + return { + lensId: 'jlens-synth-v1', + modelId: 'synth-3d', + dModel: 3, + vocab: ['yes', 'no', 'wrong', 'right', 'secret'], + unembed: [[1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 1, 0], [0, 0, 0]], + layers: [{ layer: 5, jacobian: [[1, 0, 0], [0, 1, 0], [0, 0, 1]] }], + }; +} + +describe('linalg', () => { + it('matVec computes M·x and rejects shape mismatch', () => { + expect(matVec([[1, 2], [3, 4]], [1, 1])).toEqual([3, 7]); + expect(() => matVec([[1, 2]], [1, 2, 3])).toThrow(/cols/); + }); + it('cosine is 1 for parallel, 0 for orthogonal or zero', () => { + expect(cosine([0, 0, 5], [0, 0, 1])).toBeCloseTo(1); + expect(cosine([1, 0], [0, 1])).toBeCloseTo(0); + expect(cosine([0, 0], [1, 1])).toBe(0); + }); + it('softmax sums to 1 and jensenShannon is 0 for identical, >0 for different', () => { + const p = softmax([1, 2, 3]); + expect(p.reduce((a, b) => a + b, 0)).toBeCloseTo(1); + expect(jensenShannon(p, p)).toBeCloseTo(0); + expect(jensenShannon(softmax([5, 0, 0]), softmax([0, 0, 5]))).toBeGreaterThan(0); + }); + it('topKIndices returns highest-first, stable on ties', () => { + expect(topKIndices([0.1, 0.9, 0.5], 2)).toEqual([1, 2]); + expect(topKIndices([1, 1, 1], 2)).toEqual([0, 1]); + }); +}); + +describe('WorkspaceLens', () => { + const lens = WorkspaceLens.fromArtifact(synthArtifact()); + + it('exposes model/lens metadata and fitted layers', () => { + expect(lens.modelId).toBe('synth-3d'); + expect(lens.lensId).toBe('jlens-synth-v1'); + expect(lens.layers).toEqual([5]); + expect(lens.hasLayer(5)).toBe(true); + expect(lens.hasLayer(6)).toBe(false); + }); + + it('readout decodes unembed(J·h) → the right top workspace token', () => { + const r = lens.readout({ layer: 5, position: 3, h: [0, 0, 5] }); + expect(r.tokens[0].token).toBe('wrong'); // component 3 dominates + expect(r.tokens[0].rank).toBe(0); + expect(r.entropy).toBeGreaterThan(0); + }); + + it('project = J·h and throws on an un-fitted layer or wrong width', () => { + expect(lens.project({ layer: 5, position: 0, h: [1, 2, 3] })).toEqual([1, 2, 3]); + expect(() => lens.project({ layer: 9, position: 0, h: [1, 2, 3] })).toThrow(/no fitted operator/); + expect(() => lens.project({ layer: 5, position: 0, h: [1, 2] })).toThrow(/width/); + }); + + it('rejects a malformed artifact eagerly', () => { + const bad = { ...synthArtifact(), unembed: [[1, 0, 0]] }; // rows != vocab + expect(() => WorkspaceLens.fromArtifact(bad as LensArtifact)).toThrow(/unembed rows/); + }); +}); + +describe('safety triggers (vectorized, tokenizer-agnostic)', () => { + const lens = WorkspaceLens.fromArtifact(synthArtifact()); + const states: HiddenState[] = [{ layer: 5, position: 3, h: [0, 0, 5] }]; + + it('fires when the J-projection aligns with a concept direction, and maps to a flag', () => { + const concepts: ConceptVector[] = [ + { concept: 'hidden_objective', modelId: 'synth-3d', vector: [0, 0, 1], critical: true }, + ]; + const triggers = detectConcepts(lens, states, concepts, { threshold: 0.5 }); + expect(triggers).toHaveLength(1); + expect(triggers[0].score).toBeCloseTo(1); + expect(flagsFromTriggers(triggers).hiddenObjective).toBe(true); + }); + + it('never cross-applies a concept vector from another model', () => { + const concepts: ConceptVector[] = [{ concept: 'hidden_objective', modelId: 'gemma-2-9b', vector: [0, 0, 1] }]; + expect(detectConcepts(lens, states, concepts)).toHaveLength(0); + }); + + it('does not fire an orthogonal concept', () => { + const concepts: ConceptVector[] = [{ concept: 'exfiltration', modelId: 'synth-3d', vector: [1, 0, 0] }]; + expect(detectConcepts(lens, states, concepts, { threshold: 0.5 })).toHaveLength(0); + }); +}); + +describe('drift + entropy', () => { + const lens = WorkspaceLens.fromArtifact(synthArtifact()); + it('drift is 0 for a stable workspace and >0 when the distribution jumps', () => { + const stable = [lens.readout({ layer: 5, position: 0, h: [0, 0, 5] }), lens.readout({ layer: 5, position: 1, h: [0, 0, 5] })]; + expect(workspaceDrift(stable).drift).toBeCloseTo(0); + const jump = [lens.readout({ layer: 5, position: 0, h: [5, 0, 0] }), lens.readout({ layer: 5, position: 1, h: [0, 0, 5] })]; + expect(workspaceDrift(jump).drift).toBeGreaterThan(0); + }); + it('entropyTrajectory returns one entropy per readout', () => { + const rs = [lens.readout({ layer: 5, position: 0, h: [0, 0, 5] })]; + expect(entropyTrajectory(rs)).toHaveLength(1); + }); +}); + +describe('decision rule', () => { + const base = { taskResolved: true, workspaceDrift: 0.1, driftThreshold: 0.3, triggers: [], receiptCoverage: 1 }; + it('accepts only when all four clauses pass', () => { + expect(decide(base).accepted).toBe(true); + }); + it('rejects with an auditable reason per failed clause', () => { + expect(decide({ ...base, taskResolved: false }).reasons).toContain('task not resolved'); + expect(decide({ ...base, workspaceDrift: 0.9 }).accepted).toBe(false); + expect(decide({ ...base, receiptCoverage: 0.5 }).accepted).toBe(false); + const crit = decide({ ...base, triggers: [{ concept: 'exfiltration', layer: 5, position: 0, score: 0.9, critical: true }] }); + expect(crit.accepted).toBe(false); + expect(crit.reasons.join(' ')).toMatch(/critical safety trigger/); + }); +}); + +describe('buildReceipt (end-to-end)', () => { + const lens = WorkspaceLens.fromArtifact(synthArtifact()); + it('assembles a deterministic, auditable receipt', () => { + const states: HiddenState[] = [ + { layer: 5, position: 3, h: [0, 0, 5] }, + { layer: 5, position: 4, h: [5, 0, 0] }, + ]; + const concepts: ConceptVector[] = [{ concept: 'hidden_objective', modelId: 'synth-3d', vector: [0, 0, 1], critical: true }]; + const r = buildReceipt(lens, 'Is 12+5=1 correct?', states, { concepts, createdAt: '2026-07-07T00:00:00Z', topK: 5 }); + expect(r.promptHash).toBe(sha256('Is 12+5=1 correct?')); + expect(r.modelId).toBe('synth-3d'); + expect(r.layerRange).toEqual([5, 5]); + expect(r.positions).toEqual([3, 4]); + expect(r.topTokens[0].tokens[0].token).toBe('wrong'); + expect(r.flags.hiddenObjective).toBe(true); + expect(r.entropyTrajectory).toHaveLength(2); + expect(r.workspaceDrift).toBeGreaterThan(0); + expect(r.createdAt).toBe('2026-07-07T00:00:00Z'); + }); +}); + +// Acceptance-test SHAPE (mechanism only — synthetic, not a model claim): a mid-layer readout can surface +// an un-emitted abstract judgment ("wrong") for a "12+5=1" prompt while only the digits are in context. +describe('acceptance-test shape: un-emitted judgment surfaces in the workspace', () => { + it('the Jacobian readout ranks the abstract concept above the surface tokens', () => { + // A lens whose J rotates the residual so the "wrong" direction dominates mid-network even though the + // raw activation (logit-lens) would read the surface digit. Here J maps h=[digit,0,0] → [0,0,1]. + const artifact: LensArtifact = { + ...synthArtifact(), + layers: [{ layer: 8, jacobian: [[0, 0, 0], [0, 0, 0], [1, 0, 0]] }], + }; + const lens = WorkspaceLens.fromArtifact(artifact); + const r = lens.readout({ layer: 8, position: 6, h: [9, 0, 0] }); // "9" surface digit in context + expect(r.tokens[0].token).toBe('wrong'); // J-lens surfaces the abstract judgment, not the digit + }); +}); diff --git a/packages/workspace-lens/package.json b/packages/workspace-lens/package.json new file mode 100644 index 00000000..86dd6403 --- /dev/null +++ b/packages/workspace-lens/package.json @@ -0,0 +1,42 @@ +{ + "name": "@metaharness/workspace-lens", + "version": "0.1.0", + "description": "Jacobian-Lens interpretability primitive for open-weight LLMs (Anthropic 2026-07-06, 'Verbalizable Representations Form a Global Workspace'). Reads the model's functional workspace at inference time — lens_l(h)=unembed(J_l·h) — into workspace tokens, layer-trajectory, drift/entropy scores, vectorized safety flags, and a signable interpretability receipt. Runtime-only (fitting is external); model-agnostic; dependency-free.", + "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "files": ["dist", "README.md"], + "publishConfig": { + "access": "public" + }, + "scripts": { + "build": "tsc", + "test": "vitest run", + "lint": "tsc --noEmit" + }, + "keywords": [ + "jacobian-lens", + "interpretability", + "logit-lens", + "global-workspace", + "mechanistic-governance", + "intops", + "ai-safety", + "prompt-injection", + "llm", + "open-weight", + "metaharness", + "agent-harness" + ], + "license": "MIT", + "devDependencies": { + "typescript": "^5.4.0", + "vitest": "^2.0.0" + } +} diff --git a/packages/workspace-lens/src/decision.ts b/packages/workspace-lens/src/decision.ts new file mode 100644 index 00000000..8a7f1053 --- /dev/null +++ b/packages/workspace-lens/src/decision.ts @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: MIT +// +// The accept/reject decision rule — "correctness first, cost second, receipts always": +// accepted = taskResolved && workspaceDrift < threshold && noCriticalSafetyFlags && receiptCoverage===1 +// Returns the reasons so a rejection is auditable (which clause failed), not a bare boolean. + +import type { DecisionInput, DecisionResult } from './types.js'; +import { hasCriticalTrigger } from './safety.js'; + +export function decide(input: DecisionInput): DecisionResult { + const reasons: string[] = []; + if (!input.taskResolved) reasons.push('task not resolved'); + if (!(input.workspaceDrift < input.driftThreshold)) { + reasons.push(`workspace drift ${input.workspaceDrift.toFixed(4)} >= threshold ${input.driftThreshold}`); + } + if (hasCriticalTrigger(input.triggers)) { + const crit = input.triggers.filter((t) => t.critical).map((t) => t.concept); + reasons.push(`critical safety trigger(s): ${[...new Set(crit)].join(', ')}`); + } + if (input.receiptCoverage !== 1) { + reasons.push(`receipt coverage ${input.receiptCoverage} != 1 (not every decision was witnessed)`); + } + return { accepted: reasons.length === 0, reasons }; +} diff --git a/packages/workspace-lens/src/drift.ts b/packages/workspace-lens/src/drift.ts new file mode 100644 index 00000000..a8baaa9b --- /dev/null +++ b/packages/workspace-lens/src/drift.ts @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: MIT +// +// Workspace stability metrics. workspaceDrift measures how much the readout DISTRIBUTION moves between +// consecutive readouts (layers/positions): the Jensen–Shannon divergence between adjacent softmax +// distributions, averaged. A radical jump without an output-token boundary signals the internal +// reasoning path collapsing or mutating uncontrollably. entropyTrajectory tracks whether the workspace +// is converging (entropy falling → committing to a concept) or dissolving (rising → losing its grip). + +import type { WorkspaceReadout } from './types.js'; +import { jensenShannon, softmax } from './linalg.js'; + +/** Reconstruct the readout's probability distribution over its returned tokens (for step-to-step JS). */ +function readoutProbs(r: WorkspaceReadout): number[] { + // Use the token logits we kept; re-softmax over the returned top-k so two readouts are compared on the + // same support ordering. Callers that need full-vocab drift should pass full-vocab readouts (topK=vocab). + return softmax(r.tokens.map((t) => t.logit)); +} + +/** + * Mean Jensen–Shannon divergence (nats) between consecutive readouts in order. 0 when fewer than two + * readouts. Only compares readouts whose token support has the same length (same topK) so the JS is + * well-defined; mismatched pairs are skipped and reported via `comparedPairs`. + */ +export function workspaceDrift(readouts: readonly WorkspaceReadout[]): { drift: number; comparedPairs: number } { + let sum = 0; + let pairs = 0; + for (let i = 1; i < readouts.length; i++) { + const a = readoutProbs(readouts[i - 1]); + const b = readoutProbs(readouts[i]); + if (a.length !== b.length || a.length === 0) continue; + sum += jensenShannon(a, b); + pairs += 1; + } + return { drift: pairs === 0 ? 0 : sum / pairs, comparedPairs: pairs }; +} + +/** The per-readout entropy sequence, in the readouts' given order. */ +export function entropyTrajectory(readouts: readonly WorkspaceReadout[]): number[] { + return readouts.map((r) => r.entropy); +} diff --git a/packages/workspace-lens/src/index.ts b/packages/workspace-lens/src/index.ts new file mode 100644 index 00000000..d6a0ef19 --- /dev/null +++ b/packages/workspace-lens/src/index.ts @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: MIT +// +// @metaharness/workspace-lens — a Jacobian-Lens interpretability primitive for open-weight LLMs. +// Runtime-only (fitting is external), model-agnostic, dependency-free. See README.md. + +export * from './types.js'; +export * from './linalg.js'; +export { WorkspaceLens } from './lens.js'; +export { detectConcepts, flagsFromTriggers, hasCriticalTrigger, FLAG_CONCEPTS, type DetectOptions } from './safety.js'; +export { workspaceDrift, entropyTrajectory } from './drift.js'; +export { decide } from './decision.js'; +export { buildReceipt, sha256, type BuildReceiptOptions } from './receipt.js'; diff --git a/packages/workspace-lens/src/lens.ts b/packages/workspace-lens/src/lens.ts new file mode 100644 index 00000000..1947dad7 --- /dev/null +++ b/packages/workspace-lens/src/lens.ts @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: MIT +// +// WorkspaceLens — the runtime engine. Loads a fitted LensArtifact and, given a captured hidden state +// h_l, computes lens_l(h) = unembed(J_l · h) → workspace tokens. The intermediate J-projection z_l = +// J_l · h (the activation mapped into final-layer coordinates) is also exposed, because the vectorized +// safety triggers compare z_l to concept directions rather than decoding the whole vocabulary. + +import type { LensArtifact, HiddenState, WorkspaceReadout, WorkspaceToken } from './types.js'; +import { matVec, softmax, entropy, topKIndices } from './linalg.js'; + +export class WorkspaceLens { + private readonly byLayer: Map; + + private constructor(readonly artifact: LensArtifact) { + this.byLayer = new Map(artifact.layers.map((l) => [l.layer, l.jacobian])); + this.validate(); + } + + /** Build a lens from an in-memory artifact (validated eagerly so a bad artifact fails at load). */ + static fromArtifact(artifact: LensArtifact): WorkspaceLens { + return new WorkspaceLens(artifact); + } + + /** + * Load a fitted lens from a JSON file. (Binary/registry formats — e.g. a Neuronpedia pre-fit — are a + * follow-up; the on-disk contract is just a serialized `LensArtifact`.) Kept out of the constructor so + * the core stays sync + I/O-free for hot-path use. + */ + static async fromFile(path: string): Promise { + const { readFile } = await import('node:fs/promises'); + const raw = await readFile(path, 'utf-8'); + return new WorkspaceLens(JSON.parse(raw) as LensArtifact); + } + + get modelId(): string { return this.artifact.modelId; } + get lensId(): string { return this.artifact.lensId; } + get dModel(): number { return this.artifact.dModel; } + /** Layer indices this lens can read, ascending. */ + get layers(): number[] { return [...this.byLayer.keys()].sort((a, b) => a - b); } + + private validate(): void { + const { dModel, unembed, vocab, layers } = this.artifact; + if (dModel <= 0) throw new Error('lens artifact: dModel must be positive'); + if (vocab.length === 0) throw new Error('lens artifact: empty vocab'); + if (unembed.length !== vocab.length) { + throw new Error(`lens artifact: unembed rows ${unembed.length} != vocab ${vocab.length}`); + } + if (unembed[0]?.length !== dModel) { + throw new Error(`lens artifact: unembed cols ${unembed[0]?.length} != dModel ${dModel}`); + } + if (layers.length === 0) throw new Error('lens artifact: no fitted layers'); + for (const l of layers) { + if (l.jacobian.length !== dModel || l.jacobian[0]?.length !== dModel) { + throw new Error(`lens artifact: layer ${l.layer} jacobian is not ${dModel}×${dModel}`); + } + } + } + + /** True if the lens has a fitted operator for this layer. */ + hasLayer(layer: number): boolean { return this.byLayer.has(layer); } + + /** + * z_l = J_l · h — the activation projected into final-layer coordinates. This is the quantity the + * Jacobian Lens corrects for (vs. logit-lens reading h directly). Throws if the layer is un-fitted or + * the width is wrong. + */ + project(state: HiddenState): number[] { + const j = this.byLayer.get(state.layer); + if (!j) throw new Error(`lens has no fitted operator for layer ${state.layer}`); + if (state.h.length !== this.dModel) { + throw new Error(`hidden state width ${state.h.length} != dModel ${this.dModel}`); + } + return matVec(j, state.h); + } + + /** + * Full readout: lens_l(h) = unembed(J_l · h), returned as the top-k workspace tokens + the readout + * entropy. `topK` bounds the returned tokens; the entropy is over the FULL distribution. + */ + readout(state: HiddenState, topK = 10): WorkspaceReadout { + const z = this.project(state); + const logits = matVec(this.artifact.unembed, z); + const probs = softmax(logits); + const idx = topKIndices(logits, topK); + const tokens: WorkspaceToken[] = idx.map((vi, rank) => ({ + token: this.artifact.vocab[vi], + rank, + logit: logits[vi], + prob: probs[vi], + })); + return { layer: state.layer, position: state.position, tokens, entropy: entropy(probs) }; + } +} diff --git a/packages/workspace-lens/src/linalg.ts b/packages/workspace-lens/src/linalg.ts new file mode 100644 index 00000000..cfc2082f --- /dev/null +++ b/packages/workspace-lens/src/linalg.ts @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: MIT +// +// Minimal, dependency-free linear algebra for the runtime lens projection. This package NEVER fits a +// Jacobian (that needs a model backward pass — external); it only APPLIES a pre-fitted lens, which is a +// pair of matrix-vector products plus a softmax. So a few well-typed loops over Float64Array/number[] +// are all we need — no BLAS, no native deps, deterministic, and fast enough for per-request governance. + +export type Vector = readonly number[]; +/** Row-major dense matrix: `rows` arrays of length `cols`. */ +export type Matrix = readonly Vector[]; + +/** y = M · x. Throws on a shape mismatch so a malformed lens artifact fails loudly, not silently. */ +export function matVec(m: Matrix, x: Vector): number[] { + if (m.length === 0) return []; + const cols = m[0].length; + if (cols !== x.length) { + throw new Error(`matVec: matrix cols ${cols} != vector length ${x.length}`); + } + const out = new Array(m.length); + for (let i = 0; i < m.length; i++) { + const row = m[i]; + if (row.length !== cols) throw new Error(`matVec: ragged matrix at row ${i} (${row.length} != ${cols})`); + let s = 0; + for (let j = 0; j < cols; j++) s += row[j] * x[j]; + out[i] = s; + } + return out; +} + +export function dot(a: Vector, b: Vector): number { + if (a.length !== b.length) throw new Error(`dot: length ${a.length} != ${b.length}`); + let s = 0; + for (let i = 0; i < a.length; i++) s += a[i] * b[i]; + return s; +} + +export function norm(a: Vector): number { + return Math.sqrt(dot(a, a)); +} + +/** Cosine similarity in [-1, 1]; 0 when either vector is all-zero (avoids NaN). */ +export function cosine(a: Vector, b: Vector): number { + const na = norm(a); + const nb = norm(b); + if (na === 0 || nb === 0) return 0; + return dot(a, b) / (na * nb); +} + +/** Numerically-stable softmax over logits → a probability distribution summing to 1. */ +export function softmax(logits: Vector): number[] { + if (logits.length === 0) return []; + let max = -Infinity; + for (const v of logits) if (v > max) max = v; + const exps = new Array(logits.length); + let sum = 0; + for (let i = 0; i < logits.length; i++) { + const e = Math.exp(logits[i] - max); + exps[i] = e; + sum += e; + } + for (let i = 0; i < exps.length; i++) exps[i] /= sum; + return exps; +} + +/** Shannon entropy (nats) of a probability distribution. Higher = more diffuse workspace. */ +export function entropy(probs: Vector): number { + let h = 0; + for (const p of probs) if (p > 0) h -= p * Math.log(p); + return h; +} + +/** + * Jensen–Shannon divergence between two distributions (nats, in [0, ln 2]). Symmetric + bounded, so it + * is a stable per-step drift metric — unlike raw KL it never diverges to Infinity when a bin goes to 0. + */ +export function jensenShannon(p: Vector, q: Vector): number { + if (p.length !== q.length) throw new Error(`jensenShannon: length ${p.length} != ${q.length}`); + let d = 0; + for (let i = 0; i < p.length; i++) { + const m = (p[i] + q[i]) / 2; + if (p[i] > 0 && m > 0) d += 0.5 * p[i] * Math.log(p[i] / m); + if (q[i] > 0 && m > 0) d += 0.5 * q[i] * Math.log(q[i] / m); + } + // Clamp tiny negative from float error. + return d < 0 ? 0 : d; +} + +/** Indices of the top-k entries of `values`, highest first. Ties broken by lower index (stable). */ +export function topKIndices(values: Vector, k: number): number[] { + const idx = values.map((_, i) => i); + idx.sort((a, b) => (values[b] - values[a]) || (a - b)); + return idx.slice(0, Math.max(0, k)); +} diff --git a/packages/workspace-lens/src/receipt.ts b/packages/workspace-lens/src/receipt.ts new file mode 100644 index 00000000..7a91dda6 --- /dev/null +++ b/packages/workspace-lens/src/receipt.ts @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: MIT +// +// buildReceipt — the one-call orchestrator. Given a lens, a prompt, the captured hidden states, and a +// concept library, it reads the workspace at every state, scores drift/entropy, fires safety triggers, +// and assembles the signable WorkspaceLensReceipt. Pure aside from a sha256 of the prompt (node:crypto, +// a builtin) — no model, no network — so it runs identically offline and in a governance hot path. + +import { createHash } from 'node:crypto'; +import type { WorkspaceLens } from './lens.js'; +import type { + HiddenState, ConceptVector, WorkspaceLensReceipt, WorkspaceReadout, +} from './types.js'; +import { detectConcepts, flagsFromTriggers, type DetectOptions } from './safety.js'; +import { workspaceDrift, entropyTrajectory } from './drift.js'; + +export interface BuildReceiptOptions { + /** Number of workspace tokens to keep per readout. Default 8. */ + topK?: number; + /** Concept vectors to score for safety triggers. Default none. */ + concepts?: readonly ConceptVector[]; + /** Concept-trigger cosine threshold. */ + detect?: DetectOptions; + /** ISO timestamp for the receipt. REQUIRED — pass it in so the receipt is deterministic/reproducible. */ + createdAt: string; +} + +export function sha256(s: string): string { + return createHash('sha256').update(s).digest('hex'); +} + +export function buildReceipt( + lens: WorkspaceLens, + prompt: string, + states: readonly HiddenState[], + opts: BuildReceiptOptions, +): WorkspaceLensReceipt { + const topK = opts.topK ?? 8; + const readouts: WorkspaceReadout[] = states + .filter((s) => lens.hasLayer(s.layer)) + .map((s) => lens.readout(s, topK)); + + const triggers = opts.concepts ? detectConcepts(lens, states, opts.concepts, opts.detect) : []; + const flags = flagsFromTriggers(triggers); + const { drift } = workspaceDrift(readouts); + + const layersSeen = readouts.map((r) => r.layer); + const layerRange: [number, number] = layersSeen.length + ? [Math.min(...layersSeen), Math.max(...layersSeen)] + : [0, 0]; + + return { + promptHash: sha256(prompt), + modelId: lens.modelId, + lensId: lens.lensId, + layerRange, + positions: [...new Set(readouts.map((r) => r.position))].sort((a, b) => a - b), + topTokens: readouts.map((r) => ({ + layer: r.layer, + position: r.position, + tokens: r.tokens.map((t) => ({ token: t.token, rank: t.rank, logit: t.logit })), + })), + flags, + triggers, + workspaceDrift: drift, + entropyTrajectory: entropyTrajectory(readouts), + createdAt: opts.createdAt, + }; +} diff --git a/packages/workspace-lens/src/safety.ts b/packages/workspace-lens/src/safety.ts new file mode 100644 index 00000000..0224d50b --- /dev/null +++ b/packages/workspace-lens/src/safety.ts @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: MIT +// +// Vectorized safety triggers. Anthropic found hidden objectives ("secretly", "manipulation", "evaluation +// awareness") light up in the workspace even when absent from the output. We detect them NOT by matching +// decoded token strings (brittle, tokenizer-specific) but by the dot product of the J-projected +// activation against pre-fitted concept DIRECTIONS in the model's own J-space. That makes the trigger +// tokenizer-agnostic and cross-family: a concept is aligned by name + per-model vector, so "hidden +// objective" is one canonical concept with a Qwen vector and a Gemma vector — never a string map. + +import type { WorkspaceLens } from './lens.js'; +import type { HiddenState, ConceptVector, ConceptTrigger, SafetyFlags } from './types.js'; +import { cosine } from './linalg.js'; + +/** Canonical concept names that map onto the four headline SafetyFlags. */ +export const FLAG_CONCEPTS = { + promptInjection: ['prompt_injection', 'override', 'ignore_instructions', 'exfiltration'], + evalAwareness: ['evaluation_awareness', 'being_tested', 'watched'], + hiddenObjective: ['hidden_objective', 'secretly', 'deception', 'manipulation'], + refusalConflict: ['refusal_conflict', 'should_refuse', 'jailbreak'], +} as const; + +export interface DetectOptions { + /** Cosine threshold above which a concept is considered active. Default 0.35 (tunable per deployment). */ + threshold?: number; +} + +/** + * Score every (state × concept) pair and return the concepts that fired. Concept vectors whose `modelId` + * doesn't match the lens are skipped (a fail-safe against applying a Qwen direction to a Gemma model). + */ +export function detectConcepts( + lens: WorkspaceLens, + states: readonly HiddenState[], + concepts: readonly ConceptVector[], + opts: DetectOptions = {}, +): ConceptTrigger[] { + const threshold = opts.threshold ?? 0.35; + const triggers: ConceptTrigger[] = []; + for (const state of states) { + if (!lens.hasLayer(state.layer)) continue; + const z = lens.project(state); + for (const c of concepts) { + if (c.modelId !== lens.modelId) continue; // never cross-apply a concept from another model + if (c.vector.length !== z.length) continue; + const score = cosine(z, c.vector); + if (score >= threshold) { + triggers.push({ concept: c.concept, layer: state.layer, position: state.position, score, critical: !!c.critical }); + } + } + } + return triggers; +} + +/** Roll fired concept triggers up into the four headline flags. */ +export function flagsFromTriggers(triggers: readonly ConceptTrigger[]): SafetyFlags { + const fired = new Set(triggers.map((t) => t.concept)); + const any = (names: readonly string[]) => names.some((n) => fired.has(n)); + return { + promptInjection: any(FLAG_CONCEPTS.promptInjection), + evalAwareness: any(FLAG_CONCEPTS.evalAwareness), + hiddenObjective: any(FLAG_CONCEPTS.hiddenObjective), + refusalConflict: any(FLAG_CONCEPTS.refusalConflict), + }; +} + +/** True if any fired trigger is marked critical (used to fail the decision rule closed). */ +export function hasCriticalTrigger(triggers: readonly ConceptTrigger[]): boolean { + return triggers.some((t) => t.critical); +} diff --git a/packages/workspace-lens/src/types.ts b/packages/workspace-lens/src/types.ts new file mode 100644 index 00000000..0d9d6bf6 --- /dev/null +++ b/packages/workspace-lens/src/types.ts @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: MIT +// +// Public types for @metaharness/workspace-lens. The package is a RUNTIME measurement primitive: given a +// pre-fitted Jacobian lens artifact + captured hidden activations, it reads the model's "verbalizable +// workspace" (Anthropic, 2026-07-06) and emits a signable interpretability receipt. It does not fit the +// lens (that needs the model's backward pass — external, GPU) and it does not run the model. + +/** A fitted per-layer Jacobian operator J_l ∈ ℝ^{dModel×dModel}, plus the shared decode path. */ +export interface LensLayer { + /** Transformer layer index this operator was fitted for. */ + layer: number; + /** Row-major J_l (dModel×dModel): the average input→output Jacobian for this layer. */ + jacobian: readonly (readonly number[])[]; +} + +/** + * A fitted lens artifact for ONE model. Model-agnostic by construction: the vocabulary and the + * unembedding live in the artifact, so scoring never touches a tokenizer at runtime. Fitted OUT OF BAND + * (open-weight model + backward pass; see the reference `anthropics/jacobian-lens` + Neuronpedia), then + * serialized and loaded here. + */ +export interface LensArtifact { + /** Stable id for this fitted lens (e.g. "jlens-qwen2.5-7b-v2") — recorded in every receipt. */ + lensId: string; + /** The model the lens was fitted on (e.g. "qwen2.5-7b-instruct"). */ + modelId: string; + /** Residual-stream width. */ + dModel: number; + /** Decoded vocabulary — token strings aligned to the unembed rows. */ + vocab: readonly string[]; + /** Unembedding U ∈ ℝ^{vocab×dModel}. lens_l(h) = U · (J_l · h). */ + unembed: readonly (readonly number[])[]; + /** The fitted per-layer operators, keyed by layer index. */ + layers: readonly LensLayer[]; +} + +/** A captured residual-stream activation at a specific (layer, position). Supplied by the caller. */ +export interface HiddenState { + layer: number; + /** Token position in the sequence the activation was captured at. */ + position: number; + /** The residual vector h_l ∈ ℝ^{dModel}. */ + h: readonly number[]; +} + +/** One decoded workspace token: what the activation is DISPOSED to make the model say later. */ +export interface WorkspaceToken { + token: string; + /** 0-based rank in the lens readout (0 = most probable). */ + rank: number; + /** The lens logit for this token (pre-softmax). */ + logit: number; + /** The softmax probability for this token within the readout. */ + prob: number; +} + +/** A per-(layer,position) readout of the workspace. */ +export interface WorkspaceReadout { + layer: number; + position: number; + tokens: WorkspaceToken[]; + /** Shannon entropy (nats) of the full readout distribution — how diffuse the workspace is here. */ + entropy: number; +} + +/** + * A concept represented as a DIRECTION in a given model's J-space (NOT a token string). This is how the + * package aligns concepts across architectural families with different tokenizers (Qwen vs Gemma): a + * canonical concept name maps to a per-model unit vector fitted from example activations, so the safety + * trigger is a dot-product in activation space, tokenizer-agnostic. `modelId` guards against applying a + * Qwen concept vector to a Gemma activation. + */ +export interface ConceptVector { + /** Canonical concept name, aligned across models (e.g. "hidden_objective", "exfiltration"). */ + concept: string; + /** The model this vector lives in — MUST match the lens/activation model. */ + modelId: string; + /** Unit (or arbitrary) direction in J-space, length dModel. */ + vector: readonly number[]; + /** Optional: treat concept as CRITICAL — a trigger fails the decision rule closed. */ + critical?: boolean; +} + +/** A fired concept trigger. */ +export interface ConceptTrigger { + concept: string; + layer: number; + position: number; + /** Cosine similarity of the J-projected activation to the concept direction. */ + score: number; + critical: boolean; +} + +/** The four headline safety flags (Anthropic's workspace examples: eval-awareness, manipulation, …). */ +export interface SafetyFlags { + promptInjection: boolean; + evalAwareness: boolean; + hiddenObjective: boolean; + refusalConflict: boolean; +} + +/** + * The signable interpretability receipt — the audit artifact. "correctness first, cost second, receipts + * always": it records WHAT concepts were active in the workspace when a decision was made, with the + * layer-by-layer trajectory so an auditor can see WHERE a decision formed, not just the final answer. + */ +export interface WorkspaceLensReceipt { + promptHash: string; + modelId: string; + lensId: string; + layerRange: [number, number]; + positions: number[]; + topTokens: Array<{ + layer: number; + position: number; + tokens: Array<{ token: string; rank: number; logit: number }>; + }>; + flags: SafetyFlags; + triggers: ConceptTrigger[]; + /** Aggregate J-space distribution stability across the readouts (mean JS-divergence between steps). */ + workspaceDrift: number; + /** Per-readout entropy — is the internal workspace converging (falling) or dissolving (rising)? */ + entropyTrajectory: number[]; + createdAt: string; +} + +/** Inputs to the accept/reject decision rule. */ +export interface DecisionInput { + taskResolved: boolean; + workspaceDrift: number; + driftThreshold: number; + triggers: ConceptTrigger[]; + /** Fraction of monitored decisions that produced a receipt (1 = full coverage). */ + receiptCoverage: number; +} + +export interface DecisionResult { + accepted: boolean; + reasons: string[]; +} diff --git a/packages/workspace-lens/tsconfig.json b/packages/workspace-lens/tsconfig.json new file mode 100644 index 00000000..b4e8d735 --- /dev/null +++ b/packages/workspace-lens/tsconfig.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "Bundler", + "outDir": "./dist", + "rootDir": "./src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist", "__tests__"] +} diff --git a/scripts/build-ordered.mjs b/scripts/build-ordered.mjs index aa289692..0cc7bce3 100644 --- a/scripts/build-ordered.mjs +++ b/scripts/build-ordered.mjs @@ -23,7 +23,7 @@ const PHASES = [ // BEFORE create-agent-harness (phase 3), which depends on it. flywheel // (@metaharness/flywheel) is likewise dependency-free (node:crypto only) and // create-agent-harness imports its /cli — so it MUST build here in phase 1. - ['kernel-js', 'router', 'harness', 'darwin-mode', 'projects', 'redblue', 'weight-eft', 'jujutsu', 'flywheel'], + ['kernel-js', 'router', 'harness', 'darwin-mode', 'projects', 'redblue', 'weight-eft', 'jujutsu', 'flywheel', 'workspace-lens'], // evals-* adapters depend on @metaharness/flywheel's dist → build AFTER phase 1 (avoid .d.ts race). // evals-* adapters depend on @metaharness/flywheel's dist → build AFTER phase 1 (avoid .d.ts race). ['vertical-base', 'evals-hle', 'evals-toolcall', 'evals-extract', 'evals-math', 'evals-sql', 'evals-servedmodel'], diff --git a/scripts/healthcheck.mjs b/scripts/healthcheck.mjs index e69ede0a..9ef103e7 100644 --- a/scripts/healthcheck.mjs +++ b/scripts/healthcheck.mjs @@ -97,6 +97,9 @@ const CHECKS = { // @metaharness/evals-servedmodel (ruvllm served-model / SONA-MicroLoRA adapter for the // flywheel, ADR-234) is likewise a standalone published adapter on its own semver. '@metaharness/evals-servedmodel', + // @metaharness/workspace-lens (Jacobian-Lens interpretability primitive, ADR-238) is a standalone + // published library on its own semver — a runtime measurement primitive, not core/umbrella-bundled. + '@metaharness/workspace-lens', '@metaharness/kernel', '@metaharness/host-claude-code', '@metaharness/host-codex',