diff --git a/agents/templates/_partials/user-guidance-check.md b/agents/templates/_partials/user-guidance-check.md index b9913b1..59aff5e 100644 --- a/agents/templates/_partials/user-guidance-check.md +++ b/agents/templates/_partials/user-guidance-check.md @@ -1 +1,3 @@ **Check your context JSON for a `guidance` array.** If present, these are user-provided migration directives that you MUST follow. They take precedence over default heuristics. + +**If guidance explicitly allows a narrowly-scoped unsafe, ABI, or platform boundary when no safe equivalent exists, treat that as an allowed escape hatch.** Keep it minimal, audited, and isolated behind a safe API. diff --git a/agents/templates/code-migrator.md b/agents/templates/code-migrator.md index 2ffb7af..e3d854f 100644 --- a/agents/templates/code-migrator.md +++ b/agents/templates/code-migrator.md @@ -68,12 +68,24 @@ Concretely: The parity verifier will evaluate **behavioral equivalence** (same observable inputs → same observable outputs), not structural similarity. You will NOT be penalized for using different types, different function signatures, different module layouts, or different internal patterns — as long as the externally observable behavior is preserved. +### Audited Unsafe Escape Hatch + +Default to safe, idiomatic target code. However, if a required source behavior cannot be reproduced faithfully with safe constructs alone, and the guidance explicitly permits it, you may use a narrowly-scoped unsafe or raw ABI/platform boundary. + +Rules: +- Keep the unsafe or ABI boundary in a leaf helper or boundary module, not spread through the core algorithm logic. +- Expose a safe wrapper to the rest of the codebase. +- Do NOT use wrapper crates, bindgen, or delegation to the original implementation unless the guidance explicitly allows it. +- Use unsafe only for parity-critical behavior, never for convenience or micro-optimization. +- Add a brief inline comment documenting the invariant or boundary contract that makes the unsafe code sound. + ### DO - Preserve all business logic and observable behavior exactly - Write idiomatic target-language code that a native developer would recognize - Use the target language's standard library, type system, and error model - Adapt API signatures to be natural in the target language - Handle edge cases identically to the source (same observable outcomes) +- Use a narrowly-scoped audited unsafe or ABI boundary when parity requires it and the guidance explicitly allows it - Add inline comments noting migration decisions where behavior mapping is non-obvious ### DO NOT @@ -83,6 +95,7 @@ The parity verifier will evaluate **behavioral equivalence** (same observable in - Attempt to migrate files outside your assigned task - Transliterate source-language idioms when a target-language idiom exists - Preserve source-language API shapes (pointer parameters, error codes, global state) when the target language has better patterns +- Use unsafe or raw ABI calls for convenience, code brevity, or performance tuning - Read files beyond your task scope ## Handling Difficulties diff --git a/agents/templates/parity-failure-resolver.md b/agents/templates/parity-failure-resolver.md index cb3ab36..17d3c46 100644 --- a/agents/templates/parity-failure-resolver.md +++ b/agents/templates/parity-failure-resolver.md @@ -2,6 +2,8 @@ {{> lore-index-first-principle}} +{{> user-guidance-check}} + You are the **Parity Failure Resolver** agent, invoked when a migration task cannot proceed cleanly (parity failure, build/test breakage, or blocked migration). {{> task-scope-awareness}} @@ -23,6 +25,35 @@ Resolve the failing task quickly and safely by: When fixing parity issues, produce idiomatic target-language code — do NOT revert to source-language patterns to satisfy the verifier. If a parity issue stems from the verifier misidentifying an idiomatic target-language pattern as a gap (e.g., flagging `Result` as not matching a C return code), set `scopeReduced: true` and explain in `notes` that the behavior is equivalent despite the structural difference. The goal is behavioral equivalence, not structural mimicry. +When guidance explicitly allows a narrowly-scoped unsafe or platform boundary, treat that as an available recovery strategy. A small audited leaf shim is acceptable when it is the only way to preserve behavior and it does not delegate to the original source library. + +## Guidance-Constraint Adjudication + +Before attempting a code fix, check whether the reported parity issue **cannot be resolved without violating a `guidance` constraint**. This is the most common cause of oscillating parity failures across multiple attempts. + +If the guidance explicitly permits a narrowly-scoped unsafe or platform boundary, treat that as an available option rather than a prohibited one. + +**When to set `scopeReduced: true` instead of attempting a fix:** +1. The source behavior depends on a language-specific runtime feature (e.g., compiler sanitizer hooks, inline assembly, FFI declarations) AND the guidance still prohibits the narrow unsafe/ABI/platform boundary needed to express it +2. The `priorAttempts` array shows the same issue (or semantically equivalent issue) persisting across 2+ prior attempts despite different fix strategies — this is strong evidence the issue is fundamentally unresolvable within the guidance constraints +3. The only viable fix would require violating an explicit guidance directive or expanding beyond the minimal unsafe/ABI escape hatch the guidance allows + +When adjudicating an issue as guidance-constrained: +- Set `scopeReduced: true` +- In `notes`, cite the specific guidance constraint that prevents resolution, explain why no conforming implementation can satisfy the verifier, and describe what the current implementation does as the best available approximation +- Do NOT modify the code — leave the existing best-effort implementation in place +- Set `strategyApplied` to `"Guidance-constraint adjudication"` + +## Allocator and Ownership Adjudication + +Do not treat a different internal allocation strategy as a parity failure unless the source exposes that memory behavior to callers. A Rust port may replace allocator plumbing with idiomatic ownership as long as caller-visible semantics stay the same. + +When reviewing allocator-related parity issues: +- Ask whether the source exposes user-provided allocators, free callbacks, caller-owned buffers, or explicit ownership transfer in the public API +- If not, prefer preserving observable behavior and leaving the idiomatic ownership model intact +- Do NOT spend retry budget recreating C-style internal allocation plumbing purely to satisfy a structural reading of the source +- If the verifier is objecting to an internal ownership change with no caller-visible divergence, explain that in `notes` and avoid unnecessary code churn + ## Required Process 1. **Diagnose** diff --git a/agents/templates/parity-verifier.md b/agents/templates/parity-verifier.md index d598c34..834fa9a 100644 --- a/agents/templates/parity-verifier.md +++ b/agents/templates/parity-verifier.md @@ -4,6 +4,8 @@ You are the **Parity Verifier** — a read-only analysis agent that checks wheth {{> lore-index-first-principle}} +{{> user-guidance-check}} + {{> task-scope-awareness}} **When `taskScope` is present, calibrate your analysis to the task's intended scope.** For example: @@ -26,12 +28,42 @@ Parity means **behavioral equivalence** — the migrated code must produce the s - Different data structures (e.g., Vec instead of a linked list, HashMap instead of a red-black tree) - Different module organization or file layout - Different error handling patterns (e.g., Result/Option instead of sentinel return values) -- Different memory management (e.g., ownership instead of malloc/free) +- Different internal memory management or allocator strategy (e.g., ownership instead of malloc/free), unless the public API exposes allocator selection, ownership transfer, or caller-managed memory semantics that have changed - Merged or split functions, renamed identifiers, or reorganized types — as long as all behavior is preserved - Use of target-language standard library where the source used hand-rolled implementations Do NOT flag idiomatic target-language patterns as parity issues. A Rust `Result` is equivalent to a C `int` return code + out-parameter if it conveys the same success/failure semantics. +## Guidance-Constrained Parity + +When the `guidance` array is present in your context, some source behaviors may be **intentionally impossible to replicate** in the target due to user-imposed constraints. Common examples include: +- Source code that relies on language-specific runtime features (sanitizer hooks, compiler intrinsics, FFI declarations) when guidance prohibits unsafe code or FFI in the target +- Platform-specific system calls when guidance requires a pure/portable implementation +- Source patterns that depend on undefined behavior when guidance requires safe, well-defined code + +**When a source behavior cannot be faithfully reproduced without violating a guidance constraint:** +- Classify the issue as `minor`, not `major` or `critical` +- In the `details` field, explicitly note which guidance constraint makes faithful reproduction impossible +- In the `suggestedFix` field, recommend the best available approximation that respects the guidance (e.g., no-op behind a feature flag, compile-time constant, or documented deviation) + +When guidance explicitly permits a narrowly-scoped unsafe or platform boundary as the only viable way to preserve behavior: +- Evaluate whether the unsafe/ABI surface is minimal, audited, and isolated behind a safe API +- Do NOT prefer a less faithful safe-only approximation over a minimal allowed boundary that preserves behavior + +Do NOT flag source behaviors as `major` or `critical` when the only path to resolution would require violating a user-provided guidance directive. The guidance constraints represent deliberate user decisions and take precedence over source-faithful reproduction. + +## Allocator and Ownership Contract Parity + +Changes to the target's internal allocation model do NOT by themselves create a parity failure. A Rust port may replace malloc/free plumbing, arena internals, or ad-hoc ownership tracking with RAII, Vec, Box, Arc, or other idiomatic constructs as long as callers observe the same behavior. + +Only flag allocator-related issues as `major` or `critical` when the source exposes memory behavior as part of the public contract, such as: +- User-supplied allocators or custom free callbacks +- Caller-owned buffers, explicit transfer-of-ownership rules, or required deallocation order +- Public aliasing/lifetime guarantees that affect correctness +- Allocation-failure behavior or size/accounting semantics that change observable results + +If the difference is purely internal representation or ownership discipline, do NOT describe it as a public-contract divergence. + ## Responsibilities 1. **Behavioral Parity** @@ -76,6 +108,7 @@ Do NOT flag idiomatic target-language patterns as parity issues. A Rust `Result< - Check whether the target function implements the algorithm natively or delegates to an external binding/wrapper of the source library - If the target calls into a package that wraps or binds to the source library via FFI, flag as `critical` — the migration has not actually re-implemented the logic - If the target imports or links against the source library's compiled artifacts, flag as `critical` + - Do NOT flag a minimal OS/runtime ABI shim as delegation if it does not call the original source library and the migrated algorithm remains natively implemented in the target - Compare the target function's implementation depth against the source: a source function with substantial algorithm logic should not map to a short target function that delegates to a library call 9. **Hollow Implementation Detection** (severity guidance: `critical`) diff --git a/package-lock.json b/package-lock.json index edbd064..3bf6e70 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,7 @@ "version": "0.1.0", "dependencies": { "@cadre-dev/framework": "^0.2.5", - "@jafreck/lore": "^0.3.9", + "@jafreck/lore": "0.4.0", "@modelcontextprotocol/sdk": "^1.27.1", "@types/better-sqlite3": "^7.6.13", "better-sqlite3": "^12.6.2", @@ -1134,45 +1134,20 @@ } }, "node_modules/@jafreck/lore": { - "version": "0.3.9", - "resolved": "https://registry.npmjs.org/@jafreck/lore/-/lore-0.3.9.tgz", - "integrity": "sha512-UJwTaYesHNNRwFhcyFMvVq3NUKr1VAd4Pym3uJoMrbIS0fQPTn0ttYF7MDIJeYcM9hvsxSwQ8yS/iUrGX7lszA==", + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/@jafreck/lore/-/lore-0.4.0.tgz", + "integrity": "sha512-yq4n514nnFwRf/pESOO7NNV8uZRxCx8dgLvUlVakUaiepcCy2Zcji7JjrUCPogqi+2WbOI9d8TQnZ5g/lQRt5g==", "license": "MIT", "dependencies": { "@bufbuild/protobuf": "^2.11.0", - "@elm-tooling/tree-sitter-elm": "^5.9.0", "@huggingface/transformers": "^3.8.1", "@modelcontextprotocol/sdk": "^1.27.1", "@sourcegraph/scip-python": "^0.6.6", "@sourcegraph/scip-typescript": "^0.4.0", - "@tree-sitter-grammars/tree-sitter-lua": "^0.4.1", - "@tree-sitter-grammars/tree-sitter-zig": "^1.1.2", "better-sqlite3": "^12.6.2", "fast-glob": "^3.3.3", - "node-gyp-build": "^4.8.4", "simple-git": "^3.32.3", "sqlite-vec": "^0.1.6", - "tree-sitter": "0.25.0", - "tree-sitter-bash": "^0.25.1", - "tree-sitter-c": "^0.24.1", - "tree-sitter-c-sharp": "^0.23.1", - "tree-sitter-cpp": "^0.23.4", - "tree-sitter-elixir": "^0.3.5", - "tree-sitter-go": ">=0.23.4", - "tree-sitter-haskell": "^0.23.1", - "tree-sitter-java": "^0.23.5", - "tree-sitter-javascript": "^0.25.0", - "tree-sitter-julia": "^0.23.1", - "tree-sitter-kotlin": "^0.3.8", - "tree-sitter-objc": "^3.0.2", - "tree-sitter-ocaml": "^0.24.2", - "tree-sitter-php": "^0.24.2", - "tree-sitter-python": ">=0.23.6", - "tree-sitter-ruby": "^0.23.1", - "tree-sitter-rust": "^0.24.0", - "tree-sitter-scala": "^0.24.0", - "tree-sitter-swift": "^0.7.1", - "tree-sitter-typescript": "^0.23.2", "zod": "^4.3" }, "bin": { @@ -1182,349 +1157,6 @@ "node": ">=22.0.0" } }, - "node_modules/@jafreck/lore/node_modules/@elm-tooling/tree-sitter-elm": { - "version": "5.9.0", - "resolved": "https://registry.npmjs.org/@elm-tooling/tree-sitter-elm/-/tree-sitter-elm-5.9.0.tgz", - "integrity": "sha512-9iLNjGv/FYXHAi+YYu1GnakKsQkPJ1eXsl2DWuwAmcTY8kH/aFqzylryMcwEXT2sEbYNbjNrfjC5HXql0PMkKQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.1", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.25.0" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter": { - "version": "0.25.0", - "resolved": "https://registry.npmjs.org/tree-sitter/-/tree-sitter-0.25.0.tgz", - "integrity": "sha512-PGZZzFW63eElZJDe/b/R/LbsjDDYJa5UEjLZJB59RQsMX+fo0j54fqBPn1MGKav/QNa0JR0zBiVaikYDWCj5KQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4" - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-bash": { - "version": "0.25.1", - "resolved": "https://registry.npmjs.org/tree-sitter-bash/-/tree-sitter-bash-0.25.1.tgz", - "integrity": "sha512-7hMytuYIMoXOq24yRulgIxthE9YmggZIOHCyPTTuJcu6EU54tYD+4G39cUb28kxC6jMf/AbPfWGLQtgPTdh3xw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.1", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.25.0" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-c-sharp": { - "version": "0.23.1", - "resolved": "https://registry.npmjs.org/tree-sitter-c-sharp/-/tree-sitter-c-sharp-0.23.1.tgz", - "integrity": "sha512-9zZ4FlcTRWWfRf6f4PgGhG8saPls6qOOt75tDfX7un9vQZJmARjPrAC6yBNCX2T/VKcCjIDbgq0evFaB3iGhQw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-cpp": { - "version": "0.23.4", - "resolved": "https://registry.npmjs.org/tree-sitter-cpp/-/tree-sitter-cpp-0.23.4.tgz", - "integrity": "sha512-qR5qUDyhZ5jJ6V8/umiBxokRbe89bCGmcq/dk94wI4kN86qfdV8k0GHIUEKaqWgcu42wKal5E97LKpLeVW8sKw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.1", - "node-gyp-build": "^4.8.2", - "tree-sitter-c": "^0.23.1" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-cpp/node_modules/tree-sitter-c": { - "version": "0.23.6", - "resolved": "https://registry.npmjs.org/tree-sitter-c/-/tree-sitter-c-0.23.6.tgz", - "integrity": "sha512-0dxXKznVyUA0s6PjNolJNs2yF87O5aL538A/eR6njA5oqX3C3vH4vnx3QdOKwuUdpKEcFdHuiDpRKLLCA/tjvQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-elixir": { - "version": "0.3.5", - "resolved": "https://registry.npmjs.org/tree-sitter-elixir/-/tree-sitter-elixir-0.3.5.tgz", - "integrity": "sha512-xozQMvYK0aSolcQZAx2d84Xe/YMWFuRPYFlLVxO01bM2GITh5jyiIp0TqPCQa8754UzRAI7A83hZmfiYub5TZQ==", - "hasInstallScript": true, - "license": "Apache-2.0", - "dependencies": { - "node-addon-api": "^7.1.0", - "node-gyp-build": "^4.8.0" - }, - "peerDependencies": { - "tree-sitter": "^0.21.0" - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-elixir/node_modules/node-addon-api": { - "version": "7.1.1", - "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-7.1.1.tgz", - "integrity": "sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ==", - "license": "MIT" - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-go": { - "version": "0.25.0", - "resolved": "https://registry.npmjs.org/tree-sitter-go/-/tree-sitter-go-0.25.0.tgz", - "integrity": "sha512-APBc/Dq3xz/e35Xpkhb1blu5UgW+2E3RyGWawZSCNcbGwa7jhSQPS8KsUupuzBla8PCo8+lz9W/JDJjmfRa2tw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.1", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.25.0" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-haskell": { - "version": "0.23.1", - "resolved": "https://registry.npmjs.org/tree-sitter-haskell/-/tree-sitter-haskell-0.23.1.tgz", - "integrity": "sha512-qG4CYhejveu9DLMLEGBz/n9/TTeGSFLC6wniwOgG6m8/v7Dng8qR0ob0EVG7+XH+9WiOxohpGA23EhceWuxY4w==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-java": { - "version": "0.23.5", - "resolved": "https://registry.npmjs.org/tree-sitter-java/-/tree-sitter-java-0.23.5.tgz", - "integrity": "sha512-Yju7oQ0Xx7GcUT01mUglPP+bYfvqjNCGdxqigTnew9nLGoII42PNVP3bHrYeMxswiCRM0yubWmN5qk+zsg0zMA==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-javascript": { - "version": "0.25.0", - "resolved": "https://registry.npmjs.org/tree-sitter-javascript/-/tree-sitter-javascript-0.25.0.tgz", - "integrity": "sha512-1fCbmzAskZkxcZzN41sFZ2br2iqTYP3tKls1b/HKGNPQUVOpsUxpmGxdN/wMqAk3jYZnYBR1dd/y/0avMeU7dw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.1", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.25.0" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-julia": { - "version": "0.23.1", - "resolved": "https://registry.npmjs.org/tree-sitter-julia/-/tree-sitter-julia-0.23.1.tgz", - "integrity": "sha512-3vShY0GIu8ajR6hXzE0pyUk6kkfg4pGx3Bfzm6lGmR9aC3fe+LgoBMlaFJ7JY+t0fNFccc77J8HVP67ukuDMxQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-kotlin": { - "version": "0.3.8", - "resolved": "https://registry.npmjs.org/tree-sitter-kotlin/-/tree-sitter-kotlin-0.3.8.tgz", - "integrity": "sha512-A4obq6bjzmYrA+F0JLLoheFPcofFkctNaZSpnDd+GPn1SfVZLY4/GG4C0cYVBTOShuPBGGAOPLM1JWLZQV4m1g==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^7.1.0", - "node-gyp-build": "^4.8.0" - }, - "peerDependencies": { - "tree-sitter": "^0.21.0" - }, - "peerDependenciesMeta": { - "tree_sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-kotlin/node_modules/node-addon-api": { - "version": "7.1.1", - "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-7.1.1.tgz", - "integrity": "sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ==", - "license": "MIT" - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-python": { - "version": "0.25.0", - "resolved": "https://registry.npmjs.org/tree-sitter-python/-/tree-sitter-python-0.25.0.tgz", - "integrity": "sha512-eCmJx6zQa35GxaCtQD+wXHOhYqBxEL+bp71W/s3fcDMu06MrtzkVXR437dRrCrbrDbyLuUDJpAgycs7ncngLXw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.5.0", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.25.0" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-ruby": { - "version": "0.23.1", - "resolved": "https://registry.npmjs.org/tree-sitter-ruby/-/tree-sitter-ruby-0.23.1.tgz", - "integrity": "sha512-d9/RXgWjR6HanN7wTYhS5bpBQLz1VkH048Vm3CodPGyJVnamXMGb8oEhDypVCBq4QnHui9sTXuJBBP3WtCw5RA==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-scala": { - "version": "0.24.0", - "resolved": "https://registry.npmjs.org/tree-sitter-scala/-/tree-sitter-scala-0.24.0.tgz", - "integrity": "sha512-vkMuAUrBZ1zZz2XcGDQk18Kz73JkpgaeXzbNVobPke0G35sd9jH32aUxG6OLRKM7et0TbsfqkWf4DeJoGk4K1g==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-typescript": { - "version": "0.23.2", - "resolved": "https://registry.npmjs.org/tree-sitter-typescript/-/tree-sitter-typescript-0.23.2.tgz", - "integrity": "sha512-e04JUUKxTT53/x3Uq1zIL45DoYKVfHH4CZqwgZhPg5qYROl5nQjV+85ruFzFGZxu+QeFVbRTPDRnqL9UbU4VeA==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2", - "tree-sitter-javascript": "^0.23.1" - }, - "peerDependencies": { - "tree-sitter": "^0.21.0" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@jafreck/lore/node_modules/tree-sitter-typescript/node_modules/tree-sitter-javascript": { - "version": "0.23.1", - "resolved": "https://registry.npmjs.org/tree-sitter-javascript/-/tree-sitter-javascript-0.23.1.tgz", - "integrity": "sha512-/bnhbrTD9frUYHQTiYnPcxyHORIw157ERBa6dqzaKxvR/x3PC4Yzd+D1pZIMS6zNg2v3a8BZ0oK7jHqsQo9fWA==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.21.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, "node_modules/@jridgewell/resolve-uri": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", @@ -2066,44 +1698,6 @@ "dev": true, "license": "MIT" }, - "node_modules/@tree-sitter-grammars/tree-sitter-lua": { - "version": "0.4.1", - "resolved": "https://registry.npmjs.org/@tree-sitter-grammars/tree-sitter-lua/-/tree-sitter-lua-0.4.1.tgz", - "integrity": "sha512-EwagFaU6ZveVk18/Y8qUhZkkiBKnQ7dSCHbm//TUroLVKy3i1rOYGy/cNHtSkAb1eDvS1HhCLybH2S541Cya/g==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.5.0", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.4" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/@tree-sitter-grammars/tree-sitter-zig": { - "version": "1.1.2", - "resolved": "https://registry.npmjs.org/@tree-sitter-grammars/tree-sitter-zig/-/tree-sitter-zig-1.1.2.tgz", - "integrity": "sha512-J0L31HZ2isy3F5zb2g5QWQOv2r/pbruQNL9ADhuQv2pn5BQOzxt80WcEJaYXBeuJ8GHxVT42slpCna8k1c8LOw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, "node_modules/@tsconfig/node10": { "version": "1.0.12", "resolved": "https://registry.npmjs.org/@tsconfig/node10/-/node10-1.0.12.tgz", @@ -4222,26 +3816,6 @@ "node": ">=10" } }, - "node_modules/node-addon-api": { - "version": "8.6.0", - "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.6.0.tgz", - "integrity": "sha512-gBVjCaqDlRUk0EwoPNKzIr9KkS9041G/q31IBShPs1Xz6UTA+EXdZADbzqAJQrpDRq71CIMnOP5VMut3SL0z5Q==", - "license": "MIT", - "engines": { - "node": "^18 || ^20 || >= 21" - } - }, - "node_modules/node-gyp-build": { - "version": "4.8.4", - "resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.4.tgz", - "integrity": "sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==", - "license": "MIT", - "bin": { - "node-gyp-build": "bin.js", - "node-gyp-build-optional": "optional.js", - "node-gyp-build-test": "build-test.js" - } - }, "node_modules/object-assign": { "version": "4.1.1", "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz", @@ -5337,167 +4911,6 @@ "node": ">=0.6" } }, - "node_modules/tree-sitter": { - "version": "0.22.4", - "resolved": "https://registry.npmjs.org/tree-sitter/-/tree-sitter-0.22.4.tgz", - "integrity": "sha512-usbHZP9/oxNsUY65MQUsduGRqDHQOou1cagUSwjhoSYAmSahjQDAVsh9s+SlZkn8X8+O1FULRGwHu7AFP3kjzg==", - "hasInstallScript": true, - "license": "MIT", - "peer": true, - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4" - } - }, - "node_modules/tree-sitter-c": { - "version": "0.24.1", - "resolved": "https://registry.npmjs.org/tree-sitter-c/-/tree-sitter-c-0.24.1.tgz", - "integrity": "sha512-lkYwWN3SRecpvaeqmFKkuPNR3ZbtnvHU+4XAEEkJdrp3JfSp2pBrhXOtvfsENUneye76g889Y0ddF2DM0gEDpA==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.1", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.4" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/tree-sitter-cli": { - "version": "0.23.2", - "resolved": "https://registry.npmjs.org/tree-sitter-cli/-/tree-sitter-cli-0.23.2.tgz", - "integrity": "sha512-kPPXprOqREX+C/FgUp2Qpt9jd0vSwn+hOgjzVv/7hapdoWpa+VeWId53rf4oNNd29ikheF12BYtGD/W90feMbA==", - "hasInstallScript": true, - "license": "MIT", - "bin": { - "tree-sitter": "cli.js" - }, - "engines": { - "node": ">=12.0.0" - } - }, - "node_modules/tree-sitter-objc": { - "version": "3.0.2", - "resolved": "https://registry.npmjs.org/tree-sitter-objc/-/tree-sitter-objc-3.0.2.tgz", - "integrity": "sha512-Hs0ohmx1u5M+0K7efoW+dv/corhBsfjftfIYLtp7dSGeJ+Zj4c33tDIboBYLs6qijRlz6wtHFxa0YX+FibLulA==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4", - "tree-sitter-c": "^0.23.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/tree-sitter-objc/node_modules/tree-sitter-c": { - "version": "0.23.6", - "resolved": "https://registry.npmjs.org/tree-sitter-c/-/tree-sitter-c-0.23.6.tgz", - "integrity": "sha512-0dxXKznVyUA0s6PjNolJNs2yF87O5aL538A/eR6njA5oqX3C3vH4vnx3QdOKwuUdpKEcFdHuiDpRKLLCA/tjvQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/tree-sitter-ocaml": { - "version": "0.24.2", - "resolved": "https://registry.npmjs.org/tree-sitter-ocaml/-/tree-sitter-ocaml-0.24.2.tgz", - "integrity": "sha512-H0RAeCepIyXyTPCQra6yMd7Bn5ZBYkIaddzdLNwVZpM9mCe2e8av+3O6Ojl7Z8YHrV/kYsfHvI2y+Hh7qzcYQQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.3.0", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.4" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/tree-sitter-php": { - "version": "0.24.2", - "resolved": "https://registry.npmjs.org/tree-sitter-php/-/tree-sitter-php-0.24.2.tgz", - "integrity": "sha512-zwgAePc/HozNaWOOfwRAA+3p8yhuehRw8Fb7vn5qd2XjiIc93uJPryDTMYTSjBRjVIUg/KY6pM3rRzs8dSwKfw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.2" - }, - "peerDependencies": { - "tree-sitter": "^0.22.4" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/tree-sitter-rust": { - "version": "0.24.0", - "resolved": "https://registry.npmjs.org/tree-sitter-rust/-/tree-sitter-rust-0.24.0.tgz", - "integrity": "sha512-NWemUDf629Tfc90Y0Z55zuwPCAHkLxWnMf2RznYu4iBkkrQl2o/CHGB7Cr52TyN5F1DAx8FmUnDtCy9iUkXZEQ==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.2.2", - "node-gyp-build": "^4.8.4" - }, - "peerDependencies": { - "tree-sitter": "^0.22.1" - }, - "peerDependenciesMeta": { - "tree-sitter": { - "optional": true - } - } - }, - "node_modules/tree-sitter-swift": { - "version": "0.7.1", - "resolved": "https://registry.npmjs.org/tree-sitter-swift/-/tree-sitter-swift-0.7.1.tgz", - "integrity": "sha512-pneKVTuGamaBsqqqfB9BvNQjktzh/0IVPR54jLB5Fq/JTDQwYHd0Wo6pVyZ5jAYpbztzq+rJ/rpL9ruxTmSoKw==", - "hasInstallScript": true, - "license": "MIT", - "dependencies": { - "node-addon-api": "^8.0.0", - "node-gyp-build": "^4.8.0", - "tree-sitter-cli": "^0.23", - "which": "2.0.2" - }, - "peerDependencies": { - "tree-sitter": "^0.22.1" - }, - "peerDependenciesMeta": { - "tree_sitter": { - "optional": true - } - } - }, "node_modules/ts-node": { "version": "10.9.2", "resolved": "https://registry.npmjs.org/ts-node/-/ts-node-10.9.2.tgz", diff --git a/package.json b/package.json index 0fc9ca2..1b4b65e 100644 --- a/package.json +++ b/package.json @@ -20,7 +20,7 @@ }, "dependencies": { "@cadre-dev/framework": "^0.2.5", - "@jafreck/lore": "^0.3.9", + "@jafreck/lore": "0.4.0", "@modelcontextprotocol/sdk": "^1.27.1", "@types/better-sqlite3": "^7.6.13", "better-sqlite3": "^12.6.2", diff --git a/src/agents/agent-output-schemas.ts b/src/agents/agent-output-schemas.ts index b2a97a5..d3ab0ce 100644 --- a/src/agents/agent-output-schemas.ts +++ b/src/agents/agent-output-schemas.ts @@ -27,6 +27,56 @@ export const AamfOutputBase = z.object({ export type AamfOutputBaseType = z.infer; +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function normalizeLegacyStatus(status: unknown): unknown { + if (typeof status !== 'string') return status; + + switch (status.trim().toLowerCase()) { + case 'success': + case 'succeeded': + case 'ok': + return 'completed'; + case 'needs_review': + case 'needs review': + return 'needs-review'; + case 'error': + case 'failure': + return 'failed'; + default: + return status; + } +} + +function normalizeLegacyNotes(notes: unknown): unknown { + if (!Array.isArray(notes) || !notes.every(item => typeof item === 'string')) { + return notes; + } + + return notes.join('\n'); +} + +function normalizeAamfOutput(raw: unknown): unknown { + if (!isRecord(raw)) return raw; + + const normalized: Record = { ...raw }; + + normalized.status = normalizeLegacyStatus(normalized.status); + normalized.notes = normalizeLegacyNotes(normalized.notes); + + if ( + normalized.outputFiles === undefined + && Array.isArray(normalized.written) + && normalized.written.every(item => typeof item === 'string') + ) { + normalized.outputFiles = normalized.written; + } + + return normalized; +} + /** * JSON schema for structured agent task results. * @deprecated Parity results are now extracted from aamf-json output directly. @@ -107,6 +157,8 @@ export function parseAamfOutput( } } + raw = normalizeAamfOutput(raw); + const result = schema.safeParse(raw); if (!result.success) { return { parsed: false, error: `schema validation failed: ${result.error.message}` }; diff --git a/src/config/schema.ts b/src/config/schema.ts index f8d3165..3934799 100644 --- a/src/config/schema.ts +++ b/src/config/schema.ts @@ -11,7 +11,7 @@ export const MigrationConfigSchema = z.object({ * * Examples: * - "Do NOT use any existing crates/packages that wrap the C implementation." - * - "Write a pure native Rust port — no FFI or bindgen." + * - "Write a pure native Rust port — no wrapper crates or bindgen; allow only audited leaf unsafe/ABI shims when no safe equivalent exists." * - "Preserve the original directory layout in the target output." */ guidance: z.array(z.string().min(1)).optional(), @@ -194,8 +194,10 @@ export const MigrationConfigSchema = z.object({ }).optional(), /** * Options for KB indexing (Phase 0). - * The Lore indexer always runs in Phase 0 to build a SQLite knowledge-base. - * An HTTP MCP server is started for agents to query it. + * The Lore indexer always runs in Phase 0 to build a SQLite knowledge-base. + * SCIP indexing is always enabled; optional LSP enrichment augments the + * baseline index and powers overlay updates. + * An HTTP MCP server is started for agents to query it. */ kbIndex: z.object({ /** @@ -223,14 +225,14 @@ export const MigrationConfigSchema = z.object({ pythonBin: z.string().default('python3'), }).optional(), /** - * LSP integration for the Lore indexer. + * Optional LSP enrichment for the Lore indexer. * When enabled, Lore starts language servers (e.g. clangd for C/C++, - * typescript-language-server for TS) to resolve cross-file symbol - * references, type definitions, and call targets with full semantic - * accuracy — beyond what tree-sitter can provide alone. + * typescript-language-server for TS) to enrich symbols and refs with + * semantic definition and type information, and to power overlay updates + * after the baseline SCIP index is built. */ lsp: z.object({ - /** Enable LSP-powered symbol resolution during indexing. Default: false. */ + /** Enable LSP enrichment and overlay updates. Default: false. */ enabled: z.boolean().default(false), /** Timeout in ms for each LSP request (hover, definition, references). */ requestTimeoutMs: z.number().int().min(500).default(5000), diff --git a/src/core/checkpoint.ts b/src/core/checkpoint.ts index 62c1286..46c3849 100644 --- a/src/core/checkpoint.ts +++ b/src/core/checkpoint.ts @@ -634,7 +634,24 @@ export class CheckpointManager { const ids = snapshot.completedExecutionIds; if (!Array.isArray(ids) || ids.length === 0) return; - const completedSet = new Set(state.completedTasks); + // Build the set of completed tasks. If completedTasks is empty (legacy + // checkpoints that never called completeTask()), derive the set from + // the flow checkpoint's own commit entries so we don't discard all progress. + let completedSet = new Set(state.completedTasks); + if (completedSet.size === 0) { + const TASK_ID_RE = /\/(task-[^/]+)\/[^/]+\/commit$/; + for (const id of ids) { + const m = TASK_ID_RE.exec(id); + if (m) completedSet.add(m[1]!); + } + // Back-fill completedTasks so downstream logic stays consistent. + if (completedSet.size > 0) { + state.completedTasks = [...completedSet]; + this.logger.info( + `Back-filled ${completedSet.size} completed task(s) from Phase 4 flow checkpoint commit entries`, + ); + } + } if (completedSet.size === ids.length) return; // all tasks completed, nothing to filter const filtered = ids.filter((id: string) => { diff --git a/src/core/lore-index-settings.ts b/src/core/lore-index-settings.ts new file mode 100644 index 0000000..6238140 --- /dev/null +++ b/src/core/lore-index-settings.ts @@ -0,0 +1,113 @@ +import type { EffectiveLspSettings, EffectiveScipSettings } from '@jafreck/lore'; +import type { MigrationConfig } from '../config/schema.js'; + +type KbIndexConfig = MigrationConfig['options']['kbIndex']; +type LspServerOverride = { command: string; args?: string[] }; + +const DEFAULT_LSP_REQUEST_TIMEOUT_MS = 5_000; +const DEFAULT_SCIP_TIMEOUT_MS = 120_000; + +// Lore's programmatic API expects fully-effective settings, but the helper +// functions that build those defaults are not part of the public export surface. +// Mirror the 0.4.0 defaults here so AAMF can always pass the new API shape. +const DEFAULT_LSP_SERVER_REGISTRY: EffectiveLspSettings['servers'] = { + c: { command: 'clangd', args: [] }, + rust: { command: 'rust-analyzer', args: [] }, + python: { command: 'pyright-langserver', args: ['--stdio'] }, + cpp: { command: 'clangd', args: [] }, + typescript: { command: 'typescript-language-server', args: ['--stdio'] }, + javascript: { command: 'typescript-language-server', args: ['--stdio'] }, + go: { command: 'gopls', args: [] }, + java: { command: 'jdtls', args: [] }, + csharp: { command: 'csharp-ls', args: [] }, + ruby: { command: 'solargraph', args: ['stdio'] }, + php: { command: 'intelephense', args: ['--stdio'] }, + swift: { command: 'sourcekit-lsp', args: [] }, + kotlin: { command: 'kotlin-language-server', args: [] }, + scala: { command: 'metals', args: [] }, + lua: { command: 'lua-language-server', args: [] }, + bash: { command: 'bash-language-server', args: ['start'] }, + elixir: { command: 'elixir-ls', args: [] }, + zig: { command: 'zls', args: [] }, + ocaml: { command: 'ocamllsp', args: [] }, + haskell: { command: 'haskell-language-server-wrapper', args: ['--lsp'] }, + julia: { + command: 'julia', + args: ['--startup-file=no', '--history-file=no', '--quiet', '--eval', 'using LanguageServer, SymbolServer; runserver()'], + }, + elm: { command: 'elm-language-server', args: [] }, + objc: { command: 'clangd', args: [] }, +}; + +const DEFAULT_SCIP_INDEXER_REGISTRY: EffectiveScipSettings['indexers'] = { + typescript: { command: 'scip-typescript', args: ['index', '--output', '{output}'] }, + python: { command: 'scip-python', args: ['index', '.', '--project-name', 'project', '--output', '{output}'] }, + java: { command: 'scip-java', args: ['index', '--output', '{output}'] }, + scala: { command: 'scip-java', args: ['index', '--output', '{output}'] }, + kotlin: { command: 'scip-java', args: ['index', '--output', '{output}'] }, + rust: { command: 'rust-analyzer', args: ['scip', '.'] }, + c: { command: 'scip-clang', args: ['--compdb-path={compdb}', '--index-output-path={output}'] }, + cpp: { command: 'scip-clang', args: ['--compdb-path={compdb}', '--index-output-path={output}'] }, + csharp: { command: 'scip-dotnet', args: ['index', '.', '--output', '{output}'] }, + ruby: { command: 'scip-ruby', args: ['--output', '{output}'] }, + php: { command: 'scip-php', args: ['index', '--output', '{output}'] }, + go: { command: 'scip-go', args: [] }, + dart: { command: 'scip-dart', args: ['index', '--output', '{output}'] }, +}; + +export interface LoreIndexSettings { + lsp: EffectiveLspSettings; + scip: EffectiveScipSettings; +} + +function cloneLspServerRegistry(registry: EffectiveLspSettings['servers']): EffectiveLspSettings['servers'] { + return Object.fromEntries( + Object.entries(registry).map(([language, server]) => [ + language, + { command: server.command, args: [...server.args] }, + ]), + ); +} + +function cloneScipIndexerRegistry(registry: EffectiveScipSettings['indexers']): EffectiveScipSettings['indexers'] { + return Object.fromEntries( + Object.entries(registry).map(([language, indexer]) => [ + language, + { + command: indexer.command, + args: [...indexer.args], + ...(indexer.cwd ? { cwd: indexer.cwd } : {}), + }, + ]), + ); +} + +function mergeLspServerOverrides( + overrides: Record | undefined, +): EffectiveLspSettings['servers'] { + const merged = cloneLspServerRegistry(DEFAULT_LSP_SERVER_REGISTRY); + for (const [language, override] of Object.entries(overrides ?? {})) { + const base = merged[language]; + merged[language] = { + command: override.command ?? base?.command ?? '', + args: override.args ?? base?.args ?? [], + }; + } + return merged; +} + +export function buildLoreIndexSettings(kbIndex: KbIndexConfig | undefined): LoreIndexSettings { + return { + lsp: { + enabled: kbIndex?.lsp?.enabled ?? false, + requestTimeoutMs: kbIndex?.lsp?.requestTimeoutMs ?? DEFAULT_LSP_REQUEST_TIMEOUT_MS, + servers: mergeLspServerOverrides(kbIndex?.lsp?.servers), + }, + scip: { + enabled: true, + timeoutMs: DEFAULT_SCIP_TIMEOUT_MS, + indexers: cloneScipIndexerRegistry(DEFAULT_SCIP_INDEXER_REGISTRY), + indexDir: null, + }, + }; +} \ No newline at end of file diff --git a/src/core/runtime.ts b/src/core/runtime.ts index 6bce851..122ac52 100644 --- a/src/core/runtime.ts +++ b/src/core/runtime.ts @@ -21,6 +21,7 @@ import { ContextBuilder } from '../agents/context-builder.js'; import { MetricsCollector } from '../observability/metrics-collector.js'; import { ReportGenerator } from '../observability/report-generator.js'; import { TargetIndexer } from './target-indexer.js'; +import { buildLoreIndexSettings } from './lore-index-settings.js'; import { FlowRunner, type FlowRunnerOptions } from '@cadre-dev/framework/flow'; import { migrationFlow, AamfFlowCheckpointAdapter, buildFlowUpToPhase, nodeIdToPhase } from '../flow/index.js'; import { MigrationError } from '../flow/steps/shared.js'; @@ -247,7 +248,13 @@ export class MigrationRuntime { const gitLimiter = pLimit(1); // Target codebase indexer - const targetIndexer = new TargetIndexer(this.paths.kbTargetDbFile, this.config.target.outputPath, this.logger); + const loreIndexSettings = buildLoreIndexSettings(this.config.options.kbIndex); + const targetIndexer = new TargetIndexer( + this.paths.kbTargetDbFile, + this.config.target.outputPath, + this.logger, + loreIndexSettings, + ); // If the target DB already exists (resume), mark the indexer as built. if (await fileExists(this.paths.kbTargetDbFile)) { diff --git a/src/core/target-indexer.ts b/src/core/target-indexer.ts index 341fb89..9aabe2b 100644 --- a/src/core/target-indexer.ts +++ b/src/core/target-indexer.ts @@ -11,19 +11,36 @@ */ import type { Logger } from '../logging/logger.js'; +import { buildLoreIndexSettings, type LoreIndexSettings } from './lore-index-settings.js'; export class TargetIndexer { private readonly dbPath: string; private readonly rootDir: string; private readonly logger: Logger; + private readonly indexSettings: LoreIndexSettings; private built = false; private building = false; private onFirstBuild?: () => Promise; - constructor(dbPath: string, rootDir: string, logger: Logger) { + constructor( + dbPath: string, + rootDir: string, + logger: Logger, + indexSettings: LoreIndexSettings = buildLoreIndexSettings(undefined), + ) { this.dbPath = dbPath; this.rootDir = rootDir; this.logger = logger; + this.indexSettings = indexSettings; + } + + private createBuilder(lore: typeof import('@jafreck/lore')): InstanceType { + return new lore.IndexBuilder( + this.dbPath, + { rootDir: this.rootDir }, + undefined, + this.indexSettings, + ); } /** Register a callback that fires once after the first build/update completes. */ @@ -34,7 +51,7 @@ export class TargetIndexer { /** Full build of the target index from scratch. */ async build(): Promise { const lore = await import('@jafreck/lore'); - const builder = new lore.IndexBuilder(this.dbPath, { rootDir: this.rootDir }); + const builder = this.createBuilder(lore); await builder.build(); this.built = true; this.logger.info('Target index built'); @@ -54,7 +71,7 @@ export class TargetIndexer { if (this.building) return; this.building = true; // First update — do a full build to establish the schema. - const builder = new lore.IndexBuilder(this.dbPath, { rootDir: this.rootDir }); + const builder = this.createBuilder(lore); await builder.build(); this.built = true; this.building = false; @@ -64,9 +81,16 @@ export class TargetIndexer { this.onFirstBuild = undefined; } } else { - const builder = new lore.IndexBuilder(this.dbPath, { rootDir: this.rootDir }); - await builder.update(changedFiles); - this.logger.debug(`Target index updated for ${changedFiles.length} file(s)`); + const builder = this.createBuilder(lore); + if (this.indexSettings.lsp.enabled) { + await builder.update(changedFiles); + this.logger.debug(`Target index updated for ${changedFiles.length} file(s)`); + } else { + await builder.baselineRebuild(); + this.logger.debug( + `Target index baseline rebuilt for ${changedFiles.length} changed file(s) because LSP is disabled`, + ); + } } } diff --git a/src/flow/steps/kb-indexing.ts b/src/flow/steps/kb-indexing.ts index 8d0aa6b..36abed2 100644 --- a/src/flow/steps/kb-indexing.ts +++ b/src/flow/steps/kb-indexing.ts @@ -14,6 +14,7 @@ import type { MigrationFlowContext } from '../context.js'; import type { PhaseResult } from '../../agents/types.js'; import { assertPhaseSuccess } from './shared.js'; import { startKbServer } from './kb-server-lifecycle.js'; +import { buildLoreIndexSettings } from '../../core/lore-index-settings.js'; import { fileExists } from '../../util/fs.js'; const loadLore = () => import('@jafreck/lore'); @@ -102,32 +103,23 @@ export async function buildKbIndex( } } - // ── LSP settings ── + const loreIndexSettings = buildLoreIndexSettings(ctx.config.options.kbIndex); const lspConfig = ctx.config.options.kbIndex?.lsp; - const lspSettings = lspConfig?.enabled ? { - enabled: true as const, - requestTimeoutMs: lspConfig.requestTimeoutMs ?? 5000, - servers: lspConfig.servers - ? Object.fromEntries( - Object.entries(lspConfig.servers).map(([lang, srv]) => [ - lang, { command: srv.command, args: srv.args ?? [] }, - ]), - ) - : {}, - } : undefined; - - if (lspSettings) { + + if (loreIndexSettings.lsp.enabled) { ctx.logger.info( - `LSP enabled (timeout: ${lspSettings.requestTimeoutMs}ms` + - (Object.keys(lspSettings.servers).length > 0 - ? `, servers: ${Object.keys(lspSettings.servers).join(', ')}` : '') + ')', + `LSP enabled (timeout: ${loreIndexSettings.lsp.requestTimeoutMs}ms` + + (Object.keys(lspConfig?.servers ?? {}).length > 0 + ? `, overrides: ${Object.keys(lspConfig?.servers ?? {}).join(', ')}` : '') + ')', ); - for (const [lang, srv] of Object.entries(lspSettings.servers)) { + for (const [lang, srv] of Object.entries(lspConfig?.servers ?? {})) { try { execFileSync('which', [srv.command], { stdio: 'pipe' }); } catch { ctx.logger.warn(`LSP server '${srv.command}' for '${lang}' not found on PATH`); } } } + ctx.logger.info(`SCIP enabled (timeout: ${loreIndexSettings.scip.timeoutMs}ms)`); + // ── Logger init ── const loreLogLevel = ctx.config.options.kbIndex?.logLevel ?? 'debug'; lore.initLogger({ @@ -135,7 +127,7 @@ export async function buildKbIndex( logFile: ctx.paths.loreLogFile, }); - const builder = new lore.IndexBuilder(kbDbPath, walkerConfig, ctx.embedder, { lsp: lspSettings }); + const builder = new lore.IndexBuilder(kbDbPath, walkerConfig, ctx.embedder, loreIndexSettings); // ── Retry loop ── const maxAttempts = ctx.config.options.maxRetriesPerTask; @@ -161,7 +153,7 @@ export async function buildKbIndex( ctx.logger.warn( `KB index build still running after ${Math.round(halfTimeout / 1000)}s ` + `(timeout: ${Math.round(timeout / 1000)}s)` + - (lspSettings ? ' — LSP server may still be indexing.' : ''), + (loreIndexSettings.lsp.enabled ? ' — LSP server may still be indexing.' : ''), ); }, halfTimeout), ); @@ -173,7 +165,7 @@ export async function buildKbIndex( new Promise((_, reject) => setTimeout(() => { clearHeartbeat(); - const msg = lspSettings + const msg = loreIndexSettings.lsp.enabled ? `KB index timed out after ${Math.round(timeout / 1000)}s — LSP may be stalled.` : `KB index timed out after ${Math.round(timeout / 1000)}s`; reject(new Error(msg)); diff --git a/src/flow/steps/migration.ts b/src/flow/steps/migration.ts index 55adc41..cfa81c3 100644 --- a/src/flow/steps/migration.ts +++ b/src/flow/steps/migration.ts @@ -240,6 +240,7 @@ async function runCommitSubstep( ctx: MigrationFlowContext, task: MigrationTask, ): Promise { await commitForAgent(ctx, 'code-migrator', 5, task.id, task.name); + await ctx.checkpoint.completeTask(task.id); } async function runTargetIndexSubstep( @@ -552,6 +553,7 @@ function buildPerTaskFlow( await ctx.progress.updateTask(task.id, 'completed', { sourceFiles: task.sourceFiles, targetFiles: task.targetFiles }); ctx.logger.event({ type: 'task-completed', taskId: task.id, name: task.name, duration: 0 }); await commitForTask(c.context, task); + await ctx.checkpoint.completeTask(task.id); }, })); } diff --git a/tests/agents/plan-parser.test.ts b/tests/agents/plan-parser.test.ts index 77b5d2c..55862ef 100644 --- a/tests/agents/plan-parser.test.ts +++ b/tests/agents/plan-parser.test.ts @@ -557,6 +557,31 @@ intermediate text } }); + it('should normalize legacy success status and notes arrays', () => { + const stdout = `\`\`\`aamf-json +{"status":"success","written":["strategy.md","compilation-units.json"],"notes":["First note","Second note"]} +\`\`\``; + const result = parseAamfOutput(stdout, AamfOutputBase); + expect(result.parsed).toBe(true); + if (result.parsed) { + expect(result.data.status).toBe('completed'); + expect(result.data.notes).toBe('First note\nSecond note'); + expect((result.data as Record).outputFiles).toEqual([ + 'strategy.md', + 'compilation-units.json', + ]); + } + }); + + it('should normalize needs_review status alias', () => { + const stdout = '```aamf-json\n{"status":"needs_review"}\n```'; + const result = parseAamfOutput(stdout, MigrationPlannerSchema); + expect(result.parsed).toBe(true); + if (result.parsed) { + expect(result.data.status).toBe('needs-review'); + } + }); + it('should handle CRLF line endings in the fenced block', () => { const stdout = '```aamf-json\r\n{"status":"completed"}\r\n```'; const result = parseAamfOutput(stdout, AdjudicatorSchema); diff --git a/tests/core/checkpoint.test.ts b/tests/core/checkpoint.test.ts index c9199ee..de441cd 100644 --- a/tests/core/checkpoint.test.ts +++ b/tests/core/checkpoint.test.ts @@ -1038,6 +1038,79 @@ describe('CheckpointManager', () => { expect(resumed.blockedTasks).toEqual([]); }); + it('should back-fill completedTasks from Phase 4 commit entries when completedTasks is empty', async () => { + const { writeJson } = await import('../../src/util/fs.js'); + const checkpoint = { + projectName: 'test-backfill-resume', + version: 1, + currentPhase: 4, + currentTask: null, + completedPhases: [0, 1, 2, 3], + completedTasks: [], // BUG: completeTask was never called + failedTasks: [{ taskId: 'task-bad-0', attempts: 2, lastError: 'Exit code: null', recoveryAttempted: false }], + blockedTasks: [], + phaseOutputs: {}, + tokenUsage: { total: 0, byPhase: {}, byAgent: {} }, + startedAt: new Date().toISOString(), + lastCheckpoint: new Date().toISOString(), + resumeCount: 0, + cumulativeDurationMs: 0, + completedTaskDurationsMs: [], + metricsCount: 0, + __flowCheckpoint: { + flowId: 'aamf-migration', + status: 'running', + startedAt: new Date().toISOString(), + updatedAt: new Date().toISOString(), + completedExecutionIds: ['aamf-migration/kb-index'], + outputs: {}, + executionOutputs: {}, + }, + __phase4FlowCheckpoint: { + flowId: 'phase-4-sync-epoch', + status: 'running', + startedAt: new Date().toISOString(), + updatedAt: new Date().toISOString(), + completedExecutionIds: [ + 'phase-4-sync-epoch/epoch-0-start', + // Committed task A — all substeps present + 'phase-4-sync-epoch/epoch-0-tasks-batch-0/task-ok-0/task-ok-0/migrate', + 'phase-4-sync-epoch/epoch-0-tasks-batch-0/task-ok-0/task-ok-0/commit', + 'phase-4-sync-epoch/epoch-0-tasks-batch-0/task-ok-0/task-ok-0/target-index', + 'phase-4-sync-epoch/epoch-0-tasks-batch-0/task-ok-0/task-ok-0/parity', + // Committed task B + 'phase-4-sync-epoch/epoch-0-tasks-batch-1/task-ok-1/task-ok-1/migrate', + 'phase-4-sync-epoch/epoch-0-tasks-batch-1/task-ok-1/task-ok-1/commit', + 'phase-4-sync-epoch/epoch-0-tasks-batch-1/task-ok-1/task-ok-1/parity', + // Failed task — migrated but no commit + 'phase-4-sync-epoch/epoch-0-tasks-batch-0/task-bad-0/task-bad-0/migrate', + ], + outputs: {}, + executionOutputs: {}, + }, + }; + await ensureDir(join(tempDir, 'state')); + await writeJson(join(tempDir, 'state', 'checkpoint.json'), checkpoint); + + const manager3 = new CheckpointManager(tempDir, logger); + const resumed = await manager3.load('test-backfill-resume'); + + // completedTasks should be back-filled from commit entries + expect(resumed.completedTasks).toContain('task-ok-0'); + expect(resumed.completedTasks).toContain('task-ok-1'); + expect(resumed.completedTasks).not.toContain('task-bad-0'); + + // Phase 4 flow checkpoint should preserve committed task entries + const p4fc = resumed.__phase4FlowCheckpoint as Record; + const p4Ids = p4fc.completedExecutionIds as string[]; + expect(p4Ids).toContain('phase-4-sync-epoch/epoch-0-start'); + expect(p4Ids).toContain('phase-4-sync-epoch/epoch-0-tasks-batch-0/task-ok-0/task-ok-0/migrate'); + expect(p4Ids).toContain('phase-4-sync-epoch/epoch-0-tasks-batch-0/task-ok-0/task-ok-0/commit'); + expect(p4Ids).toContain('phase-4-sync-epoch/epoch-0-tasks-batch-1/task-ok-1/task-ok-1/migrate'); + // Failed task entries should be removed + expect(p4Ids).not.toContain('phase-4-sync-epoch/epoch-0-tasks-batch-0/task-bad-0/task-bad-0/migrate'); + }); + it('should not alter flow checkpoint when status is not failed', async () => { const { writeJson } = await import('../../src/util/fs.js'); const checkpoint = { diff --git a/tests/core/lore-index-settings.test.ts b/tests/core/lore-index-settings.test.ts new file mode 100644 index 0000000..b7fccf6 --- /dev/null +++ b/tests/core/lore-index-settings.test.ts @@ -0,0 +1,33 @@ +import { describe, expect, it } from 'vitest'; +import { buildLoreIndexSettings } from '../../src/core/lore-index-settings.js'; + +describe('buildLoreIndexSettings', () => { + it('always enables SCIP with Lore 0.4 defaults', () => { + const settings = buildLoreIndexSettings(undefined); + + expect(settings.scip.enabled).toBe(true); + expect(settings.scip.timeoutMs).toBe(120_000); + expect(settings.scip.indexDir).toBeNull(); + expect(settings.scip.indexers.c.command).toBe('scip-clang'); + expect(settings.scip.indexers.rust.args).toEqual(['scip', '.']); + }); + + it('keeps LSP opt-in and merges custom server overrides', () => { + const kbIndex = { + lsp: { + enabled: true, + requestTimeoutMs: 9_000, + servers: { + c: { command: 'clangd', args: ['--compile-commands-dir=build'] }, + }, + }, + } satisfies NonNullable[0]>; + + const settings = buildLoreIndexSettings(kbIndex); + + expect(settings.lsp.enabled).toBe(true); + expect(settings.lsp.requestTimeoutMs).toBe(9_000); + expect(settings.lsp.servers.c.args).toEqual(['--compile-commands-dir=build']); + expect(settings.lsp.servers.rust.command).toBe('rust-analyzer'); + }); +}); \ No newline at end of file diff --git a/tests/core/target-indexer-lore-settings.test.ts b/tests/core/target-indexer-lore-settings.test.ts new file mode 100644 index 0000000..599074b --- /dev/null +++ b/tests/core/target-indexer-lore-settings.test.ts @@ -0,0 +1,58 @@ +import { describe, expect, it, vi, beforeEach } from 'vitest'; +import { TargetIndexer } from '../../src/core/target-indexer.js'; +import { buildLoreIndexSettings } from '../../src/core/lore-index-settings.js'; +import { createSilentLogger } from '../helpers/mocks.js'; + +const builderMethods = vi.hoisted(() => ({ + build: vi.fn().mockResolvedValue(undefined), + update: vi.fn().mockResolvedValue(undefined), + baselineRebuild: vi.fn().mockResolvedValue(undefined), +})); + +vi.mock('@jafreck/lore', () => ({ + IndexBuilder: class { + build = builderMethods.build; + update = builderMethods.update; + baselineRebuild = builderMethods.baselineRebuild; + }, +})); + +describe('TargetIndexer Lore integration', () => { + beforeEach(() => { + builderMethods.build.mockClear(); + builderMethods.update.mockClear(); + builderMethods.baselineRebuild.mockClear(); + }); + + it('uses baseline rebuilds for follow-up updates when LSP is disabled', async () => { + const logger = createSilentLogger(process.cwd()); + const indexer = new TargetIndexer( + '/tmp/aamf-target.db', + '/tmp/aamf-target-root', + logger, + buildLoreIndexSettings(undefined), + ); + + indexer.markBuilt(); + await indexer.updateForFiles(['/tmp/aamf-target-root/lib.rs']); + + expect(builderMethods.update).not.toHaveBeenCalled(); + expect(builderMethods.baselineRebuild).toHaveBeenCalledTimes(1); + }); + + it('uses overlay updates when LSP is enabled', async () => { + const logger = createSilentLogger(process.cwd()); + const indexer = new TargetIndexer( + '/tmp/aamf-target.db', + '/tmp/aamf-target-root', + logger, + buildLoreIndexSettings({ lsp: { enabled: true } }), + ); + + indexer.markBuilt(); + await indexer.updateForFiles(['/tmp/aamf-target-root/lib.rs']); + + expect(builderMethods.update).toHaveBeenCalledTimes(1); + expect(builderMethods.baselineRebuild).not.toHaveBeenCalled(); + }); +}); \ No newline at end of file diff --git a/tests/core/task-graph-builder.test.ts b/tests/core/task-graph-builder.test.ts index 4724e10..9c9a961 100644 --- a/tests/core/task-graph-builder.test.ts +++ b/tests/core/task-graph-builder.test.ts @@ -36,7 +36,8 @@ function createTestDb(dbPath: string): Database.Database { start_line INTEGER NOT NULL, end_line INTEGER NOT NULL, signature TEXT, doc_comment TEXT, resolved_type_signature TEXT, resolved_return_type TEXT, - definition_uri TEXT, definition_path TEXT + definition_uri TEXT, definition_path TEXT, + parent_symbol_id INTEGER REFERENCES symbols(id) ); CREATE TABLE IF NOT EXISTS symbol_refs ( id INTEGER PRIMARY KEY AUTOINCREMENT, diff --git a/tests/fixtures/zstd-c-project/migration.config.json b/tests/fixtures/zstd-c-project/migration.config.json index ddd5998..0509863 100644 --- a/tests/fixtures/zstd-c-project/migration.config.json +++ b/tests/fixtures/zstd-c-project/migration.config.json @@ -2,10 +2,10 @@ "projectName": "zstd-to-rust", "guidance": [ "Do NOT use any existing Rust crates that wrap the C implementation (e.g. zstd, zstd-safe, zstd-sys, lz4-sys). Write a pure native Rust port of the C source code.", - "Do NOT use FFI, bindgen, or any C interop. All code must be idiomatic safe Rust.", - "All code must be safe Rust. Do NOT use `unsafe` blocks, `unsafe fn`, raw pointers (`*const`, `*mut`), or `core::ffi::c_void`. Use slices (`&[u8]`, `&mut [u8]`), Vec, iterators, and Rust's standard byte-order methods (e.g. `u32::from_le_bytes`, `to_be_bytes`) instead of pointer casts and manual memory access. If a C pattern seems to require unsafe, find the safe Rust equivalent — it almost always exists.", + "Do NOT use bindgen or delegate to the original C implementation through FFI. If a platform or OS boundary truly has no safe Rust equivalent, a narrowly-scoped Rust-side ABI shim is allowed only at the leaf boundary, with the rest of the port remaining native Rust.", + "Prefer safe Rust everywhere. You may use audited `unsafe` blocks, `unsafe fn`, raw pointers (`*const`, `*mut`), or `core::ffi::c_void` only when no safe equivalent exists and the behavior is required for parity. Keep unsafe localized to the smallest possible boundary, document the invariant being relied on, and expose a safe API to the rest of the code.", "Preserve the algorithmic logic (same algorithmic steps, same data flow) so the port is auditable against the C reference — but use idiomatic Rust types and APIs. The algorithm should be recognizable, not the function signatures or pointer patterns.", - "Prefer correctness and safety over micro-optimization. It is acceptable for the Rust port to be slightly slower than the C original if the alternative is unsafe code. Do not use unsafe for performance reasons." + "Prefer correctness and safety over micro-optimization. It is acceptable for the Rust port to be slightly slower than the C original if the alternative is broader unsafe code. Do not use unsafe for convenience or performance reasons; if unsafe is used at all, it must be to preserve required behavior at a constrained boundary." ], "source": { "path": "./zstd-src/zstd-1.5.7", @@ -74,7 +74,7 @@ "enabled": false }, "lsp": { - "enabled": true, + "enabled": false, "servers": { "c": { "command": "clangd", "args": ["--compile-commands-dir=zstd-src/zstd-1.5.7/build"] } } diff --git a/tests/lore/kb-search-tool.test.ts b/tests/lore/kb-search-tool.test.ts index e510669..23df762 100644 --- a/tests/lore/kb-search-tool.test.ts +++ b/tests/lore/kb-search-tool.test.ts @@ -105,8 +105,9 @@ describe('kb search tool handler', () => { const result = await handler(db, { query: 'concept', mode: 'semantic' }, embedder as any); expect(result.mode_used).toBe('semantic'); - // v0.2.1 re-sorts semantic results ascending by score - expect(result.results.map((r: any) => r.symbol_id)).toEqual([2, 3]); + // Semantic results are returned in the order provided by the DB query; + // the lore library no longer re-sorts them. + expect(result.results.map((r: any) => r.symbol_id)).toEqual([3, 2]); expect(embedder.embed).toHaveBeenCalledWith(['concept']); }); diff --git a/tests/lore/kb-server.test.ts b/tests/lore/kb-server.test.ts index ad28ab0..294d7a2 100644 --- a/tests/lore/kb-server.test.ts +++ b/tests/lore/kb-server.test.ts @@ -65,6 +65,15 @@ beforeAll(async () => { } catch { // FTS table may not exist in all Lore versions — non-fatal. } + // Populate symbol_metrics so the metrics handler test works. + try { + rwDb.prepare( + `INSERT INTO symbol_metrics (symbol_id, line_count, param_count, cyclomatic, max_nesting) + VALUES (?, 20, 0, 1, 0)`, + ).run(symId); + } catch { + // symbol_metrics may not exist — non-fatal. + } } } rwDb.close();