Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,25 @@
- package provenance/license verification and immutable GitHub release artifact creation;
- weekly Dependabot and CodeQL scanning workflows.
- deterministic filesystem race fixtures for changed content, permission mode, replacement identity, deletion, and missing-to-created destinations.
- exported filesystem durability levels, defaults, configuration, and post-rename `FilesystemDurabilityError` recovery metadata;
- deterministic ordering/failure seams and cross-platform tests for file sync, parent-directory capability, strict/degraded behavior, visibility, and cleanup.
- repository ADR defining crash-durability scope and failure semantics.

### Changed

- filesystem edits now classify paths before decoding and use same-directory atomic replacement with mode, BOM, newline, and cleanup guarantees;
- package contents exclude compiled tests and include complete repository provenance metadata.
- filesystem atomic replacement now performs best-effort optimistic destination revalidation with identity, permission-mode, and SHA-256 byte-digest evidence, returning `[E_CONCURRENT_DESTINATION]` while preserving detected concurrent state and cleaning temporary files.
- callers can select `none`, `file`, or `file-and-parent-directory` durability through `FilesystemPiClient`; the default remains file sync.
- preserved destination mode is now applied before the final temporary-file sync, and parent-directory capability uses a retained pre-rename handle with identity checks around rename.

### Security

- unsafe binary, image, special-file, symlink, null-byte, and lossy UTF-8 rewrites are rejected before writing;
- security reporting, sensitive-diagnostic handling, dependency review, and recovery responsibilities are documented.
- same-size, coarse-timestamp, and permission-only destination changes are no longer silently overwritten when detected before replacement; documentation explicitly records the residual check-to-rename race and makes no compare-and-swap guarantee.
- post-rename sync failures now explicitly report that the destination is visible but crash durability is unconfirmed, preventing unsafe blind retry/rollback assumptions;
- documentation distinguishes atomic visibility, file durability, parent-directory durability, and the unsupported scope of recursively created ancestors.

## 0.1.0

Expand Down
27 changes: 25 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Provide one dependable cross-platform edit core where every edit applies to the
- File-kind and text-loading helpers for text/binary/empty-file handling.
- Text and newline normalization used by cross-platform edit tools.
- A local filesystem Pi-style client for tests and retry-oriented tooling.
- Selectable filesystem crash-durability levels with the existing file-sync behavior retained by default.
- TypeScript declarations and ESM output under `dist/`.

## Quickstart
Expand Down Expand Up @@ -44,6 +45,23 @@ const client = new FilesystemPiClient();
const rendered = await client.read({ path: 'src/file.ts' });
```

Choose durability explicitly only when the default file sync is not the desired trade-off:

```ts
import { FILESYSTEM_DURABILITY_LEVELS, FilesystemPiClient } from 'pi-anchor-edit-core';

const fastest = new FilesystemPiClient({
durability: FILESYSTEM_DURABILITY_LEVELS.NONE,
});

const renameDurable = new FilesystemPiClient({
durability: FILESYSTEM_DURABILITY_LEVELS.FILE_AND_PARENT_DIRECTORY,
unsupportedDirectorySync: 'strict',
});
```

The levels are `none`, `file`, and `file-and-parent-directory`; `DEFAULT_FILESYSTEM_DURABILITY` is `file`. Parent-directory sync support is detected from the real operation. Use `strict` when unsupported capability must throw; the default `degrade` policy completes at file durability when the operation is known to be unsupported.

Copy anchors verbatim from `rendered`; they are opaque observations.

### 4. Apply a verified edit
Expand Down Expand Up @@ -93,6 +111,10 @@ The destination changed after the filesystem adapter loaded it and before atomic

This is best-effort optimistic detection, not compare-and-swap. A residual race remains between the final revalidation and `rename`, so a successful edit is not a guarantee that no concurrent writer intervened.

### `[E_DIRECTORY_SYNC_UNSUPPORTED]` / `[E_DURABILITY_UNCONFIRMED]`

`FilesystemDurabilityError` distinguishes the commit boundary. If parent preparation/identity verification fails before rename, `destinationVisible === false`; if the pinned parent fails identity verification or sync after rename, `destinationVisible === true` and crash durability was not confirmed. Do not blindly replay or roll back a committed edit—re-read first. A classified unsupported operation is absorbed only when `unsupportedDirectorySync: 'degrade'`; parent changes and other failures always throw.

## Development

```bash
Expand All @@ -108,15 +130,16 @@ npm run benchmark

### Supported matrix

CI runs Node.js 22 on Ubuntu, Windows, and macOS, plus the Node.js 24 compatibility job on Ubuntu. Capability-sensitive symlink and permission assertions report a specific diagnostic when the host cannot provide that feature; unrelated filesystem and CRLF assertions continue to run. The thresholded coverage command enforces at least 85% line coverage and 75% branch coverage.
CI runs Node.js 22 on Ubuntu, Windows, and macOS, plus the Node.js 24 compatibility job on Ubuntu. Capability-sensitive symlink and permission assertions report a specific diagnostic when the host cannot provide that feature; unrelated filesystem and CRLF assertions continue to run. Durability tests assert successful real parent-directory sync on hosted Linux/macOS and Windows `EPERM` degradation/strict classification, in addition to deterministic injected failure fixtures. The thresholded coverage command enforces at least 85% line coverage and 75% branch coverage.

## Documentation

- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — components, control flow, and invariants.
- [`docs/adr/0001-filesystem-crash-durability.md`](docs/adr/0001-filesystem-crash-durability.md) — durability levels, ordering, capability degradation, and post-rename failure contract.
- [`docs/BRANCH_PROTECTION.md`](docs/BRANCH_PROTECTION.md) — required review/check policy and read-only verification.
- [`docs/OPERATIONS.md`](docs/OPERATIONS.md) — classified errors, filesystem recovery, and safe escalation.
- [`docs/EXAMPLES.md`](docs/EXAMPLES.md) — parsing, editing, recovery, and adapter examples.
- [`docs/PERFORMANCE.md`](docs/PERFORMANCE.md) — reproducible hash baseline.
- [`docs/PERFORMANCE.md`](docs/PERFORMANCE.md) — reproducible hash and filesystem durability benchmarks.
- [`docs/PRODUCT.md`](docs/PRODUCT.md) — vision and success metrics.
- [`docs/RELEASING.md`](docs/RELEASING.md) — package verification, immutable tags, release creation, and recovery.
- [`CONTRIBUTING.md`](CONTRIBUTING.md) — contributor workflow.
Expand Down
4 changes: 3 additions & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,12 @@ We aim to acknowledge a private report within 3 business days, provide an initia

## Trust boundary

`pi-anchor-edit-core` reads and mutates caller-selected local paths. The caller is responsible for authorization, path selection, backups, and preventing untrusted users from choosing sensitive targets. The filesystem adapter rejects symbolic links, special files, directories, images, binary/null-byte content, and invalid UTF-8 rewrites; it does not create a sandbox or establish that a path is safe to edit.
`pi-anchor-edit-core` reads and mutates caller-selected local paths. The caller is responsible for authorization, path selection, parent-path components, backups, and preventing untrusted users from choosing sensitive targets. The filesystem adapter rejects a symbolic link at the destination, plus special files, directories, images, binary/null-byte content, and invalid UTF-8 rewrites; parent-directory components follow normal OS resolution and may be symlinks. It does not create a sandbox or establish that a path is safe to edit.

Anchor diagnostics can quote nearby file content in `>>> LINE#HASH:content` retry lines. Treat all diagnostics as potentially sensitive. Redact or replace them with synthetic examples before sharing, and never send raw diagnostics to telemetry by default.

Filesystem sync is a durability control, not an authorization or confidentiality boundary. `none` intentionally omits explicit syncs; the default `file` level does not make the directory rename crash-durable; and `file-and-parent-directory` depends on operating-system, filesystem, mount, virtualization, and hardware behavior. A degraded unsupported directory sync confirms only file durability. Strict failures expose `destinationVisible`; visible post-rename failures must not trigger blind replay or rollback.

See [`docs/OPERATIONS.md`](docs/OPERATIONS.md) for safe recovery actions and [`docs/RELEASING.md`](docs/RELEASING.md) for immutable release recovery.

## Dependency and scanning maintenance
Expand Down
82 changes: 67 additions & 15 deletions benchmarks/core.mjs
Original file line number Diff line number Diff line change
@@ -1,30 +1,82 @@
import { performance } from 'node:perf_hooks';
import { computeLineHash } from '../dist/src/index.js';
import { mkdtemp, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
FILESYSTEM_DURABILITY_LEVELS,
FilesystemPiClient,
computeLineHash,
} from '../dist/src/index.js';

const samples = [];
const rounds = 100;
const operationsPerRound = 10_000;
function summarize(samples) {
const sorted = [...samples].sort((left, right) => left - right);
return {
meanMilliseconds: sorted.reduce((sum, value) => sum + value, 0) / sorted.length,
p95Milliseconds: sorted[Math.ceil(sorted.length * 0.95) - 1],
p99Milliseconds: sorted[Math.ceil(sorted.length * 0.99) - 1],
};
}

const hashSamples = [];
const hashRounds = 100;
const operationsPerHashRound = 10_000;

for (let round = 0; round < rounds; round += 1) {
for (let round = 0; round < hashRounds; round += 1) {
const start = performance.now();
for (let index = 0; index < operationsPerRound; index += 1) {
for (let index = 0; index < operationsPerHashRound; index += 1) {
computeLineHash(index + 1, `const value${index} = ${index};`);
}
samples.push((performance.now() - start) / operationsPerRound);
hashSamples.push((performance.now() - start) / operationsPerHashRound);
}

samples.sort((a, b) => a - b);
const mean = samples.reduce((sum, value) => sum + value, 0) / samples.length;
const p99 = samples[Math.ceil(samples.length * 0.99) - 1];
const directory = await mkdtemp(join(tmpdir(), 'pi-anchor-edit-core-benchmark-'));
const editRounds = 25;
const durability = {};

try {
for (const level of Object.values(FILESYSTEM_DURABILITY_LEVELS)) {
const path = join(directory, `${level}.txt`);
await writeFile(path, 'value-a');
const client = new FilesystemPiClient({
durability: level,
unsupportedDirectorySync: 'degrade',
});
let from = 'value-a';
let to = 'value-b';
const samples = [];

for (let round = 0; round < editRounds + 3; round += 1) {
const start = performance.now();
await client.edit({
path,
edits: [{ op: 'replace_text', oldText: from, newText: to }],
});
const elapsed = performance.now() - start;
[from, to] = [to, from];
if (round >= 3) samples.push(elapsed);
}

durability[level] = summarize(samples);
}
} finally {
await rm(directory, { recursive: true, force: true });
}

console.log(
JSON.stringify(
{
operation: 'computeLineHash',
rounds,
operationsPerRound,
meanMilliseconds: mean,
p99Milliseconds: p99,
runtime: { node: process.version, platform: process.platform, arch: process.arch },
computeLineHash: {
rounds: hashRounds,
operationsPerRound: operationsPerHashRound,
...summarize(hashSamples),
},
filesystemAtomicEdit: {
roundsPerLevel: editRounds,
includes: 'load, classify, transform, temp write, selected syncs, revalidation, and rename',
unsupportedDirectorySync: 'degrade',
durability,
},
},
null,
2,
Expand Down
37 changes: 37 additions & 0 deletions dist/src/filesystem-client.d.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,44 @@
import type { FileHandle } from 'node:fs/promises';
import type { EditParams, PiClient, ReadParams } from './types.js';
export declare const FILESYSTEM_DURABILITY_LEVELS: {
readonly NONE: "none";
readonly FILE: "file";
readonly FILE_AND_PARENT_DIRECTORY: "file-and-parent-directory";
};
export type FilesystemDurability = typeof FILESYSTEM_DURABILITY_LEVELS[keyof typeof FILESYSTEM_DURABILITY_LEVELS];
export declare const DEFAULT_FILESYSTEM_DURABILITY: FilesystemDurability;
export declare const UNSUPPORTED_DIRECTORY_SYNC_BEHAVIORS: {
readonly DEGRADE: "degrade";
readonly STRICT: "strict";
};
export type UnsupportedDirectorySyncBehavior = typeof UNSUPPORTED_DIRECTORY_SYNC_BEHAVIORS[keyof typeof UNSUPPORTED_DIRECTORY_SYNC_BEHAVIORS];
export declare const DEFAULT_UNSUPPORTED_DIRECTORY_SYNC_BEHAVIOR: UnsupportedDirectorySyncBehavior;
export type FilesystemPiClientConfig = {
durability?: FilesystemDurability;
unsupportedDirectorySync?: UnsupportedDirectorySyncBehavior;
};
export type FilesystemDurabilityErrorCode = 'E_DIRECTORY_SYNC_UNSUPPORTED' | 'E_DURABILITY_UNCONFIRMED';
export declare class FilesystemDurabilityError extends Error {
readonly code: FilesystemDurabilityErrorCode;
readonly destinationPath: string;
readonly durability: FilesystemDurability;
readonly destinationVisible: boolean;
constructor(code: FilesystemDurabilityErrorCode, destinationPath: string, durability: FilesystemDurability, destinationVisible: boolean, cause: unknown);
}
export declare class FilesystemPiClient implements PiClient {
private readonly durability;
private readonly unsupportedDirectorySync;
constructor(config?: FilesystemPiClientConfig);
protected beforeDestinationRevalidation(_destinationPath: string): Promise<void>;
protected applyTemporaryFileMode(temporaryPath: string, mode: number): Promise<void>;
protected synchronizeTemporaryFile(handle: FileHandle): Promise<void>;
protected replaceTemporaryFile(temporaryPath: string, destinationPath: string): Promise<void>;
protected openParentDirectoryForSync(parentPath: string): Promise<FileHandle>;
protected synchronizeParentDirectory(handle: FileHandle, _parentPath: string): Promise<void>;
private handleDirectorySyncFailure;
private openParentBeforeRename;
private verifyPinnedParent;
private synchronizeParentAfterRename;
private observeDestination;
private atomicWrite;
read({ path, offset, limit }: ReadParams): Promise<string>;
Expand Down
Loading
Loading