Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

39 changes: 39 additions & 0 deletions docs/validation-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,45 @@ Codes that fire when `vat resources validate` checks external `http(s)://` links
- **Why it matters:** A network-level failure often reflects a transient DNS or connectivity problem on the validating machine rather than a genuinely broken link, so it is surfaced as a warning rather than blocking the build.
- **Fix:** Check the host (DNS, reachability, certificate); or set `severity.EXTERNAL_URL_ERROR` to `ignore` if the host is intentionally unreachable from the validation environment.

## Authenticated External Link Codes

Codes that fire when `vat resources validate` checks external links whose host is claimed by a provider in `resources.linkAuth` (e.g. private GitHub repos, SharePoint files). The validator issues an authenticated request via the configured token source and classifies the response; outcome-to-code mapping is per-provider because hosts disagree about what a 404 means (GitHub masks `403 access-denied` as `404`; Microsoft Graph distinguishes cleanly). All five codes are configurable like any other code (`validation.severity` / `validation.allow`); the provider's `check` block routes an outcome to a *code*, never to a *severity*.

### `LINK_AUTH_DEAD`

- **Default:** `error`
- **What:** An authenticated external link returned `404` or `410` from a host whose provider declares `notFoundMeaning: dead` (i.e. honest-404 hosts like SharePoint). The request was authenticated, so `404` here is not a permissions problem — the resource is missing.
- **Why it matters:** Unlike the anonymous external-URL check, an authenticated 404 against an honest-404 host is high-confidence evidence of link rot, which is why this is the only `LINK_AUTH_*` code that defaults to `error`. The provider declares this property; hosts that mask access-denied as 404 instead use [`LINK_AUTH_DEAD_OR_UNAUTHORIZED`](#link_auth_dead_or_unauthorized).
- **Fix:** Fix or remove the link; or set `severity.LINK_AUTH_DEAD` to `ignore` if the path is expected to be transient.

### `LINK_AUTH_DEAD_OR_UNAUTHORIZED`

- **Default:** `warning`
- **What:** An authenticated external link returned `404` from a host whose provider declares `notFoundMeaning: ambiguous` (i.e. hosts that mask `403 access-denied` as `404`, like GitHub). The link is either rotted *or* inaccessible to the current identity — the response alone cannot distinguish.
- **Why it matters:** GitHub deliberately does not leak existence of private resources via `403`; both "the file doesn't exist" and "you don't have permission to see this file" return `404`. The checker cannot prove rot, so this is a warning rather than an error. Contributors on a repo where some links point at resources they lack access to will see warnings, not red builds.
- **Fix:** Verify the URL by hand (or with a more-privileged token) to disambiguate; fix or remove if rotted; or set `severity.LINK_AUTH_DEAD_OR_UNAUTHORIZED` to `ignore` if cross-identity ambiguity is expected.

### `LINK_AUTH_FORBIDDEN`

- **Default:** `warning`
- **What:** An authenticated external link returned `403`: the configured identity is authenticated, but the resource refuses access. (Only fires for hosts that emit honest `403`s — see `LINK_AUTH_DEAD_OR_UNAUTHORIZED` for hosts that mask access-denied as `404`.)
- **Why it matters:** A `403` is a permissions signal, not link rot. Treating it as an error would block builds whenever a contributor lacks access to some linked resource — common on cross-team docs. The remedy is granting access or switching identity, not editing the link.
- **Fix:** Grant the identity access to the resource; switch to an identity that has access; or set `severity.LINK_AUTH_FORBIDDEN` to `ignore` if cross-identity inaccessibility is expected.

### `LINK_AUTH_UNAUTHORIZED`

- **Default:** `warning`
- **What:** An authenticated external link returned `401`: the token sent with the request was missing, expired, or invalid for the resource's auth scheme.
- **Why it matters:** Usually a configuration or session problem — the token source resolved a value, but the host rejected it. The link itself may be perfectly fine. Strict CI lanes that should fail on stale credentials can promote this to `error`.
- **Fix:** Refresh the token (e.g. `gh auth login`, `az login`); check the `token` config in `resources.linkAuth`; or promote `severity.LINK_AUTH_UNAUTHORIZED` to `error` on strict CI lanes.

### `LINK_AUTH_UNVERIFIED`

- **Default:** `warning`
- **What:** A provider in `resources.linkAuth` claims this host, but no token source resolved — none of the configured env vars or argv commands produced a non-empty value, so no authenticated request was attempted.
- **Why it matters:** External-URL checking is opt-in, so silently skipping links the project asked to check would be misleading. But `unverified` is not link rot — the link may be fine; the validator just couldn't authenticate. The fix is configuration, not link editing. (`unverified` outcomes are never cached, because the result flips the moment a token appears.)
- **Fix:** Configure a `token` source (env var or argv command); log in to the underlying CLI (e.g. `gh auth login`, `az login`); or set `severity.LINK_AUTH_UNVERIFIED` to `ignore` if running without auth is intentional.

## Packaging-Only Codes

*Stance: see [Packaging](./skill-quality-and-compatibility.md#packaging).*
Expand Down
14 changes: 12 additions & 2 deletions packages/agent-schema/schemas/validation-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,12 @@
"FRONTMATTER_UNKNOWN_LINK",
"EXTERNAL_URL_DEAD",
"EXTERNAL_URL_TIMEOUT",
"EXTERNAL_URL_ERROR"
"EXTERNAL_URL_ERROR",
"LINK_AUTH_DEAD",
"LINK_AUTH_DEAD_OR_UNAUTHORIZED",
"LINK_AUTH_FORBIDDEN",
"LINK_AUTH_UNAUTHORIZED",
"LINK_AUTH_UNVERIFIED"
]
}
},
Expand Down Expand Up @@ -175,7 +180,12 @@
"FRONTMATTER_UNKNOWN_LINK",
"EXTERNAL_URL_DEAD",
"EXTERNAL_URL_TIMEOUT",
"EXTERNAL_URL_ERROR"
"EXTERNAL_URL_ERROR",
"LINK_AUTH_DEAD",
"LINK_AUTH_DEAD_OR_UNAUTHORIZED",
"LINK_AUTH_FORBIDDEN",
"LINK_AUTH_UNAUTHORIZED",
"LINK_AUTH_UNVERIFIED"
]
}
}
Expand Down
30 changes: 30 additions & 0 deletions packages/agent-schema/src/validation-codes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,36 @@ export const CODE_REGISTRY = {
'Check the host; or set severity: ignore.',
'external_url_error',
),
LINK_AUTH_DEAD: entry(
'error',
'Authenticated external link returned 404/410 from a host that gives honest 404s (notFoundMeaning: dead). Genuine link rot.',
'Fix or remove the link; or set severity.LINK_AUTH_DEAD to ignore if the path is expected to be transient.',
'link_auth_dead',
),
LINK_AUTH_DEAD_OR_UNAUTHORIZED: entry(
'warning',
'Authenticated external link returned 404 from a host that masks access-denied as 404 (notFoundMeaning: ambiguous, e.g. GitHub). The link is either rotted or inaccessible to the current identity — cannot tell which.',
'Verify the URL by hand (or with a more-privileged token) to disambiguate; fix or remove if rotted; or set severity.LINK_AUTH_DEAD_OR_UNAUTHORIZED to ignore if cross-identity ambiguity is expected.',
'link_auth_dead_or_unauthorized',
),
LINK_AUTH_FORBIDDEN: entry(
'warning',
'Authenticated external link returned 403: the configured identity is authenticated but lacks access to that resource. Not link rot.',
'Grant the identity access to the resource; switch to an identity that has access; or set severity.LINK_AUTH_FORBIDDEN to ignore if cross-identity inaccessibility is expected.',
'link_auth_forbidden',
),
LINK_AUTH_UNAUTHORIZED: entry(
'warning',
'Authenticated external link returned 401: the configured token is missing, expired, or invalid.',
"Refresh the token (e.g. `gh auth login`, `az login`); check the `token` config in resources.linkAuth; or promote severity.LINK_AUTH_UNAUTHORIZED to error on strict CI lanes.",
'link_auth_unauthorized',
),
LINK_AUTH_UNVERIFIED: entry(
'warning',
'A provider in resources.linkAuth claims this host, but no token source resolved (none of the configured env/command sources produced a value).',
"Configure a `token` source (env var or argv command); log in to the underlying CLI (e.g. `gh auth login`, `az login`); or set severity.LINK_AUTH_UNVERIFIED to ignore if running without auth is intentional.",
'link_auth_unverified',
),
} as const satisfies Record<string, CodeRegistryEntry>;

export type IssueCode = keyof typeof CODE_REGISTRY;
Expand Down
25 changes: 25 additions & 0 deletions packages/agent-schema/test/docs/validation-codes.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import { readFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';

import { describe, expect, it } from 'vitest';

import { CODE_REGISTRY } from '../../src/validation-codes.js';

const docsPath = fileURLToPath(new URL('../../../../docs/validation-codes.md', import.meta.url));
// Path is derived from `import.meta.url`, not user input — points at the
// repo's docs/validation-codes.md, the source of truth for code reference anchors.
// eslint-disable-next-line security/detect-non-literal-fs-filename
const docs = readFileSync(docsPath, 'utf8');

describe('CODE_REGISTRY ↔ docs/validation-codes.md coverage', () => {
for (const code of Object.keys(CODE_REGISTRY)) {
it(`${code}: docs/validation-codes.md has the matching ### heading`, () => {
expect(docs).toContain(`### \`${code}\``);
});

it(`${code}: entry.reference is the lowercased code anchor`, () => {
const entry = CODE_REGISTRY[code as keyof typeof CODE_REGISTRY];
expect(entry.reference).toBe(`#${code.toLowerCase()}`);
});
}
});
29 changes: 29 additions & 0 deletions packages/agent-schema/test/validation-codes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,35 @@ describe('CODE_REGISTRY — capability and compat codes', () => {
});
});

describe('CODE_REGISTRY — link-auth codes (#113 slice 2)', () => {
it('registers all five LINK_AUTH_* codes', () => {
const codes: IssueCode[] = [
'LINK_AUTH_DEAD',
'LINK_AUTH_DEAD_OR_UNAUTHORIZED',
'LINK_AUTH_FORBIDDEN',
'LINK_AUTH_UNAUTHORIZED',
'LINK_AUTH_UNVERIFIED',
];
for (const code of codes) {
expect(CODE_REGISTRY[code], `registry missing ${code}`).toBeDefined();
expect(CODE_REGISTRY[code].description.length).toBeGreaterThan(10);
expect(CODE_REGISTRY[code].fix.length).toBeGreaterThan(10);
expect(CODE_REGISTRY[code].reference).toMatch(/^#link_auth_/);
}
});

it('enforces expected default severities per #113 §7', () => {
// Only LINK_AUTH_DEAD is an error (honest-404 host like SharePoint —
// 404 with valid auth is link rot). The four others default to warning;
// strict CI lanes can promote LINK_AUTH_UNAUTHORIZED to error.
expect(CODE_REGISTRY.LINK_AUTH_DEAD.defaultSeverity).toBe('error');
expect(CODE_REGISTRY.LINK_AUTH_DEAD_OR_UNAUTHORIZED.defaultSeverity).toBe('warning');
expect(CODE_REGISTRY.LINK_AUTH_FORBIDDEN.defaultSeverity).toBe('warning');
expect(CODE_REGISTRY.LINK_AUTH_UNAUTHORIZED.defaultSeverity).toBe('warning');
expect(CODE_REGISTRY.LINK_AUTH_UNVERIFIED.defaultSeverity).toBe('warning');
});
});

describe('IssueCodeSchema', () => {
it('IssueCodeSchema enumerates exactly the registry keys', () => {
expect(new Set(IssueCodeSchema.options)).toEqual(new Set(Object.keys(CODE_REGISTRY)));
Expand Down
63 changes: 50 additions & 13 deletions packages/resources/src/external-link-cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,27 @@ import { promises as fs } from 'node:fs';

import { safePath } from '@vibe-agent-toolkit/utils';

/**
* Cache schema version. Incremented when CacheEntry shape changes; entries
* written by an older code path that read as a different version are treated
* as cache misses (rather than misparsed) — forward-compat for slice 3's
* content-cache additions and any future entry-shape evolution.
*/
const CACHE_VERSION = 1;

/**
* Cache entry for external link validation results
*/
interface CacheEntry {
statusCode: number;
statusMessage: string;
timestamp: number;
/**
* Schema version. Optional for backwards-compat: legacy entries without
* `version` are treated as a cache miss, forcing a refetch. Reading on
* version mismatch produces a miss rather than a parse error.
*/
version?: number;
}

/**
Expand Down Expand Up @@ -61,7 +75,13 @@ export class ExternalLinkCache {
}

/**
* Load cache from disk
* Load cache from disk. Fail-soft: any IO error (ENOENT, EACCES, EROFS,
* corrupted JSON, …) degrades to an empty cache instead of throwing.
*
* Why no error propagation: `vat resources validate` should keep running
* when the cache file isn't reachable (read-only filesystem, permission
* mismatch, full disk) — a missing cache costs an extra network round-trip,
* an exception costs the whole run. Per #125 review.
*/
private async loadCache(): Promise<CacheData> {
if (this.cache !== null) {
Expand All @@ -75,28 +95,35 @@ export class ExternalLinkCache {
const data = await fs.readFile(this.cacheFile, 'utf-8');
this.cache = JSON.parse(data) as CacheData;
return this.cache;
} catch (error) {
// Handle missing file or corrupted JSON - start with empty cache
if ((error as NodeJS.ErrnoException).code === 'ENOENT' || error instanceof SyntaxError) {
this.cache = {};
return this.cache;
}
throw error;
} catch {
// All IO and parse errors degrade to an empty cache. Subsequent
// reads see the same empty cache (this.cache is set), so we don't
// re-spam mkdir/read on every lookup within the same run.
this.cache = {};
return this.cache;
}
}

/**
* Save cache to disk
* Save cache to disk. Fail-soft: any IO error becomes a no-op instead of
* throwing. Same rationale as `loadCache` — a non-persisted entry costs an
* extra fetch on the next run, an exception costs the whole current run.
*/
private async saveCache(): Promise<void> {
if (this.cache === null) {
return;
}

// eslint-disable-next-line security/detect-non-literal-fs-filename -- cacheDir is constructor parameter, controlled by caller
await fs.mkdir(this.cacheDir, { recursive: true });
// eslint-disable-next-line security/detect-non-literal-fs-filename -- cacheFile is derived from cacheDir
await fs.writeFile(this.cacheFile, JSON.stringify(this.cache, null, 2), 'utf-8');
try {
// eslint-disable-next-line security/detect-non-literal-fs-filename -- cacheDir is constructor parameter, controlled by caller
await fs.mkdir(this.cacheDir, { recursive: true });
// eslint-disable-next-line security/detect-non-literal-fs-filename -- cacheFile is derived from cacheDir
await fs.writeFile(this.cacheFile, JSON.stringify(this.cache, null, 2), 'utf-8');
} catch {
// No-op on IO failure. The in-memory cache (`this.cache`) is still
// authoritative for the current run; only the disk persistence is
// lost.
}
}

/**
Expand Down Expand Up @@ -156,6 +183,15 @@ export class ExternalLinkCache {
return null;
}

// Forward-compat: an entry written under a different (or missing)
// schema version is treated as a miss. Forces a refetch rather than
// silently misparsing a slice-3+ shape with slice-2 reader code.
if (entry.version !== CACHE_VERSION) {
delete cache[key];
await this.saveCache();
return null;
}

if (this.isExpired(entry)) {
delete cache[key];
await this.saveCache();
Expand All @@ -180,6 +216,7 @@ export class ExternalLinkCache {
statusCode,
statusMessage,
timestamp: Date.now(),
version: CACHE_VERSION,
};

await this.saveCache();
Expand Down
Loading
Loading