Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ See [docs/structured-output.md](docs/structured-output.md).
Example:

```bash
localagent --final-schema ./schema.json -p "inspect https://github.com/openclaw/openclaw/pull/80568 and classify it"
localagent --final-schema ./examples/schemas/binary-classifier.schema.json -p "classify whether this issue is release-blocking: <text>"
```

## Options
Expand Down
35 changes: 20 additions & 15 deletions docs/structured-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Localagent supports structured final answers for workflows that need machine-readable output from a local model.

Example: inspect a GitHub PR with Pi tools, then classify whether the PR is related to local models.
Example: inspect a document, issue, or pull request with Pi tools, then classify whether it matches a criterion.

## How It Works

Expand All @@ -22,19 +22,19 @@ This keeps the agent loop freeform while making the final answer structured.
## CLI

```bash
localagent --final-schema ./schema.json -p "inspect https://github.com/openclaw/openclaw/pull/80568 and classify it"
localagent --final-schema ./examples/schemas/binary-classifier.schema.json -p "classify whether this issue is release-blocking: <text>"
```

Alias:

```bash
localagent --schema ./schema.json -p "inspect the PR and classify it"
localagent --schema ./examples/schemas/binary-classifier.schema.json -p "classify whether this issue is release-blocking: <text>"
```

You can also set a default schema with:

```bash
export LOCALAGENT_FINAL_SCHEMA=./schema.json
export LOCALAGENT_FINAL_SCHEMA=./examples/schemas/binary-classifier.schema.json
```

## Tool Allow Lists
Expand All @@ -44,7 +44,7 @@ If you pass Pi a tool allow list with `--tools` or `-t`, localagent automaticall
For example, this:

```bash
localagent --final-schema ./schema.json --tools bash -p "inspect the PR"
localagent --final-schema ./examples/schemas/binary-classifier.schema.json --tools bash -p "inspect the repository and classify the issue"
```

is passed to Pi as if the tool list were:
Expand All @@ -63,15 +63,16 @@ bash,final_json
{
"type": "object",
"additionalProperties": false,
"required": ["is_local_model_related", "interest", "confidence", "description", "caveats"],
"required": ["is_match", "label", "confidence", "summary", "reasons", "caveats"],
"properties": {
"is_local_model_related": { "type": "boolean" },
"interest": {
"is_match": { "type": "boolean" },
"label": {
"type": "string",
"enum": ["i0", "i1", "i2", "i3", "i4"]
"enum": ["match", "partial_match", "no_match", "unclear"]
},
"confidence": { "type": "number", "minimum": 0, "maximum": 1 },
"description": { "type": "string" },
"summary": { "type": "string" },
"reasons": { "type": "array", "items": { "type": "string" } },
"caveats": { "type": "array", "items": { "type": "string" } }
}
}
Expand All @@ -81,11 +82,15 @@ bash,final_json

```json
{
"is_local_model_related": true,
"interest": "i0",
"confidence": 0.9,
"description": "This PR fixes LM Studio auth resolution, which affects using local model servers from OpenClaw.",
"caveats": ["This is provider integration work, not model inference behavior."]
"is_match": true,
"label": "match",
"confidence": 0.86,
"summary": "The issue describes a startup crash that blocks the release candidate path.",
"reasons": [
"The issue affects the release branch.",
"The failure happens before the main UI can load."
],
"caveats": ["The impact on patch releases was not checked."]
}
```

Expand Down
41 changes: 41 additions & 0 deletions examples/schemas/binary-classifier.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Binary Classifier Result",
"type": "object",
"additionalProperties": false,
"required": ["is_match", "label", "confidence", "summary", "reasons", "caveats"],
"properties": {
"is_match": {
"type": "boolean",
"description": "Whether the item matches the criterion from the prompt."
},
"label": {
"type": "string",
"enum": ["match", "partial_match", "no_match", "unclear"],
"description": "A compact classification label."
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"summary": {
"type": "string",
"description": "One or two sentences explaining the classification."
},
"reasons": {
"type": "array",
"items": {
"type": "string"
},
"description": "Short evidence-backed reasons for the classification."
},
"caveats": {
"type": "array",
"items": {
"type": "string"
},
"description": "Important uncertainty, missing context, or limitations."
}
}
}
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
"files": [
"dist/src/**",
"docs/**",
"examples/**",
"README.md",
"LICENSE"
],
Expand Down
28 changes: 28 additions & 0 deletions tests/structured-output.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,34 @@ import { createLaunchPlan } from "../src/pi/launch.js";
import { createFinalSchemaRuntime, readFinalSchemaOutput } from "../src/structured/final-schema.js";

describe("structured output", () => {
it("ships a context-agnostic example schema", async () => {
const raw = await readFile(
path.join(process.cwd(), "examples", "schemas", "binary-classifier.schema.json"),
"utf8"
);
const parsed = JSON.parse(raw) as {
type?: string;
required?: string[];
properties?: { label?: { enum?: string[] } };
};

expect(parsed.type).toBe("object");
expect(parsed.required).toEqual([
"is_match",
"label",
"confidence",
"summary",
"reasons",
"caveats"
]);
expect(parsed.properties?.label?.enum).toEqual([
"match",
"partial_match",
"no_match",
"unclear"
]);
});

it("creates a final_json extension from a schema", async () => {
const stateDir = await mkdtemp(path.join(os.tmpdir(), "localagent-structured-"));
try {
Expand Down