Skip to content

feat: add structured final output#1

Merged
dutifulbob merged 1 commit into
mainfrom
feat/final-schema-output
May 19, 2026
Merged

feat: add structured final output#1
dutifulbob merged 1 commit into
mainfrom
feat/final-schema-output

Conversation

@dutifulbob

Copy link
Copy Markdown
Member

Summary

Localagent can now ask Pi to finish with schema-validated JSON while still letting the agent use normal tools first.
This matters for local-model workflows like PR classification, where the model needs to inspect evidence before returning machine-readable metadata.
The implementation creates a temporary Pi extension with a schema-backed final_json tool, runs Pi with that extension, and prints only the captured JSON after the tool is called.

What Changed

The feature lives in localagent, not Pi.
Localagent generates the final-output extension per run and wires it into the existing Pi launch path.

  • Added --final-schema <path> and --schema <path>.
  • Added LOCALAGENT_FINAL_SCHEMA.
  • Generated a temporary final_json Pi tool from the supplied object schema.
  • Suppressed Pi stdout during schema runs and printed only the captured structured JSON.
  • Added allow-list handling so --tools bash becomes bash,final_json.
  • Documented the workflow and shipped docs in the npm package.

Testing

I tested both the local code paths and the real PR-classification use case with Gemma served through LM Studio.

  • npm run check
  • node dist/src/cli/main.js --status
  • node dist/src/cli/main.js --session-dir /tmp/localagent-smoke-sessions --final-schema /tmp/localagent-local-model-classifier.schema.json --tools bash -p "Read the PR contents before answering. Use gh to inspect https://github.com/openclaw/openclaw/pull/80568, including title, body, changed files, and enough diff/context to classify it. Classify whether this PR is about local models or not. Return the final answer only by calling final_json."
  • node dist/src/cli/main.js --session-dir /tmp/localagent-smoke-sessions-assert --final-schema /tmp/localagent-local-model-classifier.schema.json --tools bash -p "..." | tee /tmp/localagent-pr-80568-classification.json | jq -e '.is_local_model_related == true and (.relevance == "relevant" or .relevance == "highly_relevant") and (.evidence | length > 0)'

The smoke test classified PR 80568 as local-model related with relevance: "highly_relevant".

Risks

The feature depends on the model actually calling the final_json tool.
If it does not, localagent fails clearly instead of returning unstructured output.

  • The root schema must be an object schema.
  • --final-schema cannot be combined with --no-tools.
  • --mode rpc is rejected because localagent owns stdout handling for this mode.

@dutifulbob dutifulbob force-pushed the feat/final-schema-output branch from 7e6bfea to 3a7584f Compare May 19, 2026 05:05
@dutifulbob dutifulbob force-pushed the feat/final-schema-output branch from 3a7584f to 88adab6 Compare May 19, 2026 05:11
@dutifulbob

Copy link
Copy Markdown
Member Author

Final implementation report:

  • Added localagent-native structured final output via --final-schema / --schema.
  • Implemented it without modifying Pi by generating a temporary Pi extension with a schema-backed final_json tool.
  • Required Pi print mode for schema runs so localagent can suppress normal Pi stdout and print only captured JSON.
  • Added tool allow-list handling for final_json and rejected --no-tools / -nt.
  • Documented the feature and included docs in the npm package.

Validation:

  • npm run check passed.
  • node dist/src/cli/main.js --status passed against LM Studio at http://127.0.0.1:1234/v1 with gemma-local available.
  • Real smoke test against https://github.com/openclaw/openclaw/pull/80568 passed: localagent printed schema JSON only and classified it as local-model related.
  • Assertion smoke passed: jq -e '.is_local_model_related == true and (.relevance == "relevant" or .relevance == "highly_relevant") and (.evidence | length > 0)' returned true.
  • CODEX_HOME=/home/bob/.codex codex review --base main is clean after two reliability fixes.
  • GitHub Actions ci / test is green.

@dutifulbob dutifulbob merged commit a8c2d67 into main May 19, 2026
1 check passed
@dutifulbob dutifulbob deleted the feat/final-schema-output branch May 19, 2026 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant