Skip to content

Add a plain-JSON export format to goldset export (--format json) #5

Description

@AshwinUgale

Context

export.py and the CLI hardcode promptfoo as the only export target — cli.py uses choices=["promptfoo"], and the module docstring literally says "The first exporter targets promptfoo." Runners that aren't promptfoo currently have no first-class path.

Task

Add a plain-JSON exporter that emits the golden (confirmed) suite as the internal case model — id, input, assertions, expected_output, trust_tier, set_role. Wire it into --format json.

Acceptance criteria

  • Same golden gate as promptfoo: fails closed with NothingGoldenError when nothing is confirmed; --include-candidates marks the artifact non-golden.
  • Structural assertions (tool_called / final_state) are preserved, not dropped.
  • Test added in tests/test_export.py; README "Export targets" section updated.

Pointers

  • src/goldset/export.py
  • src/goldset/cli.py (the --format choices for the export subcommand)

Notes for contributors

Keep it in the dependency-light core (stdlib json). Follow the existing fail-closed stance in to_promptfoo — a golden export must never silently emit an empty or unconfirmed set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions