Skip to content

conformance: deconflict expected.error_code field semantics #285

@jithinraj

Description

@jithinraj

Problem

The expected.error_code field in workflow conformance fixtures (and other conformance suites) carries two distinct meanings:

  1. Validation error code (normative) - The code a validator MUST emit when rejecting invalid input (e.g., E_WORKFLOW_SELF_PARENT)
  2. Runtime error code (informational) - The code that might appear at runtime for edge cases that pass structural validation

This overload makes it ambiguous whether a test harness should assert on error_code for edge-case fixtures where expected.valid is true but an error_code is present for documentation.

Currently the JSON Schema enforces error_code is required when valid: false (via allOf/if/then), which is correct. But the field name itself suggests it's always a validation output, which could confuse cross-language implementors.

Proposal

Split into two distinct fields with a deprecation window:

Phase 1: Add new fields (next conformance version bump)

  • validation_error_code - Normative. Required when valid: false. Test harnesses MUST assert this.
  • meta.runtime_error_code - Informational. Optional. Documents what code might appear at runtime. Test harnesses MUST NOT assert this.

Phase 2: Deprecation (1 release cycle)

  • Accept both error_code (legacy) and validation_error_code (new) for 1 release
  • Fixture schema validates either field is present when valid: false

Phase 3: Remove legacy field

  • Remove error_code from fixture schema
  • All conformance suites use validation_error_code

Affected Fixture Suites

  • specs/conformance/fixtures/workflow/ (valid.json, invalid.json, edge-cases.json)
  • specs/conformance/fixtures/agent-identity/
  • specs/conformance/fixtures/attribution/
  • specs/conformance/fixtures/dispute/
  • specs/conformance/fixtures/bundle/
  • specs/conformance/fixtures/purpose/

Acceptance Criteria

  • New field names documented in fixture README
  • JSON Schema updated with oneOf for transition period
  • All test harnesses (TS, Go) updated to read new field
  • Legacy error_code removed after 1 release cycle
  • Cross-language test harnesses updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions