Intent observability: implement versioned measurement profiles and assessment semantics

[dinpd]

Parent: #45
Depends on: #46
Builds on: #19, #20, #21
Normative target: #58

Outcome

Implement the RFC-aligned measurement and assessment layer that sits above durable AgentPass evidence, without making traces or authorization decisions the source of outcome truth.

The existing intent profile continues to define the job's desired outcome and constraints. A distinct immutable measurement profile defines how a subject is assessed, which evaluator and evidence rules apply, and which results are comparable.

Current gap

agentpass.intent-evaluation.v1 is a useful deterministic job-level evaluation receipt, but it does not identify a measurement profile, evaluator version, subject kind, assessment revision, explicit evidence-quality state, or complete compatibility key. Its goal_attainment and partial semantics can also blend known negative outcomes with indeterminate evidence.

Scope

• immutable measurement-profile schema separate from the intent profile
• subject kinds and references for action, logical_step, run, and declared cohorts
• evaluator identity, version, method, and evidence basis
• explicit dimension states for execution disposition, intent fulfillment, constraint compliance, execution discipline, and evidence quality
• evidence-quality states including sufficient, incomplete, untrusted, stale, conflicting, and indeterminate
• confidence interpretation as ordinal, calibrated probability, or explicitly defined profile score
• qualified-success derivation that cannot be produced by transport or execution success alone
• comparison/cohort key including measurement profile, intent profile, evaluator, subject kind, finalization semantics, and interoperability version
• a compatibility adapter from the current V1 evaluation receipt rather than in-place mutation of V1 schemas

Acceptance criteria

• A measurement profile declares eligibility, exclusions, required evidence/freshness, evaluator version, state mappings, numerator/denominator definitions, finalization behavior, compatibility, and privacy expectations.
• Final assessments identify subject_ref, measurement_profile_ref, evaluator_ref, lifecycle, evidence snapshot/digest, and assessment revision.
• achieved, not_achieved, and indeterminate remain distinguishable in both subject records and aggregates.
• One passed predicate plus unknown remaining predicates is not silently classified as demonstrated partial achievement.
• Goal attainment reports achieved, failed, and indeterminate weight or otherwise avoids converting unknown weight into failure.
• Evidence source presence alone—especially an empty source—cannot satisfy an evidence requirement unless the selected profile explicitly defines that rule.
• Uncalibrated numeric confidence is not represented as probability, and confidence interpretation is portable and versioned.
• Changing evaluator behavior or measurement semantics creates a new compatible cohort rather than blending results under the same intent-profile key.
• Existing agentpass.intent-evaluation.v1 consumers have a documented V1 projection or migration path.
• No universal cross-profile score is introduced.

Validation

• all-positive, known-negative, partially achieved, and indeterminate outcome fixtures
• mixed pass/fail/unknown predicate fixtures
• empty, missing, stale, untrusted, and conflicting evidence fixtures
• evaluator-version and profile-compatibility tests
• V1-to-new-assessment projection tests
• sampled-out or absent trace tests proving durable assessment independence

Non-goals

• independently adjudicating whether an authorization decision was correct; tracked separately
• implementing OTLP transport
• standardizing prompts, chain of thought, or a universal evaluator