Skip to content

fix(common): coerce null/empty/whitespace observation Result to UNAVAILABLE#193

Open
ottobolyos wants to merge 4 commits into
TrakHound:masterfrom
ottobolyos:fix/agent-empty-result-unavailable
Open

fix(common): coerce null/empty/whitespace observation Result to UNAVAILABLE#193
ottobolyos wants to merge 4 commits into
TrakHound:masterfrom
ottobolyos:fix/agent-empty-result-unavailable

Conversation

@ottobolyos

@ottobolyos ottobolyos commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

MTConnectAgent.AddObservation accepted observations whose Result value was null, the empty string "", or whitespace-only and forwarded that value verbatim onto every wire transport (HTTP /current, /sample, MQTT JSON-cppagent, SHDR). MTConnect Part 1 §Observation Information Model — Representation — Observation Values mandates "UNAVAILABLE" as the sole valid representation of a missing or undetermined observation value:

"If an Agent cannot determine a Valid Data Value for a DataItem, the value returned for the Result for the Data Entity MUST be reported as UNAVAILABLE."

The empty string is, by definition, not a Valid Data Value (it parses as no value at all under every typed DataItem schema). This PR adds an unconditional coerce step in the canonical AddObservation(string deviceKey, IObservationInput, …) overload — null / empty / whitespace Result is rewritten to Observation.Unavailable before validation, so:

  • Ignore / Warning / Remove: publishes UNAVAILABLE rather than the empty value, satisfying the spec mandate on the wire.
  • Strict: coerces to UNAVAILABLE, the observation lands rather than being silently dropped.

The coerce skips CONDITION observations (their Level enum cannot carry the empty pathology). DATA_SET / TABLE / TIME_SERIES Representation pre-fill of Count / SampleCount runs after the coerce — semantics for those representations are unchanged.

No opt-out flag is exposed: the spec mandate is MUST, so a way to opt out of compliance would itself be non-conformant.

Behaviour change

Input Pre-fix wire payload Post-fix wire payload
Result = null "" (empty) "UNAVAILABLE"
Result = "" "" "UNAVAILABLE"
Result = " " (whitespace) " " "UNAVAILABLE"
Result = "AVAILABLE" (concrete) "AVAILABLE" "AVAILABLE" (unchanged)
Result = "" under Strict observation silently dropped lands with "UNAVAILABLE"

Tests

Unit (NUnit, tests/MTConnect.NET-Common-Tests/Agents/AddObservationEmptyResultCoerceTests.cs, 10 cases):

  • empty-string Result under every non-Strict InputValidationLevel
  • the null / "" / whitespace family under Warning
  • empty Result under Strict (lands, not silently dropped)
  • concrete-value preservation (negative — the coerce does not substitute for a Valid Data Value)

Wire-level E2E (xUnit, [Trait("Category", "E2E")]):

tests/MTConnect.NET-Integration-Tests/Workflows/AddObservationEmptyResultUnavailableWorkflowTests.cs — HTTP /current:

  • null / empty / spaces / tab / newline / CRLF Result → AVAILABILITY element body is UNAVAILABLE (6-case [Theory])
  • concrete value → AVAILABILITY element body is verbatim
  • concrete → empty sequence → AVAILABILITY element reflects the latest (UNAVAILABLE)
  • empty Result under Strict validation → AVAILABILITY element body is UNAVAILABLE (lands, not silently dropped; spins its own agent + server harness)

tests/MTConnect.NET-Integration-Tests/Workflows/AddObservationEmptyResultUnavailableSampleStreamWorkflowTests.cs — HTTP /sample stream:

  • null / empty / spaces / tab / newline / CRLF Result → latest AVAILABILITY sample is UNAVAILABLE (6-case [Theory])
  • concrete value → latest AVAILABILITY sample is verbatim
  • concrete → empty sequence → two distinct samples in stream order (concrete first, then UNAVAILABLE); proves the coerce does not silently drop the second observation

All wire surfaces fed by the agent's buffer read the same coerced payload — every transport (HTTP, MQTT relay, SHDR adapter output) inherits the fix because the coerce sits above the buffer.

Files touched

  • libraries/MTConnect.NET-Common/Agents/MTConnectAgent.cs — invoke the coerce in AddObservation; add IsEmptyResult + CoerceEmptyResultToUnavailable helpers.
  • tests/MTConnect.NET-Common-Tests/Agents/AddObservationEmptyResultCoerceTests.cs — new unit fixture.
  • tests/MTConnect.NET-Integration-Tests/Workflows/AddObservationEmptyResultUnavailableWorkflowTests.cs — new wire-level E2E fixture against HTTP /current.
  • tests/MTConnect.NET-Integration-Tests/Workflows/AddObservationEmptyResultUnavailableSampleStreamWorkflowTests.cs — new wire-level E2E fixture against HTTP /sample stream.
  • .github/workflows/dotnet.yml — drop matrix expressions from the two matrix jobs' name: fields so skipped-state status checks render cleanly (build-and-test / route-check-e2e instead of build-and-test-${{ matrix.os }}).

@ottobolyos ottobolyos force-pushed the fix/agent-empty-result-unavailable branch 3 times, most recently from 66db93a to 0a58cfb Compare June 11, 2026 16:10
@ottobolyos ottobolyos marked this pull request as ready for review June 11, 2026 17:35
@ottobolyos ottobolyos marked this pull request as draft June 11, 2026 17:52
@ottobolyos ottobolyos force-pushed the fix/agent-empty-result-unavailable branch from a2c41aa to a90e23b Compare June 11, 2026 17:57
@ottobolyos ottobolyos marked this pull request as ready for review June 11, 2026 18:03
@ottobolyos ottobolyos marked this pull request as draft June 11, 2026 18:10
@ottobolyos ottobolyos force-pushed the fix/agent-empty-result-unavailable branch from a90e23b to 9d1e172 Compare June 11, 2026 18:17
@ottobolyos ottobolyos marked this pull request as ready for review June 11, 2026 18:21
@ottobolyos

Copy link
Copy Markdown
Contributor Author

@PatrickRitchie, this PR fixes a Part 1 spec violation that touches every wire output: MTConnectAgent.AddObservation was forwarding null / empty / whitespace Result values verbatim, where Part 1 §Observation Information Model — Representation — Observation Values mandates UNAVAILABLE as the sole valid representation of a missing value. The fix is small and self-contained—4 files across 4 commits, with 10 unit + 17 wire-level E2E cases pinning the contract—but the bug class is broad enough (every HTTP envelope, MQTT relay, and SHDR adapter output benefits from the coerce at the buffer entry) that a review at your convenience would be appreciated.

@ottobolyos

Copy link
Copy Markdown
Contributor Author

@PatrickRitchie — heads-up: while validating PR #189's dime-connector build I caught an unrelated multi-TFM Release-pack regression introduced by an earlier warnings sweep. The minimal fix lands in PR #194 (just opened as draft); the broader warnings cleanup plus a Release-pack CI gate ships in PR #195.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Reviewing

Development

Successfully merging this pull request may close these issues.

2 participants