RFC: ergonomic typed PSRP value layer (serde-style derive) — findings from PowerShell reference study

[irvingoujAtDevolution]

Summary

Working with PSRP values (PsValue / ComplexObject / PsObjectWithType) requires large amounts of hand-written conversion code at every layer. After studying the PowerShell reference implementation (shallow clone of PowerShell/PowerShell, src/System.Management.Automation/engine/serialization.cs and engine/remoting/), we confirmed our value model is architecturally correct — what's missing is an ergonomic access layer and a serde-style typed schema on top. This issue records the findings and the proposed target design.

The pain, quantified

Location	Cost today
ironposh-psrp/src/messages/	19 message types, ~3,800 LOC; each hand-writes the trio PsObjectWithType + From<T> for ComplexObject + TryFrom<ComplexObject>
ConnectRunspacePool (worst-case example)	2 fields of payload → ~85 lines of impl
ironposh-client-core/src/host/{params,returns}.rs	~800 LOC, 84 raw PsValue::/extended_properties manipulations
Downstream (ironposh-web/src/hostcall.rs, ironposh-client-tokio/src/hostcall.rs)	re-convert the same trees again for UI

Root causes (all in the access layer, not the model):

• No typed accessors: extracting one i32 = extended_properties.get(name) + hand-rolled missing-property error + two-layer enum match + hand-rolled type-mismatch error. Every message reinvents this closure.
• No construction helpers: inserting one property = 6 lines, property name written 3× (map key + PsProperty.name + attr).
• PsProperty { name, value } duplicates the map key it's stored under.
• PsValue::as_string_array() is a stub silently returning Some(vec![]) (TODO) — wrong-answer generator.
• Stringly-typed Guid(String) / DateTime(String) / Version(String); GUID uppercase convention enforced only in one From<Uuid> impl.
• to_ps_object(&self) forces a clone() per serialize.

Findings from the reference implementation

The thesis "CLIXML is painful for statically-typed clients because it's a reflection dump of a live .NET object graph" is true, with a twist that works in our favor:

1. Even .NET clients get property bags, not live objects. On deserialization PowerShell prefixes every type name with "Deserialized." (serialization.cs:816), sets IsDeserialized = true, and rebuilds objects as property bags (MshObject.cs:2286-2385). No live-type rehydration exists, by design. Our dynamic PsValue tree is morally identical to what the reference itself builds — Rust is not at a disadvantage on the receive side.
2. Remoting serializes at depth = 1 (fragmentor.cs:970); nested objects degrade to ToString. Wire objects are much shallower than the format's ceiling.
3. Fixed known-types table (~30 entries, serialization.cs:5167-5376) maps .NET type ↔ XML tag ↔ serializer fn. Static and version-stable — worth mirroring verbatim as a const table to kill tag-drift bugs.
4. Adapted vs extended properties (<Props> vs <MS>) matter to .NET's member-resolution chain, not to clients — the reference itself coalesces deserialized properties into one bag (AdaptedMembers). Accessors can search both.
5. RefId/TNRef dedup is codec-level concern; values need never contain references.
6. ToString is display metadata, never load-bearing for rehydration.
7. SecureString is session-key crypto injected via context (serialization.cs:1126-1191) — matches our existing DeserializationContext approach.

Validation of the disconnect/reconnect/reattach work (feat/ci-tls-jea-reconnect)

• ✅ Empty-body DisconnectResponse (Action header only) is canonical — reference client never parses the body (WSManTransportManager.cs:2018-2095). Matches our fix.
• ✅ Minimal <rsp:Disconnect/>: BufferMode/IdleTimeOut genuinely optional, server defaults apply.
• ✅ No command re-enumeration after Connect — reference doesn't either; pipelines flow back over the normal receive stream.
• ⚠️ Gap: CONNECT_RUNSPACEPOOL min/max runspaces. The reference sends the original pool's stored values (RemotingProtocol2.cs:38-39), never defaults. We hardcode 1/1; our live reattach e2e passes only because our pools are always created 1/1. A reattach to a differently-sized pool would send wrong values. The reference client knows the originals via server-side session enumeration (Get-PSSession -ComputerName), which we don't implement. Interim options: --connect-min/max-runspaces flags or documenting the 1/1 assumption; long-term, session enumeration solves it properly.

Proposed target design ("serde for CLIXML")

Goal: core/UI layers only touch typed Rust structs; the dynamic tree becomes internal interchange (the role serde_json::Value plays). Not an actual serde::Serializer backend — CLIXML breaks serde's data model (dual property bags, type-name metadata, stateful RefId/SecureString codec); we want serde's architecture with domain-specific traits and derive.

Six layers:

L0  Primitives + const KNOWN_TYPES table      Guid(Uuid), DateTime(Timestamp), SecureString(Sealed); tag dispatch from one table
L1  Dynamic value (the one breaking change)   Object { type_names, to_string, body, props: one ordered map w/ Adapted|Extended tag }
                                              + typed accessors: obj.req::<i32>("Name")?, obj.opt(..), Object::standard().extended(..)
L2  Codec                                     to_clixml/from_clixml(ctx) — RefId/TNRef, depth, SecureString crypto live ONLY here
L3  #[derive(PsSerialize, PsDeserialize)]     #[ps(message = ..., name = "...", adapted, with = "path")]; emits L1 accessor calls;
                                              hand-written impls for gnarly types use the same API; Value field = dynamic escape hatch
L4  Typed protocol stream                     Defragmenter yields enum PsrpMessage { SessionCapability(..), PipelineOutput(Value), .. }
                                              — pool/pipeline state machines stop knowing CLIXML exists
L5  Consumers                                 host call dispatch becomes `match call { HostCall::SetCursorPosition(p) => ui.set_cursor(p.x, p.y), .. }`

Before/after for ConnectRunspacePool with L1 alone (no macro): ~85 lines → ~15. With L3, a message is a 15-line annotated struct.

Expected deltas: messages/ ~3,800 → ~900 LOC; host/params+returns ~800 → ~300; similar shrink in web/tokio hostcall glue; every future MS-PSRP message becomes struct + derive + tests. Roughly 30% smaller system with strictly better type safety and error messages.

Execution strategy: evolve, don't rewrite

The codec (L2) and the e2e-validated protocol behavior are the expensive/risky parts and already work against real servers. Incremental path, each step covered by existing roundtrip tests + the no-network connector harness + live e2e:

1. L1 accessors/builder (additive, no breakage) + fix the as_string_array stub — biggest ergonomic win per line.
2. L0 known-types const table — drop-in.
3. L3 derive in ironposh-macros (syn/quote, trybuild tests; good compile errors are where the effort goes) — migrate messages one per commit.
4. Property-model swap (one ordered map; drop PsProperty) — the only breaking change; do once L1 accessors hide the representation.
5. L4 typed PsrpMessage stream, then L5 host-call migration.
6. Separately: address the CONNECT_RUNSPACEPOOL min/max gap (see above).

References

• Reference clone: PowerShell/PowerShell, key files: engine/serialization.cs (known-types table :5167, Deserialized prefix :816, depth handling :992-1032, SecureString :1126), engine/MshObject.cs (:125-203, :2286-2385), engine/remoting/common/fragmentor.cs (:970), engine/remoting/common/WireDataFormat/EncodeAndDecode.cs (:313-331, :497-532), engine/remoting/fanin/WSManTransportManager.cs (:822-866, :906-991), engine/remoting/client/RemoteRunspacePoolInternal.cs (:727-742)
• MS-PSRP §2.2.5 (serialization), §2.2.2.14 (CONNECT_RUNSPACEPOOL); MS-WSMV §3.1.4.13-15
• Current pain exemplar: crates/ironposh-psrp/src/messages/connect_runspace_pool.rs