OpenTAP Embedded Python Host Bug Report

[alexwang-keysight]

OpenTAP Embedded Python Host Bug Report

Summary

This package contains a reproducible failure in the embedded OpenTAP Python host when handling Keysight PCIe Digital Test App .NET remoting callbacks.

• The known-good standalone flow in pcie_event_demo_style.py succeeds end to end.
• The equivalent in-process OpenTAP-hosted flow can connect and receive RunEndedEvent, but it does not reliably deliver the full lifecycle or result callbacks.
• When message callbacks are enabled during NewProject True, the embedded host can also fail earlier with a MessageEventArgs remoting marshal/type-conversion error.

Files Included For R&D

• pcie_event_demo_style.py: known-good standalone event-handler flow.
• OpenTapInProcessEventRepro.py: stripped-down OpenTAP in-process repro step for the embedded host bug.
• pcie_instrument.py: current temporary workaround that shells out to the standalone script.

Environment

• Workspace package: EventHandlerComplianceApp
• OS: Windows
• OpenTAP version used for repro: 9.28
• OpenTAP version note: the compliance app did not run correctly on 9.29, so all meaningful repro work was done on 9.28
• Python venv used for standalone success: .venv\Scripts\python.exe
• Python version used for repro: 3.12.10
• pythonnet: 3.0.5
• Keysight remote assembly used in repro: keysight_dlls\Keysight.DigitalTestApps.Framework.Remote.dll
• Known-good target app IP during repro: 141.121.234.173
• Known-good test ID during repro: 6001

Known-Good Standalone Repro

Run:

.\.venv\Scripts\python.exe .\pcie_event_demo_style.py --ip 141.121.234.173 --test-id 6001 --dll-dir .\keysight_dlls --output-dir .\results

Observed behavior:

• SimpleMessageEvent received for NewProjectCreated
• TestStartingEvent received for 6001
• SimpleMessageEvent received for Running Test Step 1
• TestEndedEvent received with passed=True
• Passed: True
• GetResults populated
• CSV result file saved

OpenTAP In-Process Repro

Use the OpenTAP step from OpenTapInProcessEventRepro.py.

Default settings:

• IP Address: 141.121.234.173
• DLL Directory: package keysight_dlls
• Test ID: 6001
• Run Timeout Seconds: 30
• Enable Messages During Reset: False

Clean Failure Mode

With Enable Messages During Reset = False, the repro step:

1. configures remoting
2. connects to the app
3. creates AteEventSink
4. subscribes TestStartingEvent, TestEndedEvent, RunEndedEvent, SimpleMessageEvent, DataInputMessageEvent
5. suppresses messages during NewProject True
6. re-enables messages for the actual test run
7. selects test 6001
8. starts remote_app.Run() on a background thread
9. waits for RunEndedEvent

Observed behavior in the embedded host:

• RunEndedEvent arrives
• TestStartingEvent does not arrive
• TestEndedEvent does not arrive
• SimpleMessageEvent does not arrive for the actual test run
• GetResults is empty
• Passed remains False

This is the primary repro because it avoids the earlier marshal exception noise and isolates the missing lifecycle/result callback delivery.

Alternate Failure Mode

With Enable Messages During Reset = True, the repro can fail earlier during NewProject True.

Observed error pattern:

• NewProjectCreated message callback reaches the embedded host
• remoting fails converting a Keysight.DigitalTestApps.Framework.Remote.MessageEventArgs
• error includes text similar to:

Client's AppMessageEvent handler Void OnMessage(System.Object, Keysight.DigitalTestApps.Framework.Remote.MessageEventArgs)
threw exception: System.Runtime.Remoting.RemotingException:
The argument type 'Message=... cannot be converted into parameter type
'Keysight.DigitalTestApps.Framework.Remote.MessageEventArgs'

Expected Behavior

The embedded OpenTAP Python host should behave like the standalone script for the same remoting flow:

• receive SimpleMessageEvent callbacks
• receive TestStartingEvent
• receive TestEndedEvent
• deliver populated GetResults
• report Passed=True

Actual Behavior

The embedded host shows one of two failure signatures:

1. NewProjectCreated message callback marshal failure when prompt/message callbacks are active during reset.
2. If reset-time messages are suppressed to avoid that marshal failure, only RunEndedEvent survives; full lifecycle and result callbacks are lost.

Isolation Work Performed

The following variations were tried and did not fix the in-process bug:

• remote_app.Run() versus ExecuteArsl("Run")
• remote_app.SelectedTests = [test_id] versus ExecuteArsl("SelectedTests ...")
• CreateAteEventSink(...) versus CreateAteEventSinkCustom(...)
• creating the sink before NewProject versus after NewProject
• RedirectMessagesToClient = True versus False
• SuppressMessages = True versus False
• short settle delays after NewProject and after selection
• background Run() with RunEndedEvent wait versus synchronous calls

Key conclusion from isolation:

• the callback channel is not completely dead, because RunEndedEvent can arrive
• but richer remoting callback/event paths are broken or dropped in the embedded OpenTAP Python host

Temporary Workaround

pcie_instrument.py currently launches pcie_event_demo_style.py out of process using the workspace venv interpreter. This succeeds and preserves the event-driven behavior, but it is only a containment workaround and not the desired long-term solution.

Suspected Fault Boundary

Most likely OpenTAP-side problem areas:

• embedded Python host marshaling of .NET remoting callback delegate arguments
• callback type conversion for MessageEventArgs and related richer event payloads
• hosting-thread or apartment behavior that permits simple callback completion (RunEndedEvent) but breaks richer callback delivery

Requested OpenTAP Investigation

Please compare the embedded host behavior against the standalone Python process for the same pythonnet + Keysight remoting flow, especially:

• delegate binding and marshaling for remoted event args
• message callback path for MessageEventArgs
• why RunEndedEvent is delivered while TestStartingEvent, TestEndedEvent, SimpleMessageEvent, and GetResults are not

EventHandlerComplianceApp-Repro.zip