feat: capture system prompt, tools and reasoning in OTel; fail loud on setup

[mohammadp1001]

Why

OpenTelemetry was already wired up and exporting to Cloud Trace and Cloud Logging, but three things about a run's LLM behaviour never reached it: the system prompt, the tool definitions the model was given, and its reasoning. On top of that, a setup failure was silent, so a run could execute for hours and leave no trace at all.

What changed

1. System prompt and tool definitions are now recorded

Only ADK's experimental GenAI semantic conventions emit gen_ai.system_instructions and gen_ai.tool_definitions. The stable path records what the model said but never what it was told, or what tools it could have reached for - so a trace could not answer "why did it have that option".

setup_otel() opts in via OTEL_SEMCONV_STABILITY_OPT_IN. The variable is a CSV shared with other instrumentations, so the opt-in is appended rather than assigned, and an explicit stable from an operator is left alone.

2. Reasoning is now recorded, because it is now produced

Gemini 2.5 reasons whether or not you ask, but only returns the thought summary when include_thoughts is set. Nothing was being dropped - there was nothing to drop. Both agents now pass thinking_config (alphoryn/agents/thinking.py).

This is the part that would have broken production. Thought summaries arrive as extra text parts before the answer, flagged thought=True:

• main_agent.py took the first non-empty text part
• feedback_agent.py took parts[0].text

Both would have started parsing a thought summary as the JSON answer. That fails every trading session, and for feedback it burns all three retries and files a perfectly good evaluation as EVALUATION_FAILED - the exact failure mode FR-016a exists to prevent.

Both now route through is_thought_part(). It tests is True rather than truthiness on purpose: getattr(MagicMock(), "thought", False) returns a truthy child, so a truthiness test would report every part as a thought and silently swallow the model's answer under test. That was not hypothetical - it happened while writing this, and 34 tests caught it.

3. Setup failure is now loud (exit code 4)

setup_otel() caught every exception, logged a warning, and let the run continue with no exporters. Three ways a whole run went untraced:

Path	Old behaviour
google.auth.default() raises	Warning; run continues, zero spans
Credentials resolve with no project ID	get_gcp_exporters() returns empty hooks and raises nothing, so not even a warning fired
Crash or hard kill	Whatever was still buffered in the BatchSpanProcessor was lost

Now: setup_otel() raises TelemetrySetupError, verifies a real SDK TracerProvider was actually installed (the global default is a ProxyTracerProvider, which is what the silent path leaves behind), returns the resolved project ID, and registers an atexit flush.

The CLI reports the error and exits 4, and on success prints Telemetry -> GCP project '<id>'. Telemetry landing in the wrong project looks identical to telemetry landing nowhere, and this repo's secrets live in alphoryn while gcloud's default on at least one dev box is wortcast.

On constitution Principle IV: this does not contradict it. Principle IV governs per-event emission at run time, and TelemetryLogger.emit still falls back to stderr and never blocks. Setup is a preflight check - the same class of thing as config validation, which already exits 1.

Regression tests that fail on main

• test_decide_skips_a_thought_part_and_parses_the_answer
• test_evaluate_skips_a_thought_part_and_parses_the_answer
• test_evaluate_treats_a_thought_only_response_as_no_answer
• test_setup_otel_raises_when_credentials_carry_no_project_id
• test_setup_otel_raises_when_no_tracer_provider_was_installed
• test_run_exits_4_when_telemetry_cannot_be_set_up

Verification

• pytest: 635 passed, 100% coverage
• ruff check alphoryn/ tests/: clean

Not covered by this PR

No live run has exercised any of it - every test stubs the LLM. The real check is one alphoryn run during market hours, confirming gen_ai.system_instructions and thought parts appear in Cloud Trace for projects/alphoryn. That remains the standing top item on HANDOFF.md.

Docs

• TELEMETRY_ACCESS.md: new section on what is captured, the thought-part trap, and the exit-4 behaviour
• specs/001-etf-paper-trading-agent/contracts/cli.md: exit code 4 added to the table