Instrument current webhook, approval, queue, lock, and run-start telemetry

[ncolesummers]

Summary

Instrument only the currently implemented operational paths that ADR 0012 names, without inventing future run execution, stage transition, retry, cancellation, or validation-runner behavior.

This issue now covers the telemetry that can be wired to existing control-plane paths: GitHub webhook outcomes, development-loop run start and trace correlation, approval wait/pending state, queued-run depth, lock contention, and central observability helper boundaries.

Run/step completion, duration, retry, cancellation, stage-transition outcomes, and deterministic validation execution telemetry are deferred to #73 under #14 so those metrics are implemented with the feature that creates the runtime behavior.

Deliverables

• Emit loopworks.webhook.outcome for existing GitHub webhook route branches: accepted, rejected, duplicate, invalid signature, and unexpected error.
• Preserve loopworks.run.started for development-loop creation and keep ADR 0012 trace correlation aligned across logs, spans, loop_runs.trace_id, run_steps.trace_id, and durable observability rows.
• Emit loopworks.approval.wait_time for current approval decision paths and expose loopworks.approval.pending from existing outstanding approval rows.
• Emit loopworks.queue.depth from existing queued loop_runs state.
• Emit loopworks.lock.contention from existing idempotency lock conflict paths.
• Keep all OTel metric/span creation and metric-name literals behind src/lib/observability/ helpers.

Acceptance Criteria

• Focused tests are written first and demonstrate the red state before production code changes.
• Emitted metric names, units, and attributes match ADR 0012 exactly for the metrics in this issue.
• Tests cover webhook outcome branches, approval decision paths, pending approval gauge behavior, queue-depth gauge behavior, and lock contention where currently implemented.
• trace_id propagation remains aligned with run and step persistence for development-loop run creation.
• No instrumentation call site bypasses the central observability helpers.
• No fake production call sites are added for run/step completion, retries, cancellations, stage outcomes, or validation execution metrics.
• Runtime changes include bun run build; broad observability changes finish with bun run validate.

Notes

• Keep attributes low-cardinality and avoid raw payloads, tokens, prompts, secrets, delivery IDs, run IDs, and step IDs in metric attributes.
• Lifecycle metrics deferred from this issue should be implemented with deterministic validation and run-transition behavior in Add lifecycle telemetry to deterministic validation and run transitions #73 under [Epic]: Deterministic validation hooks #14.
• Do not block future research/development agent issues on dashboards or alerting; this issue is about emitted telemetry for current paths.

Refs #21
Refs #14
Refs #73
Refs ADR 0012