Skip to content

starknet_transaction_prover: bump prover_panics_total on panic hook fire#14170

Open
avi-starkware wants to merge 1 commit into
avi/prover-v3/http-metricsfrom
avi/prover-v3/panic-counter
Open

starknet_transaction_prover: bump prover_panics_total on panic hook fire#14170
avi-starkware wants to merge 1 commit into
avi/prover-v3/http-metricsfrom
avi/prover-v3/panic-counter

Conversation

@avi-starkware

Copy link
Copy Markdown
Collaborator

Bumps the new prover_panics_total counter from the panic hook before
constructing the backtrace, so dashboards can alert on panic rate without
log search. The pre-registered zero observation in install_exporter
keeps the series visible at scrape time even before the first panic.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

@cursor

cursor Bot commented May 24, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Observability-only changes to the panic hook and metrics registration; no change to proving, RPC, or request handling behavior.

Overview
Adds prover_panics_total Prometheus observability so panic rate can be alerted on without log search.

The global panic hook now increments that counter first (before backtrace capture and structured tracing output), so a recursive panic during hook work still records the original event. install_exporter pre-registers the series at zero like other counters so it appears on /metrics before the first panic.

A unit test asserts install_panic_hook bumps the counter by one when a panic is caught.

Reviewed by Cursor Bugbot for commit b7a8e8e. Bugbot is set up for automated code reviews on this repo. Configure here.

@reviewable-StarkWare

Copy link
Copy Markdown

This change is Reviewable

@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from cbd1def to e503ebd Compare May 24, 2026 16:48
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 51ebd37 to 6c02d2b Compare May 24, 2026 16:48
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from e503ebd to db503b7 Compare May 26, 2026 08:43
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 2af08ad to 47068b1 Compare May 26, 2026 12:16
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from db503b7 to 1da27e9 Compare May 26, 2026 12:16
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 47068b1 to eea5655 Compare May 26, 2026 12:17
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from 1da27e9 to ac98d86 Compare May 26, 2026 12:17
Comment thread crates/starknet_transaction_prover/src/server/panic_test.rs
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from ac98d86 to e4bbbdc Compare May 26, 2026 12:58
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from eea5655 to df42f2f Compare May 26, 2026 12:58
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from df42f2f to 83d004c Compare May 26, 2026 16:14
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from e4bbbdc to 06bb59e Compare May 26, 2026 16:14
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 83d004c to 9122417 Compare May 26, 2026 16:47
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch 2 times, most recently from 0b2c8cc to 4b1caba Compare May 26, 2026 16:59
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 9122417 to 23731d6 Compare May 26, 2026 16:59
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 23731d6 to 145ff10 Compare May 27, 2026 10:01
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch 2 times, most recently from 05ed9b4 to 72918b7 Compare May 27, 2026 10:35
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch 2 times, most recently from 130f7f6 to 7fa5a6d Compare May 27, 2026 12:55
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from 72918b7 to 74f4f46 Compare May 27, 2026 12:56
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 7fa5a6d to fe8282d Compare May 27, 2026 13:11
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from 74f4f46 to 728f22c Compare May 27, 2026 13:11
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from fe8282d to d9c346a Compare May 27, 2026 14:04
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from 728f22c to 966f499 Compare May 27, 2026 14:04
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from d9c346a to d3505a7 Compare May 27, 2026 14:20
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch 2 times, most recently from 646fb1e to a77477b Compare May 31, 2026 10:23
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch 2 times, most recently from 1b96cdb to f13a24c Compare June 1, 2026 08:17
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from a77477b to 8017e9e Compare June 1, 2026 08:17
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from f13a24c to 2eaa611 Compare June 1, 2026 11:18
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from 8017e9e to 23ed570 Compare June 1, 2026 11:18
Bumps the new `prover_panics_total` counter from the panic hook before
constructing the backtrace, so dashboards can alert on panic rate without
log search. The pre-registered zero observation in `install_exporter`
keeps the series visible at scrape time even before the first panic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-counter branch from 23ed570 to b7a8e8e Compare June 7, 2026 10:11
@avi-starkware avi-starkware force-pushed the avi/prover-v3/http-metrics branch from 2eaa611 to 1da46b5 Compare June 7, 2026 10:11

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b7a8e8e. Configure here.


let after = counter_value(&handle.render(), PANICS_TOTAL);
assert_eq!(after - before, 1.0);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panic tests race global hook

Medium Severity

This module now has two separate #[test] functions that each call take_hook, set_hook, and catch_unwind on the process-global panic hook. Cargo runs those tests in parallel by default, so one test can restore or replace the hook while the other is mid-flight and panic_hook_bumps_panics_total_counter may see zero prover_panics_total delta.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit b7a8e8e. Configure here.

fn panic_hook(info: &PanicHookInfo<'_>) {
// Increment first — if `Backtrace::force_capture` or the `error!` macro
// panic recursively, the counter still reflects the original panic.
metrics::counter!(PANICS_TOTAL).increment(1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counter before recorder installed

Low Severity

The panic hook now increments prover_panics_total on every panic, but main still calls install_panic_hook before install_exporter. Panics in that gap hit the hook while the Prometheus recorder is not installed yet, so those increments are not reflected in scrapes and panic-rate alerts can undercount.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit b7a8e8e. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants