fix: correct hook p95 latency claim to per-platform reality#51
Merged
Conversation
Measured on Apple Silicon macOS, the full native daemon-client path (binary exec + Unix-socket round-trip) runs at p95 ~15-17ms, not <10ms. The <10ms claim is only valid for the in-process compute path (no IPC overhead). - Raise the native daemon-client release gate from p95<10ms to p95<20ms, which brackets the measured Apple Silicon baseline while still catching regressions - Update test_claude_code_native_daemon_client_latency_target docstring to state the per-platform measured numbers and clarify which claim boundary each test defends - Add explicit docstring to test_claude_code_daemon_hot_path_latency_target explaining the <10ms gate applies to in-process compute only - Update plugins/claude-code/README.md "Claim boundary" section with split per-path numbers (in-process <10ms, full client path <20ms) - Mirror the README change to site/content/source/ Closes #37 for docs/test scope; full re-measurement on a wider platform matrix should remain in the issue for tracking.
The site/content/source/ mirror must be regenerated via sync_source_docs.py rather than edited by hand. The previous commit wrote the correct content body but left the source_sha256 frontmatter stale. Running the sync script updates the SHA256 to match the new plugins/claude-code/README.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relates to #37 (left open — broader platform re-measurement still warranted).
What changed
The
p95<10msclaim for the Claude Code hook was only accurate forin-process compute (pure passport validation + scope check + receipt
emission, no IPC). The full native daemon-client path — where the
hook binary forks, connects to the daemon over a Unix socket, and parses
the response — measures ~15–17ms p95 on Apple Silicon macOS, not
<10ms.
Files changed
python/tests/test_claude_code_hook_latency.pytest_claude_code_native_daemon_client_latency_targetgate fromp95<10mstop95<20ms(brackets measured baseline, still catches regressions). Add explicit docstrings to both gated tests clarifying which path each one measures and why the numbers differ.plugins/claude-code/README.mdsite/content/source/plugins/claude-code/README.mdWhy not just fix the number everywhere?
The
p95<10msclaim remains correct for the in-process hot-path test(
test_claude_code_daemon_hot_path_latency_target); that gate isunchanged. The issue is that one gated test was labelled "defensible
p95<10ms release claim" while measuring the full round-trip path, which
is meaningfully slower.
Test results
(Latency benchmarks are skipped unless
ARDUR_RUN_LATENCY_BENCH=1; the4 pre-existing collection errors are unrelated to this change.)