Skip to content

docs: add research benchmark metrics table#5

Merged
CoreyLeath-code merged 1 commit into
mainfrom
docs/add-research-metrics-table
Jul 12, 2026
Merged

docs: add research benchmark metrics table#5
CoreyLeath-code merged 1 commit into
mainfrom
docs/add-research-metrics-table

Conversation

@CoreyLeath-code

@CoreyLeath-code CoreyLeath-code commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replaces the compact README metrics table with research-style benchmark and recorded metrics sections.
  • Adds recorded platform targets, local smoke benchmarks, Prometheus/observability metric inventory, and project reproducibility metrics.
  • Records validation limits where native Go/C++ compiler checks could not be run in the local workspace.

Validation

  • pytest --basetemp=.pytest-tmp/run --cov=api --cov-report=term-missing tests/test_health.py tests/test_auth.py tests/test_inference.py tests/test_rate_limit.py
    • 4 passed
    • 24% focused api coverage
  • Local synthetic benchmarks:
    • SentinelModel: 577 trainable params, 0.0160 ms avg forward-pass latency, 62,470.72 RPS synthetic throughput
    • PSI/KS Python-equivalent drift calculation: 0.001549 ms avg over 10,000 runs
    • LLM fallback summary path: 0.000104 ms avg over 10,000 runs
  • git diff --check

Notes

  • Installed slowapi locally to run the FastAPI test suite because it is declared in requirements.txt but was not present in the shared workspace venv.
  • Native Go/C++ compile checks were not run because Go and C++ compilers were unavailable in this workspace.

Summary by CodeRabbit

  • Documentation
    • Replaced the basic metrics table with a dated, comprehensive research benchmarks and recorded metrics section.
    • Added platform targets, local smoke benchmark results, observability metrics, and project reproducibility metrics.
    • Included measurement and normalization details for improved transparency and reproducibility.

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 46e579bb-6846-4c63-af66-df890316afe6

📥 Commits

Reviewing files that changed from the base of the PR and between 0f8a841 and 9bc329e.

📒 Files selected for processing (1)
  • README.md

📝 Walkthrough

Walkthrough

README.md replaces the former metrics table with dated platform benchmarks, local smoke results, observability metric definitions, and project reproducibility counts measured and normalized on 2026-07-12.

Changes

README metrics documentation

Layer / File(s) Summary
Benchmark and reproducibility metrics
README.md
Adds categorized tables for platform targets, local benchmark results, observability metrics, repository and test counts, CI/CD workflows, dependencies, delivery infrastructure, and validation limitations.

Estimated code review effort: 1 (Trivial) | ~5 minutes

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/add-research-metrics-table

Comment @coderabbitai help to get the list of available commands.

@CoreyLeath-code CoreyLeath-code marked this pull request as ready for review July 12, 2026 22:49
@CoreyLeath-code CoreyLeath-code merged commit 1e01f8b into main Jul 12, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant