You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a local-only benchmark that times save and load operations on a large object graph against both the 0.12.8 baseline (no Command API support) and the latest (with supportsBulkCommands) Concourse server, so the round-trip wins from #103, #104, and #105 can be empirically measured. Modeled on the existing PreSelectPerformanceBenchmarkTest.java. @Ignored so it does not run on CI — meant to be unignored locally for comparison runs.
Scope
New file:src/test/java/com/cinchapi/runway/CommandApiPerformanceBenchmarkTest.java
Shape
Use the existing benchmark as a template. Key elements:
@Versions({ "0.12.8", "latest" })
@IgnorepublicclassCommandApiPerformanceBenchmarkTestextendsCrossVersionTest {
@OverridepublicvoidbeforeEachTest() {
// Build a large graph: e.g., 5000 Users, each linking one Company// and one Address (mirror the existing benchmark's shape, perhaps// larger to amplify per-record round-trip differences).
}
@TestpublicvoidtestBatchSavePerformance() {
// Time a single db.save(record1, ..., recordN) call where N is// large. After #104 lands, the 1.0.0 row should drop sharply// because per-field writes collapse into one submit per save.
}
@TestpublicvoidtestGraphSavePerformance() {
// Time saving a deeply-linked graph (root + many children +// grandchildren). Validates that recursive// saveModifiedReferenceWithinTransaction also flows through the// CommandGroup once #104 lands.
}
@TestpublicvoidtestBulkLoadPerformance() {
// Time db.load(User.class) on the large graph. After #103 lands,// the 1.0.0 row should drop because the multi-selection dispatch// collapses to one submit instead of thread-pool-fanned-out// per-class queries.
}
@TestpublicvoidtestMultiSelectionPerformance() {
// Time a select(s1, s2, ..., sN) call with selections spanning// multiple classes (which today exercises both combinable and// isolated branches). After #103, all N collapse to one submit.
}
// Inner Record classes (User, Company, Address, etc.) — same as the// existing benchmark.
}
For each @Test, use Benchmark from com.cinchapi.common.profile.Benchmark (the existing pattern) and record(...) to log:
Optionally the round-trip count via a forwarding Concourse that increments a counter on every wire call — gives a deterministic metric independent of wall-clock noise.
Why 0.12.8 specifically
0.12.8 is the most recent server release before the 1.0.0 RC, so the comparison isolates the Command API impact rather than mixing in unrelated improvements between e.g. 0.11 and 1.0.
The benchmark needs both versions of the server available locally (CI runners may not have both, and the test runtime is significant). The benchmark is for local validation, not CI gating — mirrors the existing PreSelectPerformanceBenchmarkTest's @Ignore.
Dependencies
None — the test is valid against the current code (it'll show the 0.12.8 vs latest difference is small / zero before the Command API work lands). Recommended workflow:
Land this ticket.
Run locally to establish baseline numbers for 0.12.8 and 1.0.0-rc against the current Runway code.
Summary
Add a local-only benchmark that times save and load operations on a large object graph against both the 0.12.8 baseline (no Command API support) and the latest (with
supportsBulkCommands) Concourse server, so the round-trip wins from #103, #104, and #105 can be empirically measured. Modeled on the existing PreSelectPerformanceBenchmarkTest.java.@Ignored so it does not run on CI — meant to be unignored locally for comparison runs.Scope
New file:
src/test/java/com/cinchapi/runway/CommandApiPerformanceBenchmarkTest.javaShape
Use the existing benchmark as a template. Key elements:
For each
@Test, useBenchmarkfromcom.cinchapi.common.profile.Benchmark(the existing pattern) andrecord(...)to log:db.properties().supportsBulkCommands()(added by AddsupportsBulkCommandsserver-version flag for Concourse 1.0.0 Command API #101) so the row is self-identifyingConcoursethat increments a counter on every wire call — gives a deterministic metric independent of wall-clock noise.Why 0.12.8 specifically
0.12.8 is the most recent server release before the 1.0.0 RC, so the comparison isolates the Command API impact rather than mixing in unrelated improvements between e.g. 0.11 and 1.0.
Why @ignore'd
The benchmark needs both versions of the server available locally (CI runners may not have both, and the test runtime is significant). The benchmark is for local validation, not CI gating — mirrors the existing
PreSelectPerformanceBenchmarkTest's@Ignore.Dependencies
None — the test is valid against the current code (it'll show the 0.12.8 vs latest difference is small / zero before the Command API work lands). Recommended workflow:
prepare()/submit()round trip #103, Bulk-submit field writes duringRunway.save()#104, Fold stage / reads / writes / commit into a 2-RT save #105 land, re-run to capture the round-trip reduction on the 1.0.0-rc row. The 0.12.8 row should stay flat since the bulk path is gated onsupportsBulkCommands.The before/after deltas on the 1.0.0-rc row are the success metric for the umbrella.
Acceptance
CommandApiPerformanceBenchmarkTestexists,@Ignored.supportsBulkCommandsflag (and round-trip count if the forwarding-counter approach is taken).prepare()/submit()round trip #103-Fold stage / reads / writes / commit into a 2-RT save #105 land show the expected drop on the 1.0.0-rc row.Parent
Part of #100 — Adopt Concourse 1.0.0 Command API in Runway.