Stop the Konsist gate passing without running, and land the Kotlin daemon measurement - #158
Merged
Merged
Conversation
… the first attempt The Kotlin daemon is a second JVM that every module's Kotlin compilation runs through, and it was previously unset. Sweeping it needed its own mode, since reading the Gradle daemon's GC log while varying the Kotlin heap measures nothing. Measured twice with the candidate order reversed, because one pass cannot separate a heap effect from the build warming up as the sweep proceeds: heap 1st order reversed mean 1g 44.1s 40.7s 42.4s 1536m 37.9s 37.2s 37.6s 2g 32.6s 33.8s 33.2s <- chosen 3g 30.1s 38.0s 34.1s That control is load-bearing. 3g looked like the clear winner in the first order and mid-pack in the second - a position effect, not a heap one. 1g is slowest even from the last slot where warming helps most, so it is really too small. 2g is the only candidate fast in both orders. Only wall time is trusted here, and the file says so: the daemon exits the moment compilation ends and its GC log was not reliably captured, so its live-set and GC figures moved ~40% between identical runs. The guard was wrong twice, which is why the first sweep looked broken and the second looked fine while reporting the same false alarm. `pgrep -f P -l` matches nothing on macOS - options must precede the pattern. Corrected, it still found nothing, because polling after the build is the wrong moment for a daemon that has already exited. It now checks the artifact instead: if the daemon did not write the GC log we asked it for, the row is labelled NO-GC-LOG rather than reported as a confident zero. The script also now flags a monotonic wall-time trend across candidates and tells you to re-run reversed, which is how the position effect above was found.
A full `:konsist:test` died with "OutOfMemoryError thrown from the UncaughtExceptionHandler in thread Test worker". Gradle forks test workers with a 512 MB heap and org.gradle.jvmargs does not reach them - that sizes the daemon, not the worker. These rules parse every .kt file in the repo into an in-memory Konsist model, and since the module joined billionbeers.jvm.library it also carries jacoco instrumentation, which pushed it over. It hid unusually well, in both directions at once. A single-class run (`--tests "*OneRuleTest*"`) fits in the default heap and passes, which is how the recent rule work was verified. An unchanged repo leaves the task UP-TO-DATE, so `make konsist` returned green without executing anything. Only a full, non-cached run reaches the ceiling - and that is the run CI does. maxHeapSize = 2g. Verified by a --rerun-tasks run: 15 tests across 13 classes execute and pass, where before the worker died before reporting any.
The architecture gate has now failed silently three separate ways. It never ran at all, because `make test` targets testDebugUnitTest and a pure-JVM module has no such task. It went UP-TO-DATE while the code it guards changed, because the rules read the repo off the filesystem where Gradle cannot see it. And most recently a full run died of OutOfMemory in the test worker while single-class runs kept passing, so it looked green from every angle anyone checked. Each of those got its own fix. This one addresses the shape rather than another symptom: after the run, every `*Test.kt` rule file on disk must have produced a JUnit result class. A rule that exists but never executes now fails the build instead of reading as coverage - which is precisely what AGENTS.md warns about twice and what nothing until now could detect. It is deliberately self-maintaining. The expected set is derived from the files present, not a hardcoded count, so adding a rule raises the bar automatically and the check cannot go stale. It also fails if it finds no rule files at all, since that would make both the check and the gate pass vacuously. A `--tests` filter skips it, because running one rule class is then the intent. Verified with a mutation probe: a rule file with no @test methods fails the build with "These Konsist rules exist but did not run: GhostRuleTest. Ran 13 of 14", and removing it returns the build to green. A single-class --tests run stays green throughout.
simtop
enabled auto-merge (squash)
August 9, 2026 14:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Picks up the two commits that #157 did not take — it merged only its first commit, so the Kotlin daemon measurement and the Konsist heap fix never reached master — and then fixes the underlying pattern rather than another instance of it.
The Konsist gate is broken on master right now
A full
./gradlew :konsist:test --rerun-taskson master dies withOutOfMemoryErrorin the test worker. Gradle forks test workers with a 512 MB heap andorg.gradle.jvmargsdoes not reach them — that sizes the daemon, not the worker. These rules parse every.ktfile in the repo into an in-memory Konsist model, and since #155 put the module onbillionbeers.jvm.libraryit also carries jacoco instrumentation.maxHeapSize = "2g"fixes it.It hid from both directions at once, which is why it survived review: a single-class run (
--tests "*OneRuleTest*") fits in the default heap and passes, and an unchanged repo leaves the taskUP-TO-DATE. Only a full, non-cached run reaches the ceiling — the run CI does.The actual problem: this gate has now failed silently three times
make testtargetstestDebugUnitTest, which a pure-JVM module does not have.Each got its own fix. Three instances of one shape is a pattern, so this adds a check for the shape: after the run, every
*Test.ktrule file on disk must have produced a JUnit result class. A rule that exists but never executes now fails the build instead of reading as coverage — exactly what AGENTS.md warns about twice, and what nothing until now could detect.It is deliberately self-maintaining. The expected set is derived from the files present rather than a hardcoded count, so adding a rule raises the bar automatically and the check cannot go stale. It also fails if it finds no rule files, since that would let both the check and the gate pass vacuously. A
--testsfilter skips it, because running one rule class is then the intent.Verified with a mutation probe — a rule file with no
@Testmethods:Removing the probe returns the build to green; a single-class
--testsrun stays green throughout.Kotlin daemon heap, measured
kotlin.daemon.jvmargswas previously unset, so the JVM every module's Kotlin compilation runs through took an undocumented plugin default. Measured twice with the candidate order reversed, because one pass cannot separate a heap effect from the build warming up as the sweep proceeds:That control is load-bearing: 3g would have been the wrong answer from a single pass. It won the first ordering outright and came mid-pack in the second — an 8-second swing that tracks position in the list, not heap size. 1g is slowest even from the last slot where warming helps most, so it is genuinely too small. 2g is the only candidate fast in both orders.
Honest limitation, also written into
gradle.properties: only wall time is trusted here. The Kotlin daemon exits the moment compilation ends and its GC log was not reliably captured, so its live-set and GC-overhead figures moved ~40% between identical runs and were not used to pick the value.The measurement script's own guard was wrong twice
Worth recording, since it is the same failure shape as everything above — a check reporting success while verifying nothing:
pgrep -f PATTERN -lmatches nothing on macOS (options must precede the pattern), so it reported "no Kotlin daemon found" while a daemon was running with exactly the requested heap.It now verifies the artifact instead of the process: if the daemon did not write the
-Xlog:gc:file=it was given, the row is labelledNO-GC-LOGrather than reported as a confident zero. The script also detects monotonic-in-order wall times and tells you to re-run reversed — which is how the position effect above was caught.Verification
make test,make konsist,make lint,spotlessCheck, plus the two mutation probes described above.