test: Tier-2 correctness coverage — formats, schema evolution, producer durability, rebalance, tracing, lifecycle#221
Merged
Merged
Conversation
Owner
Author
|
@copilot please review |
Copilot stopped work on behalf of
eschizoid due to an error
June 22, 2026 20:28
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
…er durability, rebalance, tracing, lifecycle Consolidates the Tier-2 verification fleet (surfaces #217 never touched). 11 new test classes; 57 local tests pass (0 skipped/failed), 3 Testcontainers suites run in CI. Format/SerDe edge cases (local): - JsonFormatEdgeCasesTest, ProtobufFormatEdgeCasesTest, AvroFormatEnvelopeEdgeCasesTest — wire-prefix boundary, oversized/empty/malformed payloads, skipBytes interaction. - AvroFormatSchemaEvolutionTest — writer/reader projection across FORWARD-compatible evolution (field add/remove-with-default, type promotion) via GenericDatumReader. - TypedPipelineSkipBytesTest — skipBytes prefix-strip semantics + double-strip guard. Producer (local): - KPipeProducerDurabilityTest — sendToDlq durability contract, idempotence/acks passthrough. Consumer matrix + lifecycle (local): - ProcessingModeSinkDlqMatrixTest — ProcessingMode x sink x DLQ x retry x circuit-breaker. - LifecycleLeakCycleTest — repeated start/stop/crash cycles, no thread/resource leak. Integration (CI-only, Testcontainers): - RebalanceAtScaleIntegrationTest — cooperative-sticky, mid-flight revocation, single-writer. - DlqTracePropagationIntegrationTest — W3C traceparent across the DLQ Kafka boundary. - KPipeRealBrokerBackpressureIntegrationTest — pause/resume under real broker pressure, no loss. Incidental: jqwik 1.9.3 -> 1.10.1 (carried from the fleet base). Note: branched off the #220 tip; will rebase --onto main once #220 merges.
eschizoid
force-pushed
the
test/tier2-coverage
branch
from
1a0e892 to
e2ab1e8
Compare
Owner
Author
|
@copilot please review |
Copilot stopped work on behalf of
eschizoid due to an error
June 22, 2026 20:44
…on-drain assert saturationHoldDoesNotLoseOrCorruptRecords read dispatcher.pendingCount() the instant the per-record latch released and asserted == 0. pendingCount is decremented in the worker's finally, which can run just after the latch counts down, so the immediate read flaked on slow CI runners (failed once on the #221 branch). Poll for the drain with a 5s deadline before the hard assert — the same idiom already applied to the other pendingCount drain check in this file (line 424). Pre-existing flake surfaced by #221's CI; not caused by the Tier-2 tests. Verified stable across 3 local reruns.
eschizoid
added a commit
that referenced
this pull request
Jul 3, 2026
…ker to fix CI flake RebalanceAtScaleIntegrationTest was the only class failing on main CI (3 runs: the #221 merge run + both attempts of the #218 run). The signature was identical every time: produce timed out at line 325 (send.get in the produceRecords setup helper) with org.apache.kafka.common.errors.TimeoutException, always in this one class, for its later-ordered methods (cooperativeSticky, revoke) while midFlight passed. Root cause: the class shared ONE static KafkaContainer across its three @test methods, each of which creates its own 6-partition topic + consumer groups and never tears them down. That state accumulated on the single broker until a later method's producer could not get acks within delivery.timeout.ms, failing at the produce step during setup. It is the only container test class heavy enough (3 methods x 6 partitions) to degrade its own broker this way, which is why it was the only one that flaked — a runner-wide resource problem would have scattered failures across the other five container classes, and it did not. Fix: make the @container field non-static so JUnit's default per-method lifecycle starts a fresh broker for each test, isolating them. Costs ~2 extra broker startups (~40s) for this one class; leaves the five healthy container classes untouched. Cannot run Testcontainers locally (no Docker); verified it compiles. CI is the confirmation — will re-run the branch a few times to confirm the rebalance tests stay green.
eschizoid
added a commit
that referenced
this pull request
Jul 3, 2026
…ker to fix CI flake (#224) RebalanceAtScaleIntegrationTest was the only class failing on main CI (3 runs: the #221 merge run + both attempts of the #218 run). The signature was identical every time: produce timed out at line 325 (send.get in the produceRecords setup helper) with org.apache.kafka.common.errors.TimeoutException, always in this one class, for its later-ordered methods (cooperativeSticky, revoke) while midFlight passed. Root cause: the class shared ONE static KafkaContainer across its three @test methods, each of which creates its own 6-partition topic + consumer groups and never tears them down. That state accumulated on the single broker until a later method's producer could not get acks within delivery.timeout.ms, failing at the produce step during setup. It is the only container test class heavy enough (3 methods x 6 partitions) to degrade its own broker this way, which is why it was the only one that flaked — a runner-wide resource problem would have scattered failures across the other five container classes, and it did not. Fix: make the @container field non-static so JUnit's default per-method lifecycle starts a fresh broker for each test, isolating them. Costs ~2 extra broker startups (~40s) for this one class; leaves the five healthy container classes untouched. Cannot run Testcontainers locally (no Docker); verified it compiles. CI is the confirmation — will re-run the branch a few times to confirm the rebalance tests stay green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Tier-2 of the verification roadmap: coverage for the correctness surfaces #217 never touched. 11 new test classes — 57 local tests pass (0 skipped/failed), 3 Testcontainers suites run in CI.
Format / SerDe (local, unit)
JsonFormatEdgeCasesTest,ProtobufFormatEdgeCasesTest,AvroFormatEnvelopeEdgeCasesTest— wire-prefix boundary, oversized/empty/malformed payloads,skipBytesoff-by-one. The off-by-one cases assert a decode failure (matched on the format's own error message), not any exception, so they can't green on an unrelated error.AvroFormatSchemaEvolutionTest— writer/reader projection across FORWARD-compatible evolution (field add, remove-with-default, type promotion) viaGenericDatumReader; proves the static-reader corruption by contrast with the registry path on the same bytes.TypedPipelineSkipBytesTest— pins the exact post-skip byte slice, boundary cases, and the deserialize-or-fail null contract.Producer (local)
KPipeProducerDurabilityTest—sendToDlqdurability contract (result undecided until the broker ack lands), idempotence/acks passthrough.Consumer matrix + lifecycle (local)
ProcessingModeSinkDlqMatrixTest—ProcessingMode× sink shape × DLQ × retry × circuit-breaker, each driven end-to-end through a started consumer on aMockConsumer. Multi-sink cell routes through the realCompositeMessageSinkbroadcast; batch cell uses size 5 over 12 records so a genuine trailing partial batch exercises the close()-drain path.LifecycleLeakCycleTest— repeated start/stop/crash cycles, no thread/resource accumulation.Integration (CI-only, Testcontainers)
RebalanceAtScaleIntegrationTest— cooperative-sticky, mid-flight revocation, single-writer (revoke runs on the consumer thread).DlqTracePropagationIntegrationTest— W3Ctraceparentacross the DLQ Kafka boundary.KPipeRealBrokerBackpressureIntegrationTest— pause at high watermark / resume under real broker pressure, no loss.Review
4-lens review (test-analyzer, code-reviewer, comment-analyzer, silent-failure-hunter) ran on the branch before opening. Addressed: a
§Nspec-ref in a comment, the mislabeled multi-sink test (now uses realCompositeMessageSink), catch-all "corrupt or fail" disjunctions narrowed to require the codec's own decode failure,RebalanceAtScaleIntegrationTestgivendisabledWithoutDocker = trueto match its siblings, the batch cell reworked to actually flush a partial batch at teardown, and Javadoc/comment line-wrap fixes.Notes