Skip to content

Add chaos test GitHub Actions workflow#917

Open
moran-abilea wants to merge 1 commit into
hyperledger:mainfrom
moran-abilea:chaos-test-clean
Open

Add chaos test GitHub Actions workflow#917
moran-abilea wants to merge 1 commit into
hyperledger:mainfrom
moran-abilea:chaos-test-clean

Conversation

@moran-abilea

@moran-abilea moran-abilea commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

This commit adds automated chaos testing infrastructure for ARMA:

  • Daily scheduled runs at 6 AM Israel time (03:00 UTC) from Sun till Thu, only on Fri the test is longer
  • Configurable test parameters (duration, rate, parties, shards)
  • Sequential chaos pattern (stop/start components in order)
  • Automatic result collection and artifact upload
  • Support for both short and long-running tests

The workflow runs all ARMA components as separate processes on a single GitHub-hosted runner, with chaos testing that sequentially stops and restarts components to test system resilience.

Files added:

  • .github/workflows/chaos-test.yml: Main workflow definition
  • .github/workflows/chaosRunner/chaos-test.sh: Orchestration script
  • .github/workflows/chaosRunner/start-arma-network.sh: Network startup
  • .github/workflows/chaosRunner/chaos-runner.sh: Chaos implementation
  • .github/workflows/chaosRunner/monitor-completion.sh: Test monitoring
  • .github/workflows/chaosRunner/collect-results.sh: Result collection

@moran-abilea moran-abilea self-assigned this Jun 10, 2026
@moran-abilea moran-abilea force-pushed the chaos-test-clean branch 6 times, most recently from 4ec6e20 to 97c8f81 Compare June 18, 2026 10:55
@moran-abilea moran-abilea marked this pull request as ready for review June 18, 2026 13:18
@moran-abilea moran-abilea requested a review from tock-ibm June 18, 2026 13:53
@tock-ibm tock-ibm requested a review from Copilot June 22, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a scheduled/manual GitHub Actions workflow to run automated chaos testing for ARMA on a single GitHub-hosted runner, plus a related refactor that removes “primary signature” fields/methods from batch/BAF types and updates call sites/tests accordingly.

Changes:

  • Add a new Chaos Test GitHub Actions workflow with scheduled and workflow_dispatch runs, artifact upload, and configurable parameters.
  • Add bash runner scripts to generate configs, start the ARMA network, inject chaos (stop/start components), monitor completion, and collect results.
  • Remove PrimarySignature from core Batch / BatchAttestationFragment interfaces and update constructors/call sites, mocks, and tests to match the new signatures.

Reviewed changes

Copilot reviewed 38 out of 46 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
.github/workflows/chaos-test.yml Adds the scheduled/manual chaos-test workflow and artifact upload steps.
.github/workflows/chaosRunner/chaos-test.sh Orchestrates config generation, network startup, load/receive, chaos runner, monitoring, and result collection.
.github/workflows/chaosRunner/start-arma-network.sh Starts consensus/batcher/assembler/router processes and records PIDs.
.github/workflows/chaosRunner/chaos-runner.sh Implements sequential stop/restart chaos pattern per party/shard.
.github/workflows/chaosRunner/monitor-completion.sh Monitors duration/early completion, prints periodic stats, and signals stop.
.github/workflows/chaosRunner/collect-results.sh Collects logs/statistics and writes a human-readable summary report.
common/types/types.go Removes PrimarySignature() from Batch and BatchAttestationFragment interfaces.
common/types/simple_batch.go Removes stored primary signature from SimpleBatch and updates constructor.
common/types/simple_batch_test.go Updates tests for the new NewSimpleBatch signature.
common/types/simple_baf.go Removes primary signature from SimpleBatchAttestationFragment and ASN.1 encoding; updates constructor/docs.
common/types/simpla_baf_test.go Updates tests for the new NewSimpleBatchAttestationFragment signature.
common/types/tools_test.go Updates helper tests to use the new BAF constructor signature.
node/ledger/batch_encoding.go Removes PrimarySignature() from FabricBatch implementation.
node/consensus/utils.go Updates config block metadata construction to match new batch constructor signature.
node/consensus/state/state_test.go Updates consensus state serialization tests for new BAF constructor signature.
node/consensus/setup_test.go Updates CreateBAF test helper calls for new CreateBAF signature.
node/consensus/consenter_test.go Updates tests to the new BAF constructor signature.
node/consensus/consensus_test.go Updates many test BAF creations to the new CreateBAF signature.
node/consensus/consensus_real_reconfig_test.go Updates CreateBAF usage to new signature in reconfig test.
node/batcher/batcher.go Updates Batcher.CreateBAF and CreateBAF to drop the primary signature parameter.
node/batcher/batcher_role.go Updates BAF creation to drop primary signature usage and removes signature-related TODO.
node/batcher/batcher_test.go Updates tests to new BAF constructor signature.
node/batcher/batcher_role_test.go Updates tests to new NewSimpleBatch / NewSimpleBatchAttestationFragment and mock BAFCreator signature.
node/batcher/batcher_role_old_test.go Updates legacy tests/bench helpers to new batch/BAF constructor + BAFCreator signature.
node/batcher/mocks/baf_creator.go Updates mock BAFCreator interface/signature to drop primary signature arg.
node/batcher/mocks/state_provider.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/request_verifier.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/request_inspector.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/mem_pool.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/decision_replicator.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/consenter_control_event_sender.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/consenter_control_event_sender_creator.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/consensus_decision_replicator_creator.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/config_sequence_getter.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/complainer.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/batches_puller.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/batched_requests_verifier.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/batch_ledger.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/batch_acker.go Minor mock stub invocation refactor (no functional change intended).
node/batcher/mocks/baf_sender.go Minor mock stub invocation refactor (no functional change intended).
node/assembler/utils_test.go Updates assembler tests to new NewSimpleBatch signature.
node/assembler/prefetch_benchmark/batch_generator_test.go Updates benchmark helper to new NewSimpleBatch signature.
node/assembler/collator_test.go Updates test batches to new NewSimpleBatch signature.
node/assembler/assembler_deliver_test.go Updates deliver tests to new NewSimpleBatch signature.
node/assembler/assembler_batcher_consenter_test.go Updates integration-style assembler tests to new NewSimpleBatch signature.
test/reconfig/configtx/config_disseminate_test.go Updates BAF creation in reconfig test to new constructor signature.
Files not reviewed (8)
  • node/batcher/mocks/baf_creator.go: Generated file
  • node/batcher/mocks/baf_sender.go: Generated file
  • node/batcher/mocks/batch_acker.go: Generated file
  • node/batcher/mocks/batch_ledger.go: Generated file
  • node/batcher/mocks/batched_requests_verifier.go: Generated file
  • node/batcher/mocks/batches_puller.go: Generated file
  • node/batcher/mocks/complainer.go: Generated file
  • node/batcher/mocks/config_sequence_getter.go: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/chaosRunner/chaos-test.sh
Comment thread .github/workflows/chaos-test.yml
Comment thread common/types/simple_baf.go Outdated
Comment thread common/types/types.go
@moran-abilea moran-abilea force-pushed the chaos-test-clean branch 7 times, most recently from f9b5ba7 to 34a65f6 Compare June 25, 2026 09:53
@tock-ibm tock-ibm requested a review from Copilot June 25, 2026 11:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Comment thread .github/workflows/chaosRunner/monitor-completion.sh Outdated
Comment thread .github/workflows/chaosRunner/collect-results.sh
Comment thread .github/workflows/chaosRunner/collect-results.sh
Comment thread .github/workflows/chaos-test.yml Outdated
Comment thread .github/workflows/chaosRunner/chaos-test.sh Outdated
Comment thread test/chaos/chaos-test.sh
@moran-abilea moran-abilea force-pushed the chaos-test-clean branch 4 times, most recently from 1db5473 to 81f1fac Compare June 28, 2026 11:41
@moran-abilea moran-abilea requested a review from tock-ibm June 28, 2026 11:43
@moran-abilea moran-abilea force-pushed the chaos-test-clean branch 7 times, most recently from f343bb3 to f3b3d25 Compare July 6, 2026 08:13
@tock-ibm tock-ibm force-pushed the chaos-test-clean branch from f3b3d25 to 391761b Compare July 6, 2026 10:02
also fixing review notes
adding readme.md as well

Signed-off-by: Moran Abilea <moran.abilea@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants