[JENKINS-72578] java.util.ConcurrentModificationException in Matrix job

[jenkins-infra-bot]

This is the first time We've seen this issue. About a month ago we upgraded our Jenkins controllers from "2.387.3-lts-jdk11" to "2.426.1-lts-jdk11" and matrix-project plugin from "785.v06b_7f47b_c631" to "818.v7eb_e657db_924".

We have a matrix job with a dynamic axis (for defining test suites to be run) and concurrency enabled. Today a build from this job failed with the following stacktrace when initiating the matrix jobs:

09:42:28 FATAL: null
09:42:28 java.util.ConcurrentModificationException
09:42:28 	at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
09:42:28 	at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
09:42:28 	at hudson.matrix.Combination.toCompactString(Combination.java:210)
09:42:28 	at hudson.matrix.MatrixConfiguration.getDisplayName(MatrixConfiguration.java:227)
09:42:28 	at hudson.model.AbstractItem.getFullDisplayName(AbstractItem.java:491)
09:42:28 	at hudson.console.ModelHyperlinkNote.encodeTo(ModelHyperlinkNote.java:42)
09:42:28 	at hudson.matrix.DefaultMatrixExecutionStrategyImpl.scheduleConfigurationBuild(DefaultMatrixExecutionStrategyImpl.java:242)
09:42:28 	at hudson.matrix.DefaultMatrixExecutionStrategyImpl.run(DefaultMatrixExecutionStrategyImpl.java:159)
09:42:28 	at hudson.matrix.MatrixBuild$MatrixBuildExecution.doRun(MatrixBuild.java:375)
09:42:28 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
09:42:28 	at hudson.model.Run.execute(Run.java:1895)
09:42:28 	at hudson.matrix.MatrixBuild.run(MatrixBuild.java:323)
09:42:28 	at hudson.model.ResourceController.execute(ResourceController.java:101)
09:42:28 	at hudson.model.Executor.run(Executor.java:442)
09:42:28 Finished: FAILURE 

I've attached scrubbed versions of the job config.xml and console output from the failed build.

---

Originally reported by clarkbrianj, imported from: java.util.ConcurrentModificationException in Matrix job

• assignee: kohsuke
• status: Open
• priority: Minor
• component(s): dynamic-axis-plugin, matrix-project-plugin
• resolution: Unresolved
• votes: 0
• watchers: 1
• imported: 2025-12-06

Raw content of original issue

This is the first time We've seen this issue. About a month ago we upgraded our Jenkins controllers from "2.387.3-lts-jdk11" to "2.426.1-lts-jdk11" and matrix-project plugin from "785.v06b_7f47b_c631" to "818.v7eb_e657db_924".

We have a matrix job with a dynamic axis (for defining test suites to be run) and concurrency enabled. Today a build from this job failed with the following stacktrace when initiating the matrix jobs:

09:42:28 FATAL: null
09:42:28 java.util.ConcurrentModificationException
09:42:28 	at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
09:42:28 	at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
09:42:28 	at hudson.matrix.Combination.toCompactString(Combination.java:210)
09:42:28 	at hudson.matrix.MatrixConfiguration.getDisplayName(MatrixConfiguration.java:227)
09:42:28 	at hudson.model.AbstractItem.getFullDisplayName(AbstractItem.java:491)
09:42:28 	at hudson.console.ModelHyperlinkNote.encodeTo(ModelHyperlinkNote.java:42)
09:42:28 	at hudson.matrix.DefaultMatrixExecutionStrategyImpl.scheduleConfigurationBuild(DefaultMatrixExecutionStrategyImpl.java:242)
09:42:28 	at hudson.matrix.DefaultMatrixExecutionStrategyImpl.run(DefaultMatrixExecutionStrategyImpl.java:159)
09:42:28 	at hudson.matrix.MatrixBuild$MatrixBuildExecution.doRun(MatrixBuild.java:375)
09:42:28 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
09:42:28 	at hudson.model.Run.execute(Run.java:1895)
09:42:28 	at hudson.matrix.MatrixBuild.run(MatrixBuild.java:323)
09:42:28 	at hudson.model.ResourceController.execute(ResourceController.java:101)
09:42:28 	at hudson.model.Executor.run(Executor.java:442)
09:42:28 Finished: FAILURE 

I've attached scrubbed versions of the job config.xml and console output from the failed build.

environment

Jenkins: Running in docker (2.426.1-lts-jdk11)<br/>
OS: Linux - 4.14.328-248.540.amzn2.x86_64<br/>
Java: 11.0.21 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)<br/>
Build Agent: Ubuntu-20.04 EC2 node started from ec2-plugin<br/>
---<br/>
Exclusion:0.15<br/>
Matrix-sorter-plugin:1.3<br/>
Office-365-Connector:4.20.2<br/>
PrioritySorter:5.0.0<br/>
analysis-model-api:11.13.0<br/>
ansicolor:1.0.4<br/>
ant:497.v94e7d9fffa_b_9<br/>
antisamy-markup-formatter:162.v0e6ec0fcfcf6<br/>
any-buildstep:14.ve115ec1484f0<br/>
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757<br/>
artifact-manager-s3:822.vf129d4836c31<br/>
artifactory:4.0.0<br/>
audit-trail:359.va_7c0fb_cc4f1f<br/>
authentication-tokens:1.53.v1c90fd9191a_b_<br/>
authorize-project:1.7.1<br/>
aws-credentials:218.v1b_e9466ec5da_<br/>
aws-global-configuration:128.ve2c5685a_09c3<br/>
aws-java-sdk:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-cloudformation:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-codebuild:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-ec2:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-ecr:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-ecs:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-efs:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-elasticbeanstalk:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-iam:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-kinesis:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-logs:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-minimal:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-secretsmanager:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-sns:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-sqs:1.12.586-413.v6a_6c3a_420126<br/>
aws-java-sdk-ssm:1.12.586-413.v6a_6c3a_420126<br/>
aws-secrets-manager-credentials-provider:1.213.vca_3f37306fed<br/>
aws-secrets-manager-secret-source:1.72.v61781b_35c542<br/>
azure-credentials:293.vb_d506148f506<br/>
azure-sdk:157.v855da_0b_eb_dc2<br/>
azure-vm-agents:883.v63c930b_025dc<br/>
badge:1.9.1<br/>
basic-branch-build-strategies:81.v05e333931c7d<br/>
blueocean:1.27.9<br/>
blueocean-autofavorite:1.2.5<br/>
blueocean-bitbucket-pipeline:1.27.9<br/>
blueocean-commons:1.27.9<br/>
blueocean-config:1.27.9<br/>
blueocean-core-js:1.27.9<br/>
blueocean-dashboard:1.27.9<br/>
blueocean-display-url:2.4.2<br/>
blueocean-events:1.27.9<br/>
blueocean-git-pipeline:1.27.9<br/>
blueocean-github-pipeline:1.27.9<br/>
blueocean-i18n:1.27.9<br/>
blueocean-jira:1.27.9<br/>
blueocean-jwt:1.27.9<br/>
blueocean-personalization:1.27.9<br/>
blueocean-pipeline-api-impl:1.27.9<br/>
blueocean-pipeline-editor:1.27.9<br/>
blueocean-pipeline-scm-api:1.27.9<br/>
blueocean-rest:1.27.9<br/>
blueocean-rest-impl:1.27.9<br/>
blueocean-web:1.27.9<br/>
bootstrap5-api:5.3.2-2<br/>
bouncycastle-api:2.29<br/>
branch-api:2.1135.v8de8e7899051<br/>
build-cause-run-condition:0.1<br/>
build-keeper-plugin:19.va_df8a_2c65123<br/>
build-monitor-plugin:1.14-745.ve2023a_305f40<br/>
build-name-setter:2.4.0<br/>
build-timeout:1.31<br/>
build-user-vars-plugin:1.9<br/>
buildresult-trigger:0.18<br/>
built-on-column:1.4<br/>
caffeine-api:3.1.8-133.v17b_1ff2e0599<br/>
categorized-view:1.13<br/>
checks-api:2.0.2<br/>
chucknorris:159.vdfe649cb_9c37<br/>
claim:516.v36293563731d<br/>
cloud-stats:320.v96b_65297a_4b_b_<br/>
cloudbees-bitbucket-branch-source:856.v04c46c86f911<br/>
cloudbees-folder:6.858.v898218f3609d<br/>
clover:4.14.1.577.vb_ff5a_f69b_37b_<br/>
cobertura:1.17<br/>
code-coverage-api:4.99.0<br/>
command-launcher:107.v773860566e2e<br/>
commons-httpclient3-api:3.1-3<br/>
commons-lang3-api:3.13.0-62.v7d18e55f51e2<br/>
commons-text-api:1.11.0-94.v3e1f4a_926e49<br/>
computer-queue-plugin:1.7<br/>
conditional-buildstep:1.4.3<br/>
config-file-provider:959.vcff671a_4518b_<br/>
configuration-as-code:1746.vf1673cfe690a<br/>
configuration-as-code-groovy:1.1<br/>
copy-project-link:106.veb_028794a_844<br/>
copyartifact:722.v0662a_9b_e22a_c<br/>
coverage:1.5.0<br/>
cppcheck:1.26<br/>
cppncss:1.2<br/>
credentials:1309.v8835d63eb_d8a_<br/>
credentials-binding:642.v737c34dea_6c2<br/>
cucumber-reports:5.8.0<br/>
custom-tools-plugin:0.8<br/>
cvs:2.19.1<br/>
dashboard-view:2.495.v07e81500c3f2<br/>
data-tables-api:1.13.8-1<br/>
datetime-constraint:0.1.2<br/>
delivery-pipeline-plugin:1.4.2<br/>
depgraph-view:1.0.5<br/>
description-setter:1.10-CUSTOM<br/>
disable-github-multibranch-status:1.2<br/>
display-url-api:2.200.vb_9327d658781<br/>
docker-commons:439.va_3cb_0a_6a_fb_29<br/>
docker-workflow:572.v950f58993843<br/>
downstream-ext:73.vdda_16e6eb_0da<br/>
dropdown-viewstabbar-plugin:1.7<br/>
dtkit-api:3.0.2<br/>
durable-task:523.va_a_22cf15d5e0<br/>
dynamic-axis:1.0.3<br/>
ec2:1648.vf3d852e00486<br/>
echarts-api:5.4.3-1<br/>
ecutest:2.42<br/>
email-ext:2.102<br/>
embeddable-build-status:412.v09da_db_1dee68<br/>
envinject:2.908.v66a_774b_31d93<br/>
envinject-api:1.199.v3ce31253ed13<br/>
environment-variable-page-decoration:1.3.0<br/>
excludeMatrixParent:1.1<br/>
extended-choice-parameter:376.v2e02857547b_a_<br/>
extended-read-permission:53.v6499940139e5<br/>
extensible-choice-parameter:1.8.1<br/>
external-monitor-job:215.v2e88e894db_f8<br/>
external-workspace-manager:1.3.1<br/>
ez-templates:1.3.5<br/>
favorite:2.4.3<br/>
file-parameters:316.va_83a_1221db_a_7<br/>
fitnesse:1.36<br/>
flexible-publish:0.16.1<br/>
font-awesome-api:6.4.2-1<br/>
forensics-api:2.3.0<br/>
git:5.2.1<br/>
git-client:4.5.0<br/>
git-forensics:2.0.0<br/>
git-parameter:0.9.19<br/>
git-server:99.va_0826a_b_cdfa_d<br/>
github:1.37.3.1<br/>
github-api:1.318-461.v7a_c09c9fa_d63<br/>
github-autostatus:3.6.2<br/>
github-branch-source:1752.vc201a_0235d80<br/>
github-issues:1.2.4<br/>
github-oauth:597.ve0c3480fcb_d0<br/>
github-scm-trait-commit-skip:0.4.0<br/>
global-variable-string-parameter:1.2<br/>
google-oauth-plugin:1.319.v5cb_58b_91b_389<br/>
gradle:2.9<br/>
groovy:457.v99900cb_85593<br/>
groovy-postbuild:2.5<br/>
h2-api:11.1.4.199-12.v9f4244395f7a_<br/>
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953<br/>
htmlpublisher:1.32<br/>
hubot-steps:95.va_30176518a_5a<br/>
instance-identity:185.v303dc7c645f9<br/>
ionicons-api:56.v1b_1c8c49374e<br/>
ivy:2.5<br/>
jackson2-api:2.15.3-372.v309620682326<br/>
jacoco:3.3.5<br/>
jakarta-activation-api:2.0.1-3<br/>
jakarta-mail-api:2.0.1-3<br/>
javadoc:243.vb_b_503b_b_45537<br/>
javax-activation-api:1.2.0-6<br/>
javax-mail-api:1.6.2-9<br/>
jaxb:2.3.9-1<br/>
jdk-tool:73.vddf737284550<br/>
jenkins-design-language:1.27.9<br/>
jenkins-flowdock-plugin:1.1.8<br/>
jenkins-metrics-sender:0.01-SNAPSHOT (private-08/28/2023 21:34-root)<br/>
jenkins-multijob-plugin:623.v03401733c9a_9<br/>
jersey2-api:2.41-133.va_03323b_a_1396<br/>
jfrog:1.5.0<br/>
jira:3.12<br/>
jjwt-api:0.11.5-77.v646c772fddb_0<br/>
jnr-posix-api:3.1.18-1<br/>
job-dsl:1.87<br/>
job-restrictions:0.8<br/>
jobConfigHistory:1229.v3039470161a_d<br/>
jquery:1.12.4-1<br/>
jquery3-api:3.7.1-1<br/>
jsch:0.2.8-65.v052c39de79b_2<br/>
junit:1240.vf9529b_881428<br/>
junit-attachments:205.vc0677977deb_0<br/>
keepSlaveOffline:1.0<br/>
keyboard-shortcuts-plugin:1.4<br/>
label-verifier:1.4<br/>
ldap:711.vb_d1a_491714dc<br/>
locale:314.v22ce953dfe9e<br/>
lockable-resources:1215.v895f61d7f794<br/>
log-parser:2.3.1<br/>
logstash:2.5.0218.v0a_ff8fefc12b_<br/>
mailer:463.vedf8358e006b_<br/>
mapdb-api:1.0.9-28.vf251ce40855d<br/>
mask-passwords:173.v6a_077a_291eb_5<br/>
matrix-auth:3.2.1<br/>
matrix-combinations-parameter:1.3.3<br/>
matrix-project:818.v7eb_e657db_924<br/>
maven-plugin:3.23<br/>
measurement-plots:0.1<br/>
mercurial:1260.vdfb_723cdcc81<br/>
metrics:4.2.18-442.v02e107157925<br/>
mina-sshd-api-common:2.11.0-86.v836f585d47fa_<br/>
mina-sshd-api-core:2.11.0-86.v836f585d47fa_<br/>
monitoring:1.95.0<br/>
multi-branch-priority-sorter:1.0<br/>
multibranch-action-triggers:1.8.6<br/>
multibranch-build-strategy-extension:1.1.0<br/>
nested-view:1.33<br/>
next-build-number:1.8<br/>
node-iterator-api:55.v3b_77d4032326<br/>
nodelabelparameter:1.12.0<br/>
oauth-credentials:0.646.v02b_66dc03d2e<br/>
offlineonfailure-plugin:1.1-SNAPSHOT (private-03/11/2016 13:37-gsix)<br/>
okhttp-api:4.11.0-157.v6852a_a_fa_ec11<br/>
pam-auth:1.10<br/>
parameterized-scheduler:255.v73827fcdf618<br/>
parameterized-trigger:787.v665fcf2a_830b_<br/>
pipeline-aggregator-view:1.15<br/>
pipeline-build-step:516.v8ee60a_81c5b_9<br/>
pipeline-github:2.8-155.8eab375ac9f8<br/>
pipeline-github-lib:42.v0739460cda_c4<br/>
pipeline-githubnotify-step:49.vf37bf92d2bc8<br/>
pipeline-graph-analysis:202.va_d268e64deb_3<br/>
pipeline-groovy-lib:689.veec561a_dee13<br/>
pipeline-input-step:477.v339683a_8d55e<br/>
pipeline-maven:1362.vee39a_d4b_02b_1<br/>
pipeline-maven-api:1362.vee39a_d4b_02b_1<br/>
pipeline-milestone-step:111.v449306f708b_7<br/>
pipeline-model-api:2.2151.ve32c9d209a_3f<br/>
pipeline-model-definition:2.2151.ve32c9d209a_3f<br/>
pipeline-model-extensions:2.2151.ve32c9d209a_3f<br/>
pipeline-multibranch-defaults:2.1<br/>
pipeline-rest-api:2.34<br/>
pipeline-stage-step:305.ve96d0205c1c6<br/>
pipeline-stage-tags-metadata:2.2151.ve32c9d209a_3f<br/>
pipeline-stage-view:2.34<br/>
pipeline-timeline:1.0.3<br/>
pipeline-utility-steps:2.16.0<br/>
plain-credentials:143.v1b_df8b_d3b_e48<br/>
plot:2.1.12<br/>
plugin-usage-plugin:4.2<br/>
plugin-util-api:3.6.0<br/>
pollscm:1.5<br/>
postbuild-task:1.9<br/>
postbuildscript:3.2.0-550.v88192b_d3e922<br/>
preSCMbuildstep:71.v1f2990a_37e27<br/>
prism-api:1.29.0-9<br/>
progress-bar-column-plugin:11.vdef198c2d6c1<br/>
project-build-times:1.2.1<br/>
project-description-setter:1.2<br/>
project-health-report:1.2<br/>
promoted-builds:936.va_571a_a_b_f8da_5<br/>
publish-over:0.22<br/>
publish-over-cifs:0.16<br/>
pubsub-light:1.18<br/>
python:1.3<br/>
random-string-parameter:1.0<br/>
read-only-configurations:1.10<br/>
rebuild:330.v645b_7df10e2a_<br/>
release:2.18<br/>
remote-file:1.24<br/>
resource-disposer:0.23<br/>
rich-text-publisher-plugin:1.5<br/>
role-strategy:689.v731678c3e0eb_<br/>
run-condition:1.7<br/>
saferestart:0.7<br/>
saml:4.429.v9a_781a_61f1da_<br/>
scm-api:683.vb_16722fb_b_80b_<br/>
script-security:1294.v99333c047434<br/>
sectioned-view:1.25<br/>
secure-requester-whitelist:67.vca_a_d9205723f<br/>
seed:2.1.4<br/>
shelve-project-plugin:3.2<br/>
show-build-parameters:1.0<br/>
sidebar-link:2.4.1<br/>
simple-build-for-pipeline:0.2<br/>
simple-theme-plugin:176.v39740c03a_a_f5<br/>
slave-setup:1.16<br/>
snakeyaml-api:2.2-111.vc6598e30cc65<br/>
sse-gateway:1.26<br/>
ssh-agent:346.vda_a_c4f2c8e50<br/>
ssh-credentials:308.ve4497b_ccd8f4<br/>
ssh-slaves:2.916.vd17b_43357ce4<br/>
sshd:3.312.v1c601b_c83b_0e<br/>
statusmonitor:1.3<br/>
structs:325.vcb_307d2a_2782<br/>
subversion:2.17.3<br/>
summary_report:1.15<br/>
test-stability:2.3<br/>
throttle-concurrents:2.14<br/>
timestamper:1.26<br/>
token-macro:384.vf35b_f26814ec<br/>
trilead-api:2.84.v72119de229b_7<br/>
urltrigger:1.02<br/>
validating-string-parameter:183.v3748e79b_9737<br/>
variant:60.v7290fc0eb_b_cd<br/>
versioncolumn:217.v16457a_b_eb_7eb_<br/>
view-job-filters:369.ve0513a_a_f5524<br/>
warnings-ng:10.5.1<br/>
webhook-step:327.vb_58f208a_7783<br/>
workflow-aggregator:596.v8c21c963d92d<br/>
workflow-api:1283.v99c10937efcb_<br/>
workflow-basic-steps:1042.ve7b_140c4a_e0c<br/>
workflow-cps:3812.vc3031a_b_a_c955<br/>
workflow-durable-task-step:1289.v4d3e7b_01546b_<br/>
workflow-job:1360.vc6700e3136f5<br/>
workflow-multibranch:756.v891d88f2cd46<br/>
workflow-remote-loader:1.6<br/>
workflow-scm-step:415.v434365564324<br/>
workflow-step-api:639.v6eca_cd8c04a_a_<br/>
workflow-support:865.v43e78cc44e0d<br/>
ws-cleanup:0.45<br/>
xtrigger-api:1.0<br/>
xunit:3.1.3<br/>
zentimestamp:4.2

2 attachments

• test_console_scrubbed.log
• test_job_config.xml