[JENKINS-68944] Matrix jobs do not start with custom label/expression for the built-in node

[jenkins-infra-bot]

Matrix jobs only start when specifying exactly built-in as label to select the built-in node.

When using a label expression like "built-in || master" or a different label that is configured for the built-in node, the matrix job will stay in the queue forever.

This used to work fine with Jenkins 2.264.3.

Note: the matrix jobs WILL start fine with a label or label expression that resolves e.g. to a docker cloud instead of the built-in node.

---

Originally reported by capf, imported from: Matrix jobs do not start with custom label/expression for the built-in node

• status: Open
• priority: Minor
• component(s): matrix-project-plugin
• resolution: Unresolved
• votes: 0
• watchers: 3
• imported: 2025-12-06

Raw content of original issue

Matrix jobs only start when specifying exactly built-in as label to select the built-in node. 

When using a label expression like "built-in || master" or a different label that is configured for the built-in node, the matrix job will stay in the queue forever.

This used to work fine with Jenkins 2.264.3.

Note: the matrix jobs WILL start fine with a label or label expression that resolves e.g. to a docker cloud instead of the built-in node.

environment

Jenkins 2.346.1<br/>
matrix-project-plugin 771.v574584b_39e60 at least

1 attachment

• image-2022-07-08-00-17-43-563.png

  

[jenkins-infra-bot]

volodja:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  FYI: A matrix parent job is handled as flyweight task. The labels of the built-in node are ignored during the check whether the flyweight task is bound to the built-in node. Only self-label will be respected. See https://github.com/jenkinsci/jenkins/blob/jenkins-2.346.1/core/src/main/java/hudson/model/Queue.java#L1740
  

FYI: A matrix parent job is handled as flyweight task. The labels of the built-in node are ignored during the check whether the flyweight task is bound to the built-in node. Only self-label will be respected. See https://github.com/jenkinsci/jenkins/blob/jenkins-2.346.1/core/src/main/java/hudson/model/Queue.java#L1740

[jenkins-infra-bot]

markewaite:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  I can't duplicate the problem as reported.  I've been using built-in || master as a job label for quite a while with good results.  Can you provide more details that will help others understand how you are seeing the issue?
  
  Steps I took to try to duplicate the issue:
  
  
  	
  Run a Jenkins 2.346.1 controller with matrix project plugin 772.v494f19991984 as described in my lts-with-plugins branch
  	
  Add agents to the controller, set the agent executor count on the controller to zero
  	
  Define a matrix project job with the label built-in || master and confirm that it starts but does not complete so long as executor count on the controller is zero
  	
  Change controller executor count to 1, confirm that the job runs on the controller
  
  
  

I can't duplicate the problem as reported. I've been using built-in || master as a job label for quite a while with good results. Can you provide more details that will help others understand how you are seeing the issue?

Steps I took to try to duplicate the issue:

1. Run a Jenkins 2.346.1 controller with matrix project plugin 772.v494f19991984 as described in my lts-with-plugins branch
2. Add agents to the controller, set the agent executor count on the controller to zero
3. Define a matrix project job with the label built-in || master and confirm that it starts but does not complete so long as executor count on the controller is zero
4. Change controller executor count to 1, confirm that the job runs on the controller

[jenkins-infra-bot]

volodja:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  The problem occurs when you configure label built-in || master for matrix project (parent job) itself and not for axis.
  
  
  
  <?xml version="1.1" encoding="UTF-8" standalone="no"?>
  <matrix-project plugin="matrix-project@771.v574584b_39e60">
    ...
    <assignedNode>built-in || master</assignedNode>
    <canRoam>false</canRoam>
    ...
  </matrix-project>
  
  
  

The problem occurs when you configure label built-in || master for matrix project (parent job) itself and not for axis.

"1.1" encoding="UTF-8" standalone="no"?>
"matrix-project@​771.v574584b_39e60">
  ...
  built-in || master
  false
  ...

[jenkins-infra-bot]

markewaite:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  Thanks for the clarification volodja.  I thought that flyweight tasks were strictly limited to the controller.  Isn't it creating an opportunity for confusion to allow a label on the "Restrict where this project can be run" setting of a matrix project?  That doesn't control the location where the matrix jobs run, it only controls where the flyweight task runs, even though the flyweight task can only run on the controller.
  

Thanks for the clarification volodja. I thought that flyweight tasks were strictly limited to the controller. Isn't it creating an opportunity for confusion to allow a label on the "Restrict where this project can be run" setting of a matrix project? That doesn't control the location where the matrix jobs run, it only controls where the flyweight task runs, even though the flyweight task can only run on the controller.

[jenkins-infra-bot]

capf:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  If that's the case, it appears sensible to me to remove that option altogether. We only set that option to ensure that the parent job runs on built-in.
  

If that's the case, it appears sensible to me to remove that option altogether. We only set that option to ensure that the parent job runs on built-in.

[jenkins-infra-bot]

volodja:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  Jenkins allows for "flyweight" tasks to run on a slave node, which does not occupy an executor and does not affect the slave node's lifecycle, so if Jenkins was executed some flyweight tasks on a container that was busy running a build, and then the build finishes, the container would be terminated without regard to the flyweight tasks.
  See ChannelClosedException due to spurious termination of running containers docker-plugin#653 (comment)
  
  This was the reason to force matrix project flyweight task to run on controller using explicitly configured master or built-in label for the job.
  Since PR #3983: Try to execute flyweight tasks on the master the probability that flyweigh task will be executed not on controller is very low but not excluded.
  

Jenkins allows for "flyweight" tasks to run on a slave node, which does not occupy an executor and does not affect the slave node's lifecycle, so if Jenkins was executed some flyweight tasks on a container that was busy running a build, and then the build finishes, the container would be terminated without regard to the flyweight tasks.

See jenkinsci/docker-plugin#653 (comment)

This was the reason to force matrix project flyweight task to run on controller using explicitly configured master or built-in label for the job.
Since PR #3983: Try to execute flyweight tasks on the master the probability that flyweigh task will be executed not on controller is very low but not excluded.

[jenkins-infra-bot]

markewaite:

• Comment link
• Original comment link (no redirect)
• Raw content of original comment:

  Thanks for the education volodja.  Much appreciated.  Your comments validate that the issue as reported by capf is a real issue.  Since flyweight execution is allowed on agents, it seems to me that the label expression should be evaluated.  Work around is to use the exact label expression ({built-in}} or master
  

Thanks for the education volodja. Much appreciated. Your comments validate that the issue as reported by capf is a real issue. Since flyweight execution is allowed on agents, it seems to me that the label expression should be evaluated. Work around is to use the exact label expression ({built-in}} or master