What feature do you want to see added?
I'd like the plugin to optionally manage EC2 cloud and agent-template configuration from a YAML file held in a Git repository, and sync it into Jenkins on a schedule. It would all be configured from Manage Jenkins > System.
Background:
At my work we run a fleet of large Jenkins controllers, and managing EC2 cloud and agent-template config by hand across all of them isn't feasible. To deal with that I built a small companion plugin that reads a YAML file and writes the EC2 clouds and templates into Jenkins, and we sync it every night. The nightly sync means we can be confident the config on every controller is in a known, version-controlled state, even if someone changes it out of band. With this proposal we could also disable manual changes entirely.
Maintaining this internal companion plugin can be a headache. Because it constructs EC2Cloud and SlaveTemplate objects directly, it breaks whenever this plugin changes those classes (the AWS SDK v2 migration and the new SlaveTemplate constructor arguments are recent examples). Our controllers run jobs at all hours and it can take a week or more to schedule a maintenance window, so rebuilding and redeploying the companion plugin after every ec2-plugin release (on every server) is slow and painful. Building this into ec2-plugin directly, and binding values with @DataBoundSetter instead of positional constructors, removes that breakage and gives the feature a proper home with a UI.
For context, I recently wrote the proxmox-cloud-plugin, which already does this: git-sourced YAML, cron sync, settings in Manage Jenkins, an allow/deny manual changes option, and a per-cloud status indicator. That approach has worked well for us, so this is largely a port of the same design into ec2-plugin, adapted to EC2's cloud and template properties.
Proposed behavior:
- A settings section in Manage Jenkins > System for: enable on/off, Git repo URL, branch, Git credentials, YAML file path, and a sync cron schedule, plus Test Git Connection and Sync Now buttons.
- A periodic job that pulls the file on the configured schedule (for example nightly) and reconciles only the EC2 clouds. Clouds created manually are not touched.
- An "Allow manual changes" toggle and a status indicator on the cloud config page (for example "Config Managed" with the last-synced time, and read-only when manual changes are disallowed), matching the proxmox-cloud-plugin behavior.
- A layered YAML schema with cloud and agent defaults, per-region defaults, and a way to attach an agent to one or more clouds, so large fleets stay terse:
cloudDefaults:
region: ap-southeast-2
sshKeysCredentialsId: ec2-agent-key
cloudConfigurations:
prod-apse2:
name: "Prod (ap-southeast-2)"
agentDefaults:
remoteFS: /home/agent
numExecutors: "1"
agentDefaults-ap-southeast-2:
subnetId: subnet-abc123
securityGroups: sg-buildagents
agentConfigurations:
docker-host:
cloudIds: [prod-apse2]
ami: ami-0123456789
labelString: docker-hostThe layering matters because it is far less repetitive than a flat config. Adding a new agent is usually only a few lines: you set the handful of fields that are specific to it and inherit everything else from the cloud, agent and region defaults, rather than copy-pasting a whole cloud or template block each time. On a large fleet that keeps the config small and easy to read.
Why not just JCasC:
JCasC is the obvious comparison, but it applies configuration at startup or reload only. Our controllers run jobs around the clock and a restart can be a week or more away, so we need to re-sync on a cron schedule (e.g. nightly) and on demand without restarting Jenkins, and to push a change immediately when something needs to move. This does that, reconciles only the EC2 clouds rather than the whole instance, and uses a terser schema, while still building the same EC2Cloud and SlaveTemplate objects underneath. I see it as complementary to JCasC rather than a replacement.
Benefits:
- EC2 config is version-controlled and re-applied automatically on a schedule, so every controller stays in a known state, and manual changes can be blocked if wanted.
- Syncs on a schedule and on demand with no Jenkins restart, unlike JCasC.
- No separate companion plugin to keep rebuilding every time ec2-plugin changes.
- Works at large scale, where managing cloud config by hand isn't practical.
- The layered defaults keep day-to-day changes small: a new agent is a few lines of overrides rather than a full copy of every property.
Upstream changes
No Jenkins core changes needed. The additions are all within this plugin:
- New runtime dependencies (BOM-managed):
io.jenkins.plugins:snakeyaml-api and org.jenkins-ci.plugins:git-client, plus structs for data-bound instantiation if it isn't already pulled in.
- A new
GlobalConfiguration, an AsyncPeriodicWork scheduler, and a few additive status fields on EC2Cloud (config-managed and last-synced). These are backward compatible and work with JCasC.
- YAML to object mapping via
@DataBoundSetter / DescribableModel rather than positional constructors, so it doesn't break when fields change.
Are you interested in contributing this feature?
Yes, I plan to implement it fully with tests and documentation. Since it mirrors the proxmox-cloud-plugin, I expect it to be a straightforward port.
But since it is a non-trivial change, I wanted to raise the proposal here first for discussion and approval from maintainers before I commit to work on it.
What feature do you want to see added?
I'd like the plugin to optionally manage EC2 cloud and agent-template configuration from a YAML file held in a Git repository, and sync it into Jenkins on a schedule. It would all be configured from Manage Jenkins > System.
Background:
At my work we run a fleet of large Jenkins controllers, and managing EC2 cloud and agent-template config by hand across all of them isn't feasible. To deal with that I built a small companion plugin that reads a YAML file and writes the EC2 clouds and templates into Jenkins, and we sync it every night. The nightly sync means we can be confident the config on every controller is in a known, version-controlled state, even if someone changes it out of band. With this proposal we could also disable manual changes entirely.
Maintaining this internal companion plugin can be a headache. Because it constructs
EC2CloudandSlaveTemplateobjects directly, it breaks whenever this plugin changes those classes (the AWS SDK v2 migration and the newSlaveTemplateconstructor arguments are recent examples). Our controllers run jobs at all hours and it can take a week or more to schedule a maintenance window, so rebuilding and redeploying the companion plugin after every ec2-plugin release (on every server) is slow and painful. Building this into ec2-plugin directly, and binding values with@DataBoundSetterinstead of positional constructors, removes that breakage and gives the feature a proper home with a UI.For context, I recently wrote the proxmox-cloud-plugin, which already does this: git-sourced YAML, cron sync, settings in Manage Jenkins, an allow/deny manual changes option, and a per-cloud status indicator. That approach has worked well for us, so this is largely a port of the same design into ec2-plugin, adapted to EC2's cloud and template properties.
Proposed behavior:
The layering matters because it is far less repetitive than a flat config. Adding a new agent is usually only a few lines: you set the handful of fields that are specific to it and inherit everything else from the cloud, agent and region defaults, rather than copy-pasting a whole cloud or template block each time. On a large fleet that keeps the config small and easy to read.
Why not just JCasC:
JCasC is the obvious comparison, but it applies configuration at startup or reload only. Our controllers run jobs around the clock and a restart can be a week or more away, so we need to re-sync on a cron schedule (e.g. nightly) and on demand without restarting Jenkins, and to push a change immediately when something needs to move. This does that, reconciles only the EC2 clouds rather than the whole instance, and uses a terser schema, while still building the same
EC2CloudandSlaveTemplateobjects underneath. I see it as complementary to JCasC rather than a replacement.Benefits:
Upstream changes
No Jenkins core changes needed. The additions are all within this plugin:
io.jenkins.plugins:snakeyaml-apiandorg.jenkins-ci.plugins:git-client, plusstructsfor data-bound instantiation if it isn't already pulled in.GlobalConfiguration, anAsyncPeriodicWorkscheduler, and a few additive status fields onEC2Cloud(config-managed and last-synced). These are backward compatible and work with JCasC.@DataBoundSetter/DescribableModelrather than positional constructors, so it doesn't break when fields change.Are you interested in contributing this feature?
Yes, I plan to implement it fully with tests and documentation. Since it mirrors the proxmox-cloud-plugin, I expect it to be a straightforward port.
But since it is a non-trivial change, I wanted to raise the proposal here first for discussion and approval from maintainers before I commit to work on it.