Use puppet_repositories from theforeman.operations collection

[ehelms]

No description provided.

[ekohl]

This should document how to get the ansible collection.

Another thing I dislike is that the requirements states pulp_installer is needed, but I won't need that for most operations. It means I'll pull in a lot more than I really need.

Now I also need to get stuff from galaxy. As someone who dislikes Ansible, this feels like a step back. I'm leaning to holding off on using any collections until there's a real benefit. I can see it in complex roles like where we now use hammer, but here it feels complex. It is literally more lines of code even if you don't take the added dependency into account.

[evgeni]

The idea of the operations collection was that we also dogfood it, and forklift seems like a good place for dogfooding :)

We could split the requirements files into Foreman and Pulp easily, would that help? Eric and I expected this to be a problem at some point due to different usecases.

[ekohl]

We could split the requirements files into Foreman and Pulp easily, would that help? Eric and I expected this to be a problem at some point due to different usecases.

I think that would be a good idea.

[ehelms]

We could split the requirements files into Foreman and Pulp easily, would that help? Eric and I expected this to be a problem at some point due to different usecases.

I am not seeing the value in splitting. Having one requirements.yaml means users running a single command, means our CI not having to track job-to-job which file to install from and when I checked locally this totaled 6 MB. If I extrapolate, and we started using FAM in some of the seeding roles, would that be another requirements.yaml file?

[ekohl]

I still think this is a bad execution.

First of all, we lose the ability to upgrade Puppet between releases. theforeman/foreman-operations-collection#20 is IMHO a blocker to merging it.

Then it unconditionally enables Katello repositories. That's also a big regression IMHO. Foreman must be able to install without Katello and the best way to test it is to not have the repository present. That it slows down tests is maybe a smaller issue but it also means that Foreman's pipeline can be red if the Katello repositories are offline.

Lastly I really dislike adding a dependency on ansible-galaxy, especially if it also pulls in the whole pulp installer with all its dependencies.

[ehelms]

Then it unconditionally enables Katello repositories. That's also a big regression IMHO. Foreman must be able to install without Katello and the best way to test it is to not have the repository present. That it slows down tests is maybe a smaller issue but it also means that Foreman's pipeline can be red if the Katello repositories are offline.

I am missing something here. Katello repositories are only enabled when a variable is defined for that purpose: https://github.com/theforeman/foreman-operations-collection/blob/develop/roles/foreman_repositories/tasks/redhat.yml#L26

Lastly I really dislike adding a dependency on ansible-galaxy, especially if it also pulls in the whole pulp installer with all its dependencies.

You aren't really saying why you dislike having the dependency, or why it is bad for Forklift. Can you expand in a way that I can either make changes to address or explain more about it?

[ehelms]

I'll get around to updating this one once #1313 is in as I feel like I should get that set of changes in first and then make this one.

[ekohl]

In general I'm not a fan of Ansible and having to rely on more Ansible tools is something that I'm hesitant about.

For example, if I want to make a change to a role that's used and test it out, I know how to do it and easily send a PR. Instead of ansible-galaxy, can I point it to a git checkout to develop on?

[ehelms]

For example, if I want to make a change to a role that's used and test it out, I know how to do it and easily send a PR. Instead of ansible-galaxy, can I point it to a git checkout to develop on?

You can do:

collections:
  - name: git+file:///home/user/path/to/repo/.git
    type: git

I could add this to the docs to give some lead in for developers to know how to do it.

[ehelms]

See #1313 for updated documentation regarding questions here. That PR is the first use of the collection that I want to get in and then come back and update this PR.

[evgeni]

Thanks @ehelms!

I think there is one thing left: the old role used puppet_repositories_version to indicate which version to install, while the new one uses foreman_puppet_repositories_version, but nothing sets this new variable right now. We don't deploy anything but 6 right now, so it won't break today, but I guess soon we'll want to try 7.

We can either add a translation step that does foreman_puppet_repositories_version: "{{ puppet_repositories_version }}" or teach the forklift_versions role to set the right variable directly.

I prefer the later.

[ehelms]

Done (I hope)

[evgeni]

Ha!

git grep says you also want to edit:

forklift/vagrant/lib/forklift/box_factory.rb

Line 102 in 46e7829

'puppet_repositories_version' => version['puppet']

[evgeni]

@ehelms would you mind finishing this up? the recent puppet.com key rotation broke nightly pipelines, and I expect this to fix it once and forever

[ehelms]

Fixed up that one spot and dropped that playbook

[ekohl]

Should we drop https://github.com/theforeman/forklift/blob/master/playbooks/bats_pipeline_foreman_nightly.yml at the same time for consistency?

[evgeni]

probably a good idea, yeah, even if unrelated to the original change here.

[ehelms]

There's probably a few things we ought to clean up, I am happy to do that separately from the intent of this change.

[evgeni]

I dropped the one file Ewoud pointed out in #1383

If you see more, please do the same :)