package.install with repository (Debian) does not work

[farax4de]

Maybe someone can give me a hand.

this part works

actions:
  - action: command.run
    command: sh
    args: ["-c" , "curl -cfs https://apt.releases.hashicorp.com/gpg  | gpg --dearmor > /usr/share/keyrings/hashicorp-archive-keyring.gpg"]

at this point I get an error

ERROR execute:manifest_load{manifest=“nomad.yaml”}:

  - action: package.install
    provider: apt
    name: nomad
    repository: deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com {{ os.codemane }} main

[martintc]

I'll take a look and see what is going on. I've been meaning to take a look at the repository functionality since it's never been well documented and we need some documentation in that area.

[farax4de]

...never been well documented..
I can add the documentation once it is clear how it works.

[martintc]

Just started looking at the action tonight and I did notice in what you had pasted, looks like there is a spelling error.

os.codemane Not sure if this was just an error typing it into GitHub or if this was an error in what was ran.

[martintc]

Also out of curiosity, I noticed you specified apt with the provider. We don't necessarily get a lot of feedback unless it's in either discord or GitHub issues, do you often use multiple package managers on the same system?

Just curious to gather a data point that people do in fact actually make use of that.

[farax4de]

I want to add an apt source to install Debian packages from this new source. In this specific case, I want to automate the installation of Nomad. The description of the manual steps can be found here: https://developer.hashicorp.com/nomad/docs/install
In general, I would like to use comtrya to control the installation and configuration of systems - similar to Saltstack.

[martintc]

Try something like this. For apt, the repository setting isn't respected since I believe that was more added in for homebrew where taps/casks (its versions of repos) are easily specified when installing. Homebrew doesn't also have a requirement for gpg keys or anything. Its not too dissimilar from go modules where a tap/casks is just a specially structures git repository. We do have actions for specifying adding repositories separately from the install which with the current build of Comtrya, would be the way to do it for apt-based systems. Admittedly, this is the first time I tried running the package.repository action.

It did fail for me. But I am also not sure if HashiCorp provides the bits in their repo for aarch64 since I was running this on with Debian aarch64 in a UTM virtual machine.

This is also the bit that isn't well documented. And never really has been for the linux package managers. In the old documentation, there was some documentation on using package.repository with home-brew, but home-brew is a simple case as I stated above.

[martintc]

Alright, more experimenting and looking at the code. The package.repository action is gonna need a lot of love. Starting some work on it now.

[farax4de]

According to my analysis, there are probably two errors.

• The first major one concerns the call of apt-add-repository ( .../providers/aptitude.rs) in the add_repository function. In “add_repository() -> steps.extend” line 110 the repository entry is extended by “arch=$(dpkg --print-architecture)”. This is a subshell call. It only works if the command apt-add-repository is executed in a shell. This is probably not the case in atoms::command::Exec.
  • As the native architecture is always referenced, the entire optional “arch=...” entry can be removed.
  • In addition, the second step ("add_repository() line 123) ‘apt update’ is not necessary as apt-add-repository already executes this implicitly (see apt-add-repository --help -> option ‘-n’).

actions:
  # load an convert gpg key
  - action: command.run
    command: sh
    args: ["-c" , "curl -cfs https://apt.releases.hashicorp.com/gpg  | gpg --dearmor > /usr/share/keyrings/hashicorp-archive-keyring.asc"]

  - action: package.repository
    # url: "https://apt.releases.hashicorp.com/ $(lsb_release -cs) main"
    url: https://apt.releases.hashicorp.com/ {{os.codename}} main
    key:
      name: hashicorp-archive-keyring
      url: file:///usr/share/keyrings/hashicorp-archive-keyring.asc

produce:

root@nomad0:/var/tmp/comtrya-roles# comtrya -vd nomad apply
DEBUG execute:{manifest="nomad"}:{action=package.install}: stdout: Reading package lists...
Building dependency tree...
Reading state information...
coreutils is already the newest version (9.1-1).
curl is already the newest version (7.88.1-10+deb12u8).
bridge-utils is already the newest version (1.7.1-1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

 INFO execute:{manifest="nomad"}:{action=package.install}: Installing packages
DEBUG execute:{manifest="nomad"}:{action=command.run}: stdout: 
 INFO execute:{manifest="nomad"}:{action=command.run}: Running sh command
DEBUG execute:{manifest="nomad"}:{action=package.repository}: stdout: 
DEBUG execute:{manifest="nomad"}:{action=package.repository}: exit code: 0
DEBUG execute:{manifest="nomad"}:{action=package.repository}: stdout: Unable to handle repository shortcut 'deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.asc] https://apt.releases.hashicorp.com/ bookworm main'

DEBUG execute:{manifest="nomad"}:{action=package.repository}: stderr: 
DEBUG execute:{manifest="nomad"}:{action=package.repository}: Atom failed to execute: Command failed with exit code: 1
 INFO execute:{manifest="nomad"}:{action=package.repository}: Adding repository https://apt.releases.hashicorp.com/ bookworm main
ERROR execute:{manifest="nomad"}: Failed

with

url: "https://apt.releases.hashicorp.com/ $(lsb_release -cs) main"

this result:

root@nomad0:/var/tmp/comtrya-roles# comtrya -vd nomad apply
DEBUG execute:{manifest="nomad"}:{action=package.install}: stdout: Reading package lists...
Building dependency tree...
Reading state information...
coreutils is already the newest version (9.1-1).
curl is already the newest version (7.88.1-10+deb12u8).
bridge-utils is already the newest version (1.7.1-1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

 INFO execute:{manifest="nomad"}:{action=package.install}: Installing packages
DEBUG execute:{manifest="nomad"}:{action=command.run}: stdout: 
 INFO execute:{manifest="nomad"}:{action=command.run}: Running sh command
DEBUG execute:{manifest="nomad"}:{action=package.repository}: stdout: 
DEBUG execute:{manifest="nomad"}:{action=package.repository}: exit code: 0
DEBUG execute:{manifest="nomad"}:{action=package.repository}: stdout: Unable to handle repository shortcut 'deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.asc] https://apt.releases.hashicorp.com/ $(lsb_release -cs) main'

DEBUG execute:{manifest="nomad"}:{action=package.repository}: stderr: 
DEBUG execute:{manifest="nomad"}:{action=package.repository}: Atom failed to execute: Command failed with exit code: 1
 INFO execute:{manifest="nomad"}:{action=package.repository}: Adding repository https://apt.releases.hashicorp.com/ $(lsb_release -cs) main
ERROR execute:{manifest="nomad"}: Failed

• The second error concerns the replacement of variables such as “{{ os.code_xxx_name }}”. If the variable is misspelled or not known, the error “ERROR execute:manifest_load{manifest=”nomad.yaml“}:” is always generated. This also applies if the syntax is anywhere within a comment!

[farax4de]

Unfortunately there is also a Debian 12 specific problem. The entry for the GPG key is apparently not accepted by apt-add-repository, see #995895 [i| | ] [software-properties-common] software-properties: add-apt-repository signed-by attribute invalid

[martintc]

I noticed that last night and started working on a branch to try some things and also was running into more issues. I will check out that link you shared because that may be the answer to what I was seeing. I was having an issue adding the docker repos using apt-add-repository even from the CLI.