wget/apt-key command failing from master but succeeding on minions

[Justin Lloyd]

I've been trying to upgrade all of my minion's salt-minion packages, but I'm having problems with many minions failing the wget of the GPG key when initiated by the master but succeeding when initiated locally. I have a pkgrepo.managed state in a salt.pkgrepo formula but for a simpler case I created a salt.pkgrepo-simple formula that just uses file.managed for saltstack.list and cmd.run for the key-fetching command. The following examples boil down the key fetching issue.

### Running on the master
master# salt "minion-name" cmd.run 'wget -q -O - https://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest/SALTSTACK-GPG-KEY.pub | apt-key add -'

minion-name:

    gpg: no valid OpenPGP data found.

ERROR: Minions returned with non-zero exit code


### Running on the minion

minion# salt-call cmd.run 'wget -q -O - https://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest/SALTSTACK-GPG-KEY.pub | apt-key add -'         

[INFO    ] Executing command 'wget -q -O - https://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest/SALTSTACK-GPG-KEY.pub | apt-key add -' in directory '/root'

local:

    OK
minion# apt-key list | grep -C 1 Salt

pub   2048R/DE57BFBE 2014-06-24

uid                  SaltStack Packaging Team <pack...@saltstack.com>

sub   2048R/17928113 2014-06-24

There's really no useful information in any of the salt daemon logs. Any thoughts why this may be happening? The masters are all Ubuntu 14.04 and 16.04 (the master-of-master example above is 16.04) running Salt 2017.7.0. The minions are a mix of 12, 14, and 16.04 and various ages back to 2015.5.3, hence the upgrades, but none of that seems to make a difference as far as I can tell.

[Daniel Wallace]

Remember, running commands from the salt master through the
salt-minion command is run in the environment of the init system
process, while running it from salt-call runs it as an interactive
shell process.

GPG usually likes an interactive shell.

I would highly recommend using the `pkg.add_repo_key` from the aptpkg
module for adding the key.

It should work with both methods.

Thanks,
Daniel

> --
> You received this message because you are subscribed to the Google Groups
> "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to salt-users+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/salt-users/e9ceb662-a599-4d19-b297-bc49b4f3769b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Justin Lloyd]

I thought about the shell thing when I stepped away from my computer to get a drink, haven't tested a way to try to make use of that. However, it looks like pkg.add_repo_key was just added in 2017.7.0 and my problem stems from trying to upgrade all of my minions. Also, many of my minions are Ubuntu 12.04 with some running salt-minion as old as 2015.5.3 and the latest version for 12.04 is only 2016.11.3.

# salt -E minion-name grains.item osrelease saltversion

minion-name:
    ----------

    osrelease:

        12.04

    saltversion:
        2015.5.3

minion# salt -E minion-name pkg.add_repo_key https://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest/SALTSTACK-GPG-KEY.pub
minion-name:
    'pkg.add_repo_key' is not available.

[Justin Lloyd]

For what it's worth, I've also been having serious problems even trying to fetch from the Salt repo over the last couple of days, making this whole upgrade process even harder. For example sometimes this will happen when trying to start, or even DURING a download:

Failed to fetch http://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest/pool/main/s/salt/salt-common_2016.11.3+ds-1_all.deb  503  Service Unavailable