How to use S3 credentials from pillar

[Andy Lee]

I'm trying to source a managed file from S3.  I've put valid S3 credentials into a pillar:

# salt 'minion00' pillar.item s3.key
minion00:
    ----------
    s3.key:
        CVmsdkfjak...


# salt 'minion00' pillar.item s3.keyid
minion00:
    ----------
    s3.keyid:
        AKAKAKAKAK....

and set up a managed file state:

/path/to/some.file:
  file:
    - managed
    - source: s3://some-bucket/some.file
    - source_hash: sha512=...
    - user: testuser
    - group: testgroup
    - mode: 644

but applying the state fails:

2015-03-02 22:45:18,597 [salt.state                                  ][INFO    ] Running state [/path/to/some.file] at time 22:45:18.597076
2015-03-02 22:45:18,598 [salt.state                                  ][INFO    ] Executing state file.managed for /path/to/some.file
2015-03-02 22:45:18,610 [urllib3.connectionpool                      ][INFO    ] Starting new HTTP connection (1): <ip address>
2015-03-02 22:45:18,615 [urllib3.connectionpool                      ][INFO    ] Starting new HTTP connection (1): <ip address>
2015-03-02 22:45:18,617 [salt.utils.iam                              ][ERROR   ] Failed to read metadata. Giving up on IAM credentials.
2015-03-02 22:45:18,622 [salt.state                                  ][ERROR   ] Unable to manage file: Could not fetch from s3://some-bucket/some.file
2015-03-02 22:45:18,622 [salt.state                                  ][INFO    ] Completed state [/path/to/some.file] at time 22:45:18.621957

where does one wire up the credentials such that they get used? I haven't been able to find any docs that explain how the pillar vars should get picked up

[Bruce Wang]

For s3fs to work, you need to put s3.key/s3.keyid in your *master config* (e.g. /etc/salt/master) or use IAM Roles on salt master, see http://docs.saltstack.com/en/latest/ref/file_server/all/salt.fileserver.s3fs.html

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

simple is good
http://brucewang.net
http://twitter.com/number5

[Andy Lee]

The file.managed docs claim downloading directly from Amazon S3 compatible URLs is supported. It mentions "s3.get state documentation" but I haven't found any such thing.  The s3 module docs refer to putting the credentials in a pillar, but don't mention what comes after that.

[Bruce Wang]

Hi Andy,

Sorry it should be s3 module instead of s3fs module, and from the code https://github.com/saltstack/salt/blob/develop/salt/modules/s3.py#L214 it use config.option to get the s3.key/s3.keyid, which will only get it from master config or minion config, since for pillar you need to use ':' (e.g. s3:key)

If pass None for both parameters, salt.utils.s3.get will use IAM credentials.

So either the docs is wrong, or the code is wrong, might be good to file an issue on github.

Cheers,

Bruce