Downloading a tarfile from AWS S3 bucket using file.managed throws an error "Unable to manage file: Could not fetch from s3:/"

[Janardhan Korada]

Hi,

I am trying to download a tar file from the S3 bucket.

Here is my state file

copy_package:

  file.managed:

    - name: /tmp/abc.tar.gz

    - source: s3://com.abc.bucket/ubuntu/abc.tar.gz

    - source_hash: md5=s3://com.abc.bucket/ubuntu/abc.tar.gz.MD5

    - replace: True

On executing 

minion name:

----------

          ID: copy_package

    Function: file.managed

        Name: /tmp/abc.tar.gz

      Result: False

     Comment: Unable to manage file: Could not fetch from s3://com.abc.bucket/ubuntu/abc.tar.gz

     Started: 18:05:14.837937

    Duration: 1339.16 ms

     Changes:

Minion Log:

[INFO    ] Starting new HTTPS connection (1): com.abc.bucket.s3-us-west-2.amazonaws.com

[DEBUG   ] Setting read timeout to None

[DEBUG   ] "GET /ubuntu/abc.tar.gz HTTP/1.1" 200 423282764

[DEBUG   ] S3 Response Status Code: 200

[DEBUG   ] Saving to local file: /var/cache/salt/minion/extrn_files/base/com.abc.bucket/ubuntu/abc.tar.gz

[ERROR   ] File sum set for file /tmp/gd4.4.1.150623.0044_install_u.tar.gz of s3://com.abc.bucket/ubuntu/abc.tar.gz.MD5 does not match real sum of 37215df030b61b3fde6a69d987a791f2

[INFO    ] Completed state [/tmp/gd4.4.1.150623.0044_install_u.tar.gz] at time 18:05:41.056426

[DEBUG   ] File /var/cache/salt/minion/accumulator/140522782184720 does not exist, no need to cleanup.

[INFO    ] Returning information for job: 20150707180506075002

The file gets downloaded at a different location -  /var/cache/salt/minion/extrn_files/base/com.abc.bucket/ubuntu/ instead of copying in /tmp

Can anyone please help me out. Give correct configuration for downloading the files from S3 bucket.

Salt Version - salt 2014.7.5 (Helium)

Any kind of help is appreciated.

Thanks,

Janardhan

[Colton Myers]

Janardhan,

Salt doesn't immediately download files to the destination location. It downloads the files to a cache directory so it can compare hashes and make sure everything is in order, before copying it into place. This process is failing at the hash comparison point. Did you double check that your hash does match the file at that URL?

Hope that helps!

--

Colton Myers

Platform Engineer, SaltStack

@basepi on Twitter/Github/IRC

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jcrimes]

Hi,

same issue here. See md5 is correct. Did anyone get it to work?

I am using salt-call 2015.5.8 (Lithium). And looks to me that it should be solved in my version: https://github.com/saltstack/salt/issues/15209

   file.managed:
    - user: root
    - group: root
    - mode: 644
    - source: s3://{{ mybucket }}/{{ myproduct }}
    - source_hash: md5=s3://{{ mybucket }}/{{ myproduct }}.MD5
    - backup: minion
    - force: True

    Function: file.managed

      Result: False

     Comment: File sum set for file <> of s3://<>.md5 does not match real sum of 094a537101026371daacce2018912d2c

     Started: 16:39:16.589472

    Duration: 463.134 ms

     Changes: