SignatureDoesNotMatch error when uploading to s3 fog works fine

[Tarik Ansari]

Hi,

I have a very odd issue with CarrierWave and S3, I have a form
uploading an image to S3, works fine on my development machine but I
cannot get it to work on Heroku.

I've used the Heroku console and run a test upload to S3 with the same
credentials used by my app and everything works fine, but once handled
by CarrierWave, I get a 403 error:



UsersController# (Excon::Errors::Forbidden) "Expected(200) Actual(403
Forbidden)\n reqMore

/app/.bundle/gems/ruby/1.9.1/gems/excon-0.6.3/lib/excon/connection.rb:
179:in `request'
/app/.bundle/gems/ruby/1.9.1/gems/fog-0.8.2/lib/fog/core/connection.rb:
20:in `request'
/app/.bundle/gems/ruby/1.9.1/gems/fog-0.8.2/lib/fog/storage/aws.rb:
329:in `request'
/app/.bundle/gems/ruby/1.9.1/gems/fog-0.8.2/lib/fog/storage/requests/
aws/put_object.rb:43:in `put_object'
/app/.bundle/gems/ruby/1.9.1/gems/fog-0.8.2/lib/fog/storage/models/aws/
file.rb:119:in `save'
/app/.bundle/gems/ruby/1.9.1/gems/fog-0.8.2/lib/fog/core/collection.rb:
50:in `create'
/app/.bundle/gems/ruby/1.9.1/gems/carrierwave-0.5.4/lib/carrierwave/
storage/fog.rb:229:in `store'
/app/.bundle/gems/ruby/1.9.1/gems/carrierwave-0.5.4/lib/carrierwave/
storage/fog.rb:79:in `store!'




My configuration is as follow: initializers/carrierwave.rb:

# CarrierWave S3 config
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => SETTINGS[Rails.env]['s3']
['access_key_id'], # required
:aws_secret_access_key => SETTINGS[Rails.env]['s3']
['secret_access_key'], # required
#:region => 'eu-west-1' # optional, defaults to
'us-east-1'
}
config.fog_directory = SETTINGS[Rails.env]
['bucket'] # required
#config.fog_host = 's3.amazonaws.com' # optional,
defaults to nil
#config.fog_public = false #
optional, defaults to true
#config.fog_attributes = {'Cache-Control'=>'max-age=315576000'} #
optional, defaults to {}
end





picture_uploader.rb:

class PictureUploader < CarrierWave::Uploader::Base

# Include RMagick or ImageScience support:
include CarrierWave::MiniMagick
# include CarrierWave::ImageScience

# Choose what kind of storage to use for this uploader:
storage :fog

# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be
mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end

# fix for Heroku, unfortunately, it disables caching, see:
https://github.com/jnicklas/carrierwave/wiki/How-to%3A-Make-Carrierwave-work-on-Heroku
def cache_dir
"#{Rails.root}/tmp/uploads"
end

# Create different versions of your uploaded files:
version :thumb do
process :resize_to_fill => [96, 96]
end

version :mini do
process :resize_to_fill => [48, 48]
end

end






I've been dealing with this issue since yesterday night trying to test
everything...

[Trevor Turk]

You should post more info, but maybe this could help:

https://groups.google.com/forum/#!topic/ruby-fog/kaehIbyr3kk

I had some problems with Heroku that were resolved with this hack. 

[Tarik Ansari]

I've activated logging, and reproduced the error (happens every time), but no logs got generated on the bucket.. so I would not suspect a clock issue...?

Here are the request details that I get in Exceptional (I've replaced certain strings with "[hidden]"):

Expected(200) <=> Actual(403 Forbidden) request => {:headers=>{"Content-Length"=>26292, "x-amz-acl"=>"public-read", "Content-Type"=>"image/jpeg", "Date"=>"Fri, 10 Jun 2011 04:28:55 +0000", "Authorization"=>"AWS AKIAII5L2KWAARSQ2VHA:NMLcDqlzdiABhjC084Hgc2emYi4=", "Host"=>"com:443"}, :host=>"com", :mock=>nil, :path=>"/uploads%2Fuser%2Fpicture%2F14%2Ftarik_fencing.jpg", :port=>"443", :query=>nil, :scheme=>"https", :body=>"[lots of encoded binary data]", :expects=>200, :idempotent=>true, :method=>"PUT"} response => #<Excon::Response:0x000000047e6f68 @body="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><StringToSignBytes>50 55 54 0a 0a 69 6d 61 67 65 2f 6a 70 65 67 0a 46 72 69 2c 20 31 30 20 4a 75 6e 20 32 30 31 31 20 30 34 3a 32 38 3a 35 35 20 2b 30 30 30 30 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 75 62 6c 69 63 2d 72 65 61 64 0a 2f 63 6f 6d 2f 75 70 6c 6f 61 64 73 25 32 46 75 73 65 72 25 32 46 70 69 63 74 75 72 65 25 32 46 31 34 25 32 46 74 61 72 69 6b 5f 66 65 6e 63 69 6e 67 2e 6a 70 67</StringToSignBytes><RequestId>959CE18098F2189C</RequestId><HostId>lFequVVvCpr+/36MtBrHRdvTb0ra1Q1wIhjEsBR7cmxKLEvdILYSJ2yote+I3wCk</HostId><SignatureProvided>NMLcDqlzdiABhjC084Hgc2emYi4=</SignatureProvided><StringToSign>PUT\n\nimage/jpeg\nFri, 10 Jun 2011 04:28:55 +0000\nx-amz-acl:public-read\n/com/uploads%2Fuser%2Fpicture%2F14%2Ftarik_fencing.jpg</StringToSign><AWSAccessKeyId>[hidden]</AWSAccessKeyId></Error>", @headers={"x-amz-request-id"=>"959CE18098F2189C", "x-amz-id-2"=>"lFequVVvCpr+/36MtBrHRdvTb0ra1Q1wIhjEsBR7cmxKLEvdILYSJ2yote+I3wCk", "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked", "Date"=>"Fri, 10 Jun 2011 04:28:55 GMT", "nnCoection"=>"close", "Server"=>"AmazonS3"}, @status=403>

Time

10 Jun at 05:28AM

URL

POST http://staging.mojo.co/account/signup/12345678

Params

{ "utf8" : "✓", "authenticity_token" : "[hidden]", "user" : { "min_height" : { "inches" : "", "feet" : "", }"max_height" : { "inches" : "", "feet" : "", } "min_age" : "", "dob(1i)" : "1986", "dob(2i)" : "9", "password_confirmation" : "[hidden]", "dob(3i)" :"12", "filter_age" : "0", "picture_cache" : "", "sex" : "male", "last_name" : "Ansari", "height" : { "inches" : "", "feet" : "", } "filter_height" :"0", "cellphone" : "[hidden]", "max_age" : "", "sex_preference" : "female", "password" : "[FILTERED]", "email" : "ta...@mojo.co","invitation_token" : "12345678", "first_name" : "Tarik", "picture" : { "name" : "user[picture]", "head" : "Content-Disposition: form-data; name="user[picture]"; filename="tarik_fencing.jpg" Content-Type: image/jpeg ", "type" : "image/jpeg", "filename" : "tarik_fencing.jpg", "tempfile" :"#<File:0x000000053ddf10>", } } }

Referer

http://staging.mojo.co/account/signup/12345678

User Agent

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.91 Safari/534.30

[geemus]

Is the bucket you are trying to access somewhere other than us-
east-1? I think it is probably following redirects but that it is
destructive to params[:host] (hence why :host ends up being 'com'). I
will fix this, but in the mean time you should set the region to match
the bucket location (if it isn't). Let me know if that doesn't help
and I can try something else, but it seems the likely culprit).
Thanks!
wes

On Jun 9, 9:50 pm, Tarik Ansari <ta...@mojo.co> wrote:
> I've activated logging, and reproduced the error (happens every time), but
> no logs got generated on the bucket.. so I would not suspect a clock
> issue...?
>
> Here are the request details that I get in Exceptional (I've replaced
> certain strings with "[hidden]"):
>
> Expected(200) <=> Actual(403 Forbidden) request =>
> {:headers=>{"Content-Length"=>26292, "x-amz-acl"=>"public-read",
> "Content-Type"=>"image/jpeg", "Date"=>"Fri, 10 Jun 2011 04:28:55 +0000",
> "Authorization"=>"AWS AKIAII5L2KWAARSQ2VHA:NMLcDqlzdiABhjC084Hgc2emYi4=",
> "Host"=>"com:443"}, :host=>"com", :mock=>nil,
> :path=>"/uploads%2Fuser%2Fpicture%2F14%2Ftarik_fencing.jpg", :port=>"443",
> :query=>nil, :scheme=>"https", :body=>"[lots of encoded binary data]",
> :expects=>200, :idempotent=>true, :method=>"PUT"} response =>
> #<Excon::Response:0x000000047e6f68 @body="<?xml version=\"1.0\"
> encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The
> request signature we calculated does not match the signature you provided.
> Check your key and signing method.</Message><StringToSignBytes>50 55 54 0a
> 0a 69 6d 61 67 65 2f 6a 70 65 67 0a 46 72 69 2c 20 31 30 20 4a 75 6e 20 32
> 30 31 31 20 30 34 3a 32 38 3a 35 35 20 2b 30 30 30 30 0a 78 2d 61 6d 7a 2d
> 61 63 6c 3a 70 75 62 6c 69 63 2d 72 65 61 64 0a 2f 63 6f 6d 2f 75 70 6c 6f
> 61 64 73 25 32 46 75 73 65 72 25 32 46 70 69 63 74 75 72 65 25 32 46 31 34
> 25 32 46 74 61 72 69 6b 5f 66 65 6e 63 69 6e 67 2e 6a 70

> 67</StringToSignBytes><RequestId>959CE18098F2189C</RequestId><HostId>lFequV VvCpr+/36MtBrHRdvTb0ra1Q1wIhjEsBR7cmxKLEvdILYSJ2yote+I3wCk</HostId><Signatu reProvided>NMLcDqlzdiABhjC084Hgc2emYi4=</SignatureProvided><StringToSign>PU T\n\nimage/jpeg\nFri,

> 10 Jun 2011 04:28:55

> +0000\nx-amz-acl:public-read\n/com/uploads%2Fuser%2Fpicture%2F14%2Ftarik_fe ncing.jpg</StringToSign><AWSAccessKeyId>[hidden]</AWSAccessKeyId></Error>",

> @headers={"x-amz-request-id"=>"959CE18098F2189C",

> "x-amz-id-2"=>"lFequVVvCpr+/36MtBrHRdvTb0ra1Q1wIhjEsBR7cmxKLEvdILYSJ2yote+I 3wCk",

> "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked",
> "Date"=>"Fri, 10 Jun 2011 04:28:55 GMT", "nnCoection"=>"close",
> "Server"=>"AmazonS3"}, @status=403>Time
>
> 10 Jun at 05:28AM
> URL
>

> POSThttp://staging.mojo.co/account/signup/12345678

[Tarik Ansari]

Found the issue :) I had to reset the heroku instance after bundler
issues, and forgot to re-set
heroku config:add RACK_ENV=staging

So it was looking at the bucket of the production environnement in the
configuration which was not set...

The funny thing is that the error I got when testing a wrong bucket
was:
UsersController# (Excon::Errors::MovedPermanently) "Expected(200) <=>
Actual(301 Moved Permanently)

However when not set it seems to be 403 instead, so I didn't expect
the bucket to be the culprit.

Thank you both for the help and sorry for the trouble! -- PS: region
is us-east-1