SignatureDoesNotMatch error on certain save calls

[Karl]

For some reason, specific objects are givning a SignatureDoesNotMatch
error when inserting into simpledb. Pretty sure this does not have
anything to do with my access key as a call to save for a different
object will work.

Aws::AwsError: SignatureDoesNotMatch: The request signature we
calculated does not match the signature you provided. Check your AWS
Secret Access Key and signing method. Consult the service
documentation for details.
REQUEST(sdb.amazonaws.com:443/)
from /usr/lib64/ruby/gems/1.8/gems/aws-2.1.12/lib/awsbase/
right_awsbase.rb:389:in `request_info_impl'
from /usr/lib64/ruby/gems/1.8/gems/aws-2.1.12/lib/sdb/
right_sdb_interface.rb:144:in `request_info'
from /usr/lib64/ruby/gems/1.8/gems/aws-2.1.12/lib/sdb/
right_sdb_interface.rb:415:in `put_attributes'
from /usr/lib64/ruby/gems/1.8/gems/aws-2.1.12/lib/sdb/active_sdb.rb:
816:in `save'
from /usr/lib64/ruby/gems/1.8/gems/simple_record-1.1.65/lib/
simple_record.rb:343:in `save'
from /data/weshop/releases/20100118163255/app/models/
simpledb_message.rb:13:in `create_from_email_message!'
from /data/weshop/releases/20100118163255/app/models/email/message.rb:
112:in `store_on_s3'
from (irb):28
from :0

Has anyone come across this?

thx.

-karl

[Karl Baum]

Ok.  I've drilled down a bit and it seems to be related to storing this value:

"other/2009-11-10/04/84.eml"

Can simpledb handle the '/'?

thx

-karl

[Travis Reeder]

I just created a test saving this value and it seems to work fine. Can you write a test case to reproduce this?

Travis

--
You received this message because you are subscribed to the Google Groups "SimpleRecord" group.
To post to this group, send email to simple...@googlegroups.com.
To unsubscribe from this group, send email to simple-recor...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/simple-record?hl=en.

[Karl Baum]

Hi Travis.  I was off with what i thought the problem was.  The issue seems to happen whenever we do a post because the  service_params are greater than 2000 params.  Within the right_sdb_interface, the following logic is within the generate_request method:

     if service_params.size > 2000

        if signature_version == '2'

          # resign the request because HTTP verb is included into signature

          service_params = signed_service_params(@aws_secret_access_key, service_hash, :post, @params[:server], service)

        end

        request      = Net::HTTP::Post.new(service)

        request.body = service_params

        request['Content-Type'] = 'application/x-www-form-urlencoded'

      else

        request = Net::HTTP::Get.new("#{service}?#{service_params}")

      end

Whenever the service_params size is greater than 2000 characters, I get the AwsError.  I will try and reproduce with a test case.

Thanks for your help.

-karl

[Travis Reeder]

So you're saying the signature is invalid when it's using that Net::HTTP::Post and it is valid when it uses the Net::HTTP::Get? Interesting.

If you could throw a quick test together, that would be great, then I'll get it fixed up.

[Karl Baum]

Great.  Working on an rspec test as we speak.

[Karl Baum]

Hi Travis.  I reproduced the problem here:

https://gist.github.com/750c98ab32ea216d01b4

I think the problem is the combination of 2000 or more characters and the funny character after Karen Smith in the subject.  When I replaced it with a normal "apostrophe s" instead of the wierd character, the test passes.  I don't pretend to understand the ins and outs of character encodings, but is this something that I should be scrubbing on my end?

Thanks for your help.

-karl

[Karl Baum]

It looks like this fix addresses the problem i am having:

http://code.google.com/p/ruby-aws/issues/detail?id=1

Only issue is that for these objects, the text within simpledb is url encoded.  In my tests I have been just calling CGI::unescape on the value when i pull it back from simpledb, but I don't see why we wouldn't do that for all strings within simple_record or aws.

I tried to fork aws and apply the change, but github seems to be having issues.

thx.

-karl

[Travis Reeder]

This sounds eerily similar to this issue that someone else just posted here for the ec2 library: http://github.com/appoxy/aws/issues#issue/3

Travis

[Travis Reeder]

SDB should be able to support unicode, so there is some double escaping if also escape it which probably isn't what you want.

I think I might know the issue, after scanning the code, perhaps when doing a POST, we do not need to URL escape the values as we're doing when it's a GET. See line 82 in right_awsbase.rb. So when it switches to POST (because the request is greater than 2000), it might need to handle the values differently.

Any chance you could turn your test into a regular unit test so it fits with the rest of our test cases?

Travis

[Karl Baum]

Sure.  I'll have a stab at it.  Do you want the unit test in simple_record or aws?

[Travis Reeder]

Hmmm, probably would make more sense in aws. Thanks!

Travis