403 for HEAD request where GET request for same object is fine

[Shane]

During testing of a new application I found that HEAD requests fail
for an object (a small image). A GET request works fine for the same
object via browser.

I'm doing the HEAD request via 'curl --head url-to-object'

I'm using the jets3t library to put the objects to S3.

This is the 403 response I'm getting via curl:

jackal:~ shane$ curl --head
http://mybucketname.s3.amazonaws.com/7NhwEeNc/images/2_xnFa4689gXqC.jpg?AWSAccessKeyId=myaccesskeyid&Expires=1216749099&Signature=3sWDjMYHRW3nzTH%2FqXdJW00lO%2Bo%3D
[1] 17092
[2] 17093

jackal:~ shane$ HTTP/1.1 403 Forbidden
x-amz-request-id: 925B24B7FD98EC22
x-amz-id-2:
Fp8oShq6L7Zn18G4TtVkDt1TixSHrx01f3jLfe0NL9PMQa5IISLS6aCF6D7ZR3tQ
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Wed, 09 Jul 2008 23:22:00 GMT
Server: AmazonS3

Does anyone have any idea why the GET request is giving a 403?

[James Murty]

How are you generating the Signature you are using with Curl?

If the object is public (as it seems, since you can download it with a browser) there is no need to include the AWSAccessKey, Signature or other URL parameters. Just send the HEAD request to the object's URL.

If the object is private, you will need to generate a signature value for the HEAD request. This signature value will be different from the one generated for a GET request, so you cannot just take a signed GET request URL and use it to perform a HEAD request.

HTH,
James

[Shane]

James,

Thanks for the explaination. After closer inspection I realized
there's different methods for signing GET and HEAD requests.

Background surrounding my issue: A third party is attempting to use a
single URL that I provide to do both a HEAD and GET request. First a
HEAD request to verify the URL is valid and a subsequent GET request
to actually retrieve the object.

My next question is that after visually inspecting the signed URLs for
a GET and HEAD request they don't seem to be different. Here's what
I'm seeing:

Signing HEAD...

[Shane]

My last message was cut off. I have a couple more questions:

1. Why do the signatures for GET and HEAD signed URLs look the same?
2. How can I provide a single URL for private objects which will work
for both GET and HEAD requests? If this isn't possible, then I see two
other possible workarounds: setup a proxy to provide signed urls to
the third party depending on what type of request it is, or making the
URLs public so signing isn't necessary.

Thanks again,
Shane

[James Murty]

Hi Shane,

1. Why do the signatures for GET and HEAD signed URLs look the same?

I'm not sure what you mean by looking the same. All S3 signed URLs have the same parameters, but the important thing is that the values of those parameters will be very different for every signature, even signatures for the same object made milliseconds apart.
 

2. How can I provide a single URL for private objects which will work
for both GET and HEAD requests?

You can't. The S3 signature values is calculated based on the request method (GET, HEAD, DELETE etc) as well as other factors, so one signature cannot be used to perform a different operation.
 

If this isn't possible, then I see two
other possible workarounds: setup a proxy to provide signed urls to
the third party depending on what type of request it is, or making the
URLs public so signing isn't necessary.

Yes, these are both options. The JetS3t project includes a Gatekeeper servlet that acts as a signature provider to client applications, and can provide signatures appropriate for whichever task you need to perform. The trick is to write a client application that can communicate with the Gatekeeper and correctly apply the signatures it generates. See the first link below for an explanation of how the Gatekeeper works, and the second for a description of the Cockpit Lite client application that interacts with the Gatekeeper to perform all its S3-related activities.

http://jets3t.s3.amazonaws.com/applications/gatekeeper-concepts.html
http://jets3t.s3.amazonaws.com/applications/cockpitlite.html
 
Cheers,
James

[Shane]

James,

I was mistaken. The GET and HEAD signed URLs do have different values
for the signature request parameter.

In the case of Gatekeeper, I'd probably have to extend it accommodate
my requirements. I have to provide just one URL to a third party which
responds to both HEAD and GET requests.

Thanks,
Shane