S3ResponseError: 403 Forbidden with CNAME bucket

[Sebastian]

Hi all!

I have my media files on S3 in a bucket called "media.example.org". A
CNAME record points requests for media.example.org to S3.

If I try to access my bucket via boto, I get the following error:

send: 'GET https://media.example.org.s3.amazonaws.com:443/?&max-keys=0
HTTP/1.1\r\nHost: media.example.org.s3.amazonaws.com:443\r\nAccept-
Encoding: identity\r\nDate: Thu, 21 Oct 2010 13:52:04 GMT\r\nContent-
Length: 0\r\nAuthorization: AWS AKIAJURBA2V4BF5NN6XA:cSEoitXF
+3GEdL2LGKQhwA36BbE=\r\nUser-Agent: Boto/2.0b3 (linux2)\r\n\r\n'
reply: 'HTTP/1.1 307 Temporary Redirect\r\n'
header: x-amz-request-id: C39F12CF3632D9FA
header: x-amz-id-2: DbBgYqLFP/f+q6svsWsXfYercSR1Hacpixix7u/
fsSslGPEMTkr22Wo9YjNmacHX
header: Location: https://media.example.org.s3-external-3.amazonaws.com/?&max-keys=0
header: Content-Type: application/xml
header: Transfer-Encoding: chunked
header: Date: Thu, 21 Oct 2010 13:52:04 GMT
header: Server: AmazonS3
send: 'GET /?&max-keys=0 HTTP/1.1\r\nHost: s3.amazonaws.com:443\r
\nAccept-Encoding: identity\r\nDate: Thu, 21 Oct 2010 13:52:04 GMT\r
\nContent-Length: 0\r\nAuthorization: AWS AKIAJURBA2V4BF5NN6XA:cSEoitXF
+3GEdL2LGKQhwA36BbE=\r\nUser-Agent: Boto/2.0b3 (linux2)\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: x-amz-request-id: DD280BFC31B05113
header: x-amz-id-2: elN7sOHC
+5zAgKgFjISdM4PNjl9GRQsh9nRAxccBeQmf6KfGQaGRSCVYKx0oaJh5
header: Content-Type: application/xml
header: Transfer-Encoding: chunked
header: Date: Thu, 21 Oct 2010 13:52:05 GMT
header: Server: AmazonS3
Traceback (most recent call last):
File "test_boto.py", line 13, in <module>
main()
File "test_boto.py", line 10, in main
bucket = conn.get_bucket(bucket)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/connection.py",
line 325, in get_bucket
bucket.get_all_keys(headers, maxkeys=0)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/bucket.py", line
295, in get_all_keys
'', headers, **params)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/bucket.py", line
262, in _get_all
response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403
Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request
signature we calculated does not match the signature you provided.
Check your key and signing method.</Message><StringToSignBytes>47 45
54 0a 0a 0a 54 68 75 2c 20 32 31 20 4f 63 74 20 32 30 31 30 20 31 33
3a 35 32 3a 30 34 20 47 4d 54 0a 2f</
StringToSignBytes><RequestId>DD280BFC31B05113</
RequestId><HostId>elN7sOHC
+5zAgKgFjISdM4PNjl9GRQsh9nRAxccBeQmf6KfGQaGRSCVYKx0oaJh5</
HostId><SignatureProvided>cSEoitXF+3GEdL2LGKQhwA36BbE=</
SignatureProvided><StringToSign>GET


Thu, 21 Oct 2010 13:52:04 GMT
/</StringToSign><AWSAccessKeyId>AKIAJURBA2V4BF5NN6XA</AWSAccessKeyId></
Error>

Can someone please point me in the right direction?

Bheers,
Sebastian

[Sebastian]

> Hi all!
>
> I have my media files on S3 in a bucket called "media.example.org". A
> CNAME record points requests for media.example.org to S3.
>
> If I try to access my bucket via boto, I get the following error:
>

> send: 'GEThttps://media.example.org.s3.amazonaws.com:443/?&max-keys=0

I've come a step closer to a solution. When executing the following
code (which I forgot to include in my previous e-mail)

conn = boto.connect_s3(ACCESS_KEY, SECRET_KEY, debug=2)
bucket = conn.get_bucket('media.example.org')

I get this traceback:

send: 'GET https://media.example.org.s3.amazonaws.com:443/?&max-keys=0
HTTP/1.1\r\nHost: media.example.org.s3.amazonaws.com:443\r\nAccept-

Encoding: identity\r\nDate: Fri, 22 Oct 2010 12:24:01 GMT\r\nContent-
Length: 0\r\nAuthorization: AWS AKIAJURBA2V4BF5NN6XA:McpSbR2p/
xQ4T0XJkROLTogaPpc=\r\nUser-Agent: Boto/2.0b3 (linux2)\r\n\r\n'

reply: 'HTTP/1.1 307 Temporary Redirect\r\n'

header: x-amz-request-id: 77D2B42169AC7C50
header: x-amz-id-2: 9LaFssmJv2w6JCwUSyAaKrvfDCeUOqmgTa4JjOVt4MyL/
QLC53vG37j6zEQyg1j9

header: Location: https://media.example.org.s3-external-3.amazonaws.com/?&max-keys=0
header: Content-Type: application/xml
header: Transfer-Encoding: chunked

header: Date: Fri, 22 Oct 2010 12:23:58 GMT

header: Server: AmazonS3
send: 'GET /?&max-keys=0 HTTP/1.1\r\nHost: s3.amazonaws.com:443\r

\nAccept-Encoding: identity\r\nDate: Fri, 22 Oct 2010 12:24:01 GMT\r

\nContent-Length: 0\r\nAuthorization: AWS

AKIAJURBA2V4BF5NN6XA:McpSbR2p/xQ4T0XJkROLTogaPpc=\r\nUser-Agent: Boto/

2.0b3 (linux2)\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'

header: x-amz-request-id: 4EC1E3B7863C9B74
header: x-amz-id-2: U6Z
+l6fh9iD4qCN4iX9iUwPHEDmhtM1Tf6PQj84eFtS0r9rEmBXTWPhGLf9UaVZ2

header: Content-Type: application/xml
header: Transfer-Encoding: chunked

header: Date: Fri, 22 Oct 2010 12:23:58 GMT

header: Server: AmazonS3
Traceback (most recent call last):
File "test_boto.py", line 13, in <module>
main()
File "test_boto.py", line 10, in main
bucket = conn.get_bucket(bucket)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/connection.py",
line 325, in get_bucket
bucket.get_all_keys(headers, maxkeys=0)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/bucket.py", line
295, in get_all_keys
'', headers, **params)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/bucket.py", line
262, in _get_all
response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request
signature we calculated does not match the signature you provided.
Check your key and signing method.</Message><StringToSignBytes>47 45

54 0a 0a 0a 46 72 69 2c 20 32 32 20 4f 63 74 20 32 30 31 30 20 31 32
3a 32 34 3a 30 31 20 47 4d 54 0a 2f</
StringToSignBytes><RequestId>4EC1E3B7863C9B74</RequestId><HostId>U6Z
+l6fh9iD4qCN4iX9iUwPHEDmhtM1Tf6PQj84eFtS0r9rEmBXTWPhGLf9UaVZ2</
HostId><SignatureProvided>McpSbR2p/xQ4T0XJkROLTogaPpc=</
SignatureProvided><StringToSign>GET


Fri, 22 Oct 2010 12:24:01 GMT

/</StringToSign><AWSAccessKeyId>AKIAJURBA2V4BF5NN6XA</AWSAccessKeyId></
Error>

Apparently S3 expects the request to submitted again to
https://media.example.org.s3-external-3.amazonaws.com/?&max-keys=0.
Since the signature was calculated for media.example.org, it doesn't
match anymore.

Sebastian

[Mitchell Garnaat]

Hi -

Try doing this:

   from boto.s3.connection import OrdinaryCallingFormat

   conn = boto.connect_s3(ACCESS_KEY, SECRET_KEY, debug=2, calling_format=OrdinaryCallingFormat())

   bucket = conn.get_bucket('media.example.org')

What is the result?

Mitch

Sebastian

--
You received this message because you are subscribed to the Google Groups "boto-users" group.
To post to this group, send email to boto-...@googlegroups.com.
To unsubscribe from this group, send email to boto-users+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/boto-users?hl=en.

[Mitchell Garnaat]

Hmm.  I just tried a similar exercise and it works for me.  Here's the output:

In [1]: import boto

In [2]: boto.set_stream_logger('boto')

In [3]: c = boto.connect_s3(debug=2)

In [4]: c.get_bucket('images.cloudright.com')

2010-10-22 11:07:02,433 boto [DEBUG]:Canonical: GET

Fri, 22 Oct 2010 15:07:02 GMT

/images.cloudright.com/

2010-10-22 11:07:02,433 boto [DEBUG]:Method: GET

2010-10-22 11:07:02,435 boto [DEBUG]:Path: /?&max-keys=0

2010-10-22 11:07:02,435 boto [DEBUG]:Data: 

2010-10-22 11:07:02,435 boto [DEBUG]:Headers: {'Date': 'Fri, 22 Oct 2010 15:07:02 GMT', 'Content-Length': '0', 'Authorization': 'AWS <omitted>', 'User-Agent': 'Boto/2.0b3 (darwin)'}

2010-10-22 11:07:02,435 boto [DEBUG]:Host: images.cloudright.com.s3.amazonaws.com:443

2010-10-22 11:07:02,435 boto [DEBUG]:establishing HTTP connection

send: 'GET /?&max-keys=0 HTTP/1.1\r\nHost: images.cloudright.com.s3.amazonaws.com:443\r\nAccept-Encoding: identity\r\nDate: Fri, 22 Oct 2010 15:07:02 GMT\r\nContent-Length: 0\r\nAuthorization: AWS 0<omitted>\r\nUser-Agent: Boto/2.0b3 (darwin)\r\n\r\n'

reply: 'HTTP/1.1 200 OK\r\n'

header: x-amz-id-2: p9QngNlHMs2lHbWWPOyNEn0wcW3YAEFgwDp/Mru/telfpNAi9BMemjZSDZZWeXHE

header: x-amz-request-id: 1CBDE57590229FE3

header: Date: Fri, 22 Oct 2010 15:07:03 GMT

header: Content-Type: application/xml

header: Transfer-Encoding: chunked

header: Server: AmazonS3

2010-10-22 11:07:02,736 boto [DEBUG]:<?xml version="1.0" encoding="UTF-8"?>

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>images.cloudright.com</Name><Prefix></Prefix><Marker></Marker><MaxKeys>0</MaxKeys><IsTruncated>false</IsTruncated></ListBucketResult>

Out[4]: <Bucket: images.cloudright.com>

[Sebastian]

Hi Mitch,

when I try this, I get the following error:

send: 'GET https://s3.amazonaws.com:443/media.example.org/?&max-keys=0

HTTP/1.1\r\nHost: s3.amazonaws.com:443\r\nAccept-Encoding: identity\r

\nDate: Tue, 26 Oct 2010 08:53:45 GMT\r\nContent-Length: 0\r
\nAuthorization: AWS AKIAJURBA2V4BF5NN6XA:5dvgtuEwZKGWCk4izGj1beubXtY=

\r\nUser-Agent: Boto/2.0b3 (linux2)\r\n\r\n'

reply: 'HTTP/1.1 301 Moved Permanently\r\n'
header: x-amz-request-id: D674291E5D35BF5C
header: x-amz-id-2:
4ju7E0FMSDobtHEouD1UdTYQnd6X96NjVDZ8J4NtyIUea6A07jboHk9bKunru659

header: Content-Type: application/xml
header: Transfer-Encoding: chunked

header: Date: Tue, 26 Oct 2010 08:53:44 GMT

header: Server: AmazonS3
Traceback (most recent call last):

File "test_boto.py", line 15, in <module>
main()
File "test_boto.py", line 12, in main

bucket = conn.get_bucket(bucket)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/connection.py",
line 325, in get_bucket
bucket.get_all_keys(headers, maxkeys=0)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/bucket.py", line
295, in get_all_keys
'', headers, **params)
File "/home/basti/.virtualenvs/4f/src/boto/boto/s3/bucket.py", line
262, in _get_all
response.status, response.reason, body)

boto.exception.S3ResponseError: S3ResponseError: 301 Moved Permanently

<?xml version="1.0" encoding="UTF-8"?>

<Error><Code>PermanentRedirect</Code><Message>The bucket you are
attempting to access must be addressed using the specified endpoint.
Please send all future requests to this endpoint.</
Message><RequestId>D674291E5D35BF5C</
RequestId><Bucket>media.example.org</
Bucket><HostId>4ju7E0FMSDobtHEouD1UdTYQnd6X96NjVDZ8J4NtyIUea6A07jboHk9bKunru659</
HostId><Endpoint>media.example.org.s3.amazonaws.com</Endpoint></Error>

Seems I need to specify a different endpoint, whatever that means.
Sebastian

> >> HTTP/1.1\r\nHost<https://media.example.org.s3.amazonaws.com:443/?&max-keys=0HTTP/1.1%5...>:

> >> boto-users+...@googlegroups.com<boto-users%2Bunsu...@googlegroups.com>
> >> .