Boto, S3, and leading slashes in key paths

[Jason Chu]

I searched through boto-users, boto-dev, and the issues and didn't
find any references to this problem.

I use s3sync (http://www.s3sync.net/wiki) to sync some files to s3.
The keys it creates start with / (eg. /wals/db2/...).

I'm able to choose a bucket and list keys but when I try to delete a
key that starts with a leading / the request seems to be wrong. I
turned on debugging and saw the DELETE request was going to a url like
this: /mybucket/wals/db2/2010-09-06-030002/00000004000003AA0000004D.
If the key starts with '/' shouldn't the url be /mybucket//wals/
db2/2010-09-06-030002/00000004000003AA0000004D?

I looked into the code a bit and, in boto/connection.py, I found
AWSConnection.get_path. That seems to be removing "extra" slashes
that, in my case, are very important. Shouldn't keys be totally
untouched when it comes to url creation? Even though they may look
like urls, they're really just opaque strings.

Has anyone else ever run into this? It's a little annoying when the
result of a bucket.list() can't be passed directly into a
bucket.delete_key(). Should this be posted in the issue tracker
(google code or github?)?

Jason

[Mitchell Garnaat]

Hi -

I need to look into this a bit more carefully but, yes, you really should create an issue (we are using googlecode for that at the moment) to track this.

Thanks,

Mitch

--
You received this message because you are subscribed to the Google Groups "boto-users" group.
To post to this group, send email to boto-...@googlegroups.com.
To unsubscribe from this group, send email to boto-users+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/boto-users?hl=en.

[Jason Chu]

http://code.google.com/p/boto/issues/detail?id=454

Sadly, I don't think I can even find a workaround...

> > boto-users+...@googlegroups.com<boto-users%2Bunsubscribe@googlegrou ps.com>
> > .

[Robert Coup]

Hi Jason,

On Wed, Sep 22, 2010 at 6:02 AM, Jason Chu <xen...@gmail.com> wrote:

I use s3sync (http://www.s3sync.net/wiki) to sync some files to s3.
The keys it creates start with / (eg. /wals/db2/...).

The key you *ask* it to use starts with /, it doesn't force or recommend it...

s3sync local_path/db2/ mybucket:wals/db2/ (no leading /)

vs

s3sync local_path/db2/ mybucket:/wals/db2/ (what you're using?)

 

I'm able to choose a bucket and list keys but when I try to delete a
key that starts with a leading / the request seems to be wrong.  I
turned on debugging and saw the DELETE request was going to a url like
this: /mybucket/wals/db2/2010-09-06-030002/00000004000003AA0000004D.
If the key starts with '/' shouldn't the url be /mybucket//wals/
db2/2010-09-06-030002/00000004000003AA0000004D?

Hmm, this does look like a problem...

Rob :)