Support for legacy S3 bucket names with underscores

[Ben Hutchison]

I have several existing S3 buckets with alot of data inside whose
names have_underscores_in_them. This was previously allowed by amazon,
and while it now seem "deprecated", as renaming buckets isnt
supported, I am stuck with those names.

I would like to able able to access them via S3QL. Therefore, can
there be some switch to allow legacy bucket names added? Right now,
there is a regex check, about line 135 of s3.py, that rejects my
attempts to access my buckets. Which is a pity because they are
actually valid buckets.

Also, the Typical Usage example docs at [http://code.google.com/p/
s3ql/] uses underscores, even though S3QL will not allow this, which
is a bit confusing...

eg "mount.s3ql s3://nikratio_s3ql_bucket /mnt/s3ql"

[Nikolaus Rath]

This seems reasonable. Probably only mkfs.s3ql should do the regexp
check, while mount.s3ql should try to work with whatever it gets. Could
you file this as a bug report?

Thanks!

-Nikolaus

--
�Time flies like an arrow, fruit flies like a Banana.�

PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

[Nikolaus Rath]

On 10/16/2010 10:23 PM, Ben Hutchison wrote:

> I have several existing S3 buckets with alot of data inside whose
> names have_underscores_in_them. This was previously allowed by amazon,
> and while it now seem "deprecated", as renaming buckets isnt
> supported, I am stuck with those names.

On second thought: you are aware that S3QL only works with buckets that
have been initialized with mkfs.s3ql, right? You will *not* be able to
access any data that you put into the bucket without using S3QL.

In other words: S3QL stores its data in S3 buckets, but it is not a
general interface to S3 that would allow you to work with any data that
happens to be in a bucket.

Best,

[Srikanth V]

Hi

I have the same problem. I have an S3 bucket created long back, but through S3QL only. It has underscore. Because of that I am unable to upgrade S3QL to the latest version. Could you please help me in telling which python code I should change to accommodate underscore in S3 bucket name?

Thanks

Srikanth 

On Sunday, October 17, 2010 9:46:52 PM UTC+5:30, Nikolaus Rath wrote:

On 10/16/2010 10:23 PM, Ben Hutchison wrote:
> I have several existing S3 buckets with alot of data inside whose
> names have_underscores_in_them. This was previously allowed by amazon,
> and while it now seem "deprecated", as renaming buckets isnt
> supported, I am stuck with those names.

On second thought: you are aware that S3QL only works with buckets that
have been initialized with mkfs.s3ql, right? You will *not* be able to
access any data that you put into the bucket without using S3QL.

In other words: S3QL stores its data in S3 buckets, but it is not a
general interface to S3 that would allow you to work with any data that
happens to be in a bucket.

Best,

   -Nikolaus

--
 ï¿½Time flies like an arrow, fruit flies like a Banana.�

[Nikolaus Rath]

Srikanth V <vi.sr...@gmail.com> writes:
> Hi
>
> I have the same problem. I have an S3 bucket created long back, but through
> S3QL only. It has underscore. Because of that I am unable to upgrade S3QL
> to the latest version. Could you please help me in telling which python
> code I should change to accommodate underscore in S3 bucket name?

Sure, here's a patch for S3QL 2.4:

diff --git a/src/s3ql/backends/s3.py b/src/s3ql/backends/s3.py
--- a/src/s3ql/backends/s3.py
+++ b/src/s3ql/backends/s3.py
@@ -46,7 +46,7 @@
bucket_name = hit.group(1)

# http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/BucketRestrictions.html
- if not re.match('^[a-z0-9][a-z0-9.-]{1,60}[a-z0-9]$', bucket_name):
+ if not re.match('^[a-z0-9][a-z0-9._-]{1,60}[a-z0-9]$', bucket_name):
raise QuietError('Invalid bucket name.')

# Dots in the bucket cause problems with SSL certificate validation,


Best,

-Nikolaus

--
»Time flies like an arrow, fruit flies like a Banana.«

[Srikanth V]

Thanks, Nikolaus.

- Srikanth