Anonymous S3 requests

[Thomas O'Dowd]

Hi all,

Is there any way with boto to send S3 requests anonymously? Ie, if a
bucket allows public-read-write, I would like to be able to PUT an
object using the boto api and get a bucket listing etc. If I leave the
access key etc undefined as None I just get an exception.

If there isn't currently any way to do this, where would be the best
place to change the code? I can make a patch if you point me in the
right direction.

Tom.
--
Gemini Mobile Technologies - http://geminimobile.com/
S3 REST API Compliant Cloud Storage with Cloudian™

[Mitchell Garnaat]

Hmm.  Interesting question.  It's kind of baked in to boto to expect credentials but this is an excellent example of where they are not, or at least should not, be required.

The first problem is that the HmacKeys class in boto/auth.py is explicitly looking for credentials and raising an exception if they aren't there.  So, that needs to be reworked.  Are anonymous requests possible in some other services?  All other services?  I'm pretty sure that SQS supports policies that enable anonymous reads and writes to queues.  Are there other examples?  I'm just wondering if the assumption that you need credentials is universally wrong or just wrong in a couple of specific cases.

Once that is figured out, I think some additional checking would be required in several places.  For example, we create an HMAC using the secret access key as the key.  Passing None will cause an exception so that needs to be sorted out.  And then, we need to disable the entire signature computation for the case of anonymous requests.

I think it's all doable and it makes sense to do it.  It just might be a bit ugly and a little tricky.  I'm willing to help.

Mitch

--
You received this message because you are subscribed to the Google Groups "boto-users" group.
To post to this group, send email to boto-...@googlegroups.com.
To unsubscribe from this group, send email to boto-users+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/boto-users?hl=en.

[Thomas O'Dowd]

Hi Mitch,

I'm not really sure about Amazon services other than S3 for the
anonymous user. As I find time over the next few days I'm thinking of
working on this as it would be useful to me.

If anyone has further input keep the thread going :-)

Tom.

> +unsub...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/boto-users?hl=en.
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "boto-users" group.
> To post to this group, send email to boto-...@googlegroups.com.
> To unsubscribe from this group, send email to boto-users

> +unsub...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/boto-users?hl=en.

--

[Thomas O'Dowd]

Mitch,

Just thinking out loud here... I'm guessing the best way to do an
anonymous request api wise is to add a parameter of some kind to the
s3connection object constructor. If set, that connecton would ignore any
credentials etc set in the environment/config. What do you think?

Tom.

[Michael Schwartz]

I wrote a no-op boto auth plugin that allows anonymous requests, for gsutil. The way we use it in gsutil is if a user hasn't set up credentials (via the gsutil config command) all requests use this plugin; once they set up credentials requests use the credentials.

I can't speak for how this would work beyond the boto storage interfaces; but we use it for Google Cloud Storage. If folks are interested I'd be willing to move this plugin from its home in the gsutil source repo (http://code.google.com/p/gsutil/source/browse/trunk/src/gslib/no_op_auth_plugin.py) to boto.

This way would allow you to make anonymous requests without adding new params to existing boto methods, but you'd have to have some conditional logic to decide whether to load the no-op plugin, like we do in gsutil.

Mike

[Mitchell Garnaat]

That sounds like a very clean approach.  I like it.

Mitch

--

You received this message because you are subscribed to the Google Groups "boto-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/boto-users/-/fArT3lHJ-uQJ.

To post to this group, send email to boto-...@googlegroups.com.

To unsubscribe from this group, send email to boto-users+...@googlegroups.com.

[Thomas O'Dowd]

Thanks for the idea Michael.

I needed it today - so I added a handler like you suggested. Seems to
work well for my purposes. I asked Mitch to review the branch on github.

https://github.com/boto/boto/pull/481

If you have suggestions, please let me know.

Tom.

On Wed, 2012-01-18 at 07:16 -0800, Mitchell Garnaat wrote:
> That sounds like a very clean approach. I like it.
>
>
> Mitch
>
> On Wed, Jan 18, 2012 at 7:14 AM, Michael Schwartz
> <mfsch...@google.com> wrote:
> I wrote a no-op boto auth plugin that allows anonymous
> requests, for gsutil. The way we use it in gsutil is if a user
> hasn't set up credentials (via the gsutil config command) all
> requests use this plugin; once they set up credentials
> requests use the credentials.
>
>
>
> I can't speak for how this would work beyond the boto storage
> interfaces; but we use it for Google Cloud Storage. If folks
> are interested I'd be willing to move this plugin from its
> home in the gsutil source repo
> (http://code.google.com/p/gsutil/source/browse/trunk/src/gslib/no_op_auth_plugin.py) to boto.
>
>
> This way would allow you to make anonymous requests without
> adding new params to existing boto methods, but you'd have to
> have some conditional logic to decide whether to load the
> no-op plugin, like we do in gsutil.
>
>
> Mike
>
>
>
--