Double-Secret security credentials

[shlomo....@gmail.com]

Hi David,

You've probably seen Mitch's article:
http://www.elastician.com/2009/06/managing-your-aws-credentials-part-2.html

Inspired by that article, I would like to request a feature in
lifeguard to allow the PoolManager to use two different credentials:
- the Secret Credentials, which it would pass to the service instances
via the user-data
- the Double Secret Credentials, which it would use for itself,
allowing it to launch/terminate EC2 instances (in addition to
accessing the necessary SQS queues).

What do you think?

.. Shlomo

[David Kavanagh]

Shlomo, that sounds reasonable.Seems like an easy add also. I need to
refactor some code at work today (moving to JPA). Let me look at this
afterwards.

David

[David]

I'm going to get this in there today, along with the ability to
specify a eucalyptus cluster (instead of EC2). (will require
Eucalyptus 1.5.2)

On Jun 25, 9:12 am, David Kavanagh <dkavan...@gmail.com> wrote:
> Shlomo, that sounds reasonable.Seems like an easy add also. I need to
> refactor some code at work today (moving to JPA). Let me look at this
> afterwards.
>
> David
>
>
>
> On Thu, Jun 25, 2009 at 6:27 AM, <shlomo.swid...@gmail.com> wrote:
>
> > Hi David,
>
> > You've probably seen Mitch's article:

> >http://www.elastician.com/2009/06/managing-your-aws-credentials-part-...

[David]

Shlomo, Would you be in a position to test the double secret support
in lifeguard? I have code I could check in, but only if you can test
fairly soon.. otherwise, I'll hold it will I can test more. I don't
want to break anybody who uses from SVN. I'll test the "regular non
double secret" mode today, then maybe I'll commit.

David

[shlomo]

I would be able to test it on Sunday. Is there anything non-obvious I
need to know to set it up?

[David Kavanagh]

I put comments in the aws.properties file. Use either old creds props,
or new set (as seen there). Make sure you pick up the new beans.xml
(or bring over the additions). I'm heading out of town in 15 minutes,
but thanks to my android phone, will still be somewhat in touch. :-)

David

[David]

Oi! Just remembered that I haven't updated the IngestorBase.. it needs
to detect new props and use double secret to ingest.
I did the queue grants in the PoolManager. Of course, I didn't get a
chance to test. Looks good on paper.

[shlomo]

David,

Well, I did get the new code, and played with it a bit, but didn't get
very far testing it. As you said, IngestorBase needs to be updated. I
can force things to work by mixing and matching properties, but that
really wouldn't be a test of the code, so....

Awaiting your commit of IngestorBase.java.

.. Shlomo

[David Kavanagh]

Upon further reflection, the FileIngestor is all I'm going to change.
IngestorBase takes a set of credentials, and those would be regulated
by the caller. So, the FileIngestor, which reads the aws.properties
file, is going to deal with double secret. To clarify, double secret
are used on lifeguard server, to control instances, etc. secret are
used on service instances, to read/write bucket data and deal with
work requests and status.
For ingestion, you'd want to select based on where that code is
running. So, if you run a server that runs lifeguard and a web app
which handles ingestion, maybe you run double secret.. but maybe that
breaks the model if that's a web server... or, maybe not if that's
internal, inside your firewall.
So, maybe which set of access/secret keys are used, depends on the
lifeguard configuration/installation. Make sense? Doing nothing for
the time being (about this).
I think I still need to deal with S3 ACLs.

David

[shlomo....@gmail.com]

Hmm... regarding which credentials to use, I think the line to draw is not "where the code is running", but "what permissions you want to grant".
I was under the impression that all the processes that access SQS and S3 would use the secret credentials, and the double-secret credentials would only be used for one activity in the PoolManager: to launch & terminate instances. This setup would obviate the need to manage ACLs on S3 and permissions in SQS.

[shlomo....@gmail.com]

Just to clarify my proposal a bit more:

The PoolManager would use the double-secret credentials in its calls to the EC2 API to launch and terminate instances. But, the PoolManager would place the secret credentials in the UserData passed to the service instances.

All other activities the PoolManager performs would use only the secret credentials.

[David]

Reading Mitch's post, I was left with the impression that what
credential are used where depends more on how safe you think those
credentials are (i.e. where those servers are running). I think he's
downstairs now, I'll go track him down and see what we come up with.

David

On Jul 20, 3:48 pm, shlomo.swid...@gmail.com wrote:
> Just to clarify my proposal a bit more:
>
> The PoolManager would use the double-secret credentials in its calls to the
> EC2 API to launch and terminate instances. But, the PoolManager would place
> the secret credentials in the UserData passed to the service instances.
>
> All other activities the PoolManager performs would use only the secret
> credentials.
>
>
>
> ---------- Forwarded message ----------
> From: <shlomo.swid...@gmail.com>
> Date: Mon, Jul 20, 2009 at 10:45 PM
> Subject: Re: [lifeguard] Re: Double-Secret security credentials
> To: lifegu...@googlegroups.com
>
> Hmm... regarding which credentials to use, I think the line to draw is not
> "where the code is running", but "what permissions you want to grant".
> I was under the impression that all the processes that access SQS and S3
> would use the secret credentials, and the double-secret credentials would
> only be used for one activity in the PoolManager: to launch & terminate
> instances. This setup would obviate the need to manage ACLs on S3 and
> permissions in SQS.
>

> On Mon, Jul 20, 2009 at 9:50 PM, David Kavanagh <dkavan...@gmail.com> wrote:
>
> > Upon further reflection, the FileIngestor is all I'm going to change.
> > IngestorBase takes a set of credentials, and those would be regulated
> > by the caller. So, the FileIngestor, which reads the aws.properties
> > file, is going to deal with double secret. To clarify, double secret
> > are used on lifeguard server, to control instances, etc. secret are
> > used on service instances, to read/write bucket data and deal with
> > work requests and status.
> > For ingestion, you'd want to select based on where that code is
> > running. So, if you run a server that runs lifeguard and a web app
> > which handles ingestion, maybe you run double secret.. but maybe that
> > breaks the model if that's a web server... or, maybe not if that's
> > internal, inside your firewall.
> > So, maybe which set of access/secret keys are used, depends on the
> > lifeguard configuration/installation. Make sense? Doing nothing for
> > the time being (about this).
> > I think I still need to deal with S3 ACLs.
>
> > David
>

[David]

Before Mitch arrived today, I properly got my head around things and
just confirmed with him as well.
So, the services instances get the secret (less secure) keys. those
keys should really not belong to the account that owns any resources.
they should access resources via sharing mechanisms (which are in
place for SQS and S3 at the moment). The lifeguard server will use
double secret (more secure) keys to launch instances, create queues,
grant access to the less secure keys as needed. SimpleDB is a pain,
and so, all the data stored there would be through the secret (less
secure) keys. We used SimpleDB for data logging in our lifeguard
installation, but not everybody has to deal with that.

Eucalyptus was another complication and since it has it's own set of
keys, it could simply replace the double secret keys, but Eucalyptus
doesn't provided all of the services that AWS does. So, we need to
decide what parts of Eucalyptus to support. I'm shooting for the host,
and storage (Walrus). I say this, even though the later is very
immature. I'm not even certain if Walrus supports ACLs like S3 does.
It certainly isn't multi-threaded yet and so supporting it is somewhat
risky. But, since computing "near" the data is pretty important for
system performance, I would maintain that we don't want to support
local compute via Eucalyptus and remote data via Amazon S3. I think
using SQS for both internal and external clouds is acceptable.

David

[shlomo....@gmail.com]

Ah. So the service instances will be passed the secret credentials via their user-data (but not the double-secret keys), and all the other activities of lifeguard will happen with the double-secret keys.

This should work for IngestorBase.ingest (which will use the double-secret key, if I understand correctly) to put objects into S3. But what about the S3 objects that are created by AbstractServiceBase.execute()? If the service does not have the double-secret keys, how will it put those objects into S3? Will there be someone (who? the ingestor?) who has pre-granted the secret keys write access to the bucket in advance?

Come to think of it, I already use lifeguard services that themselves do ingestion into different workflows. If the service instance does not have the double-secret key, then the service cannot ingest anything.

I think I am confused. Would you mind please explaining this again?

Thanks.