Location and (j)Clouds

[Alex Heneveld]

Hello!

I'm Alex, CTO at Cloudsoft, a firm that specializes in application
mobility. We've been using jclouds for provisioning an overlay network
for our Monterey platform across multiple clouds. jclouds is fantastic
but then you already knew that.

I'm writing because there is one area where we'd like to help.
Internally we've built and been using quite a rich concept of
"location", which we use to figure out where app components should be
moved. We think migrating this work to the jclouds community would make
the concept better and also make jclouds even better. For example in
jclouds, while the Location API is placement specific (ex. which
datacenter, region, etc), it doesn't expose details such as coordinates,
jurisdiction, security, qos, network topology, which we require and
which we think will become increasingly more important at least
somewhere in the provisioning process.

Of course right now most us of this data isn't available (none of it for
some providers), and it's premature to try to represent all of this in
the API day one, but there is plenty to get started with. We see
jclouds as the best way to do this because it can be made relevant very
quickly and of course because it's open.

So if it would be welcomed, we'd be pleased to contribute effort and
design to enriching the Location API in jclouds. We'll start the ball
rolling, pretty much immediately, with discussions on the mailing list,
irc, and in-person. If you want to get involved, please raise your hand
and let us know.

Many thanks,
Alex

[Adrian Cole]

Alex.

Welcome, thanks for the kind words, and thanks for volunteering to make things better.  I've BCC'ed a few folks on this I hope will help us steer this the right way :)

To review for those who haven't had a close look at our current Location api. 

We currently support the following use cases:

1. collocation: generally, with the same Location, resources are on the same network and use private IP addresses to communicate, less implied cost, etc.
2. correlation to a service provider: blobstore and compute are provided by the same service (accounts are correlated, etc.)
3. compatible scopes: we use the same vocabulary across vcloud, ec2, etc. to describe that one scope contains another
4. assignability: out of the universe of locations in a provider, which are available and assignable

There are more aspects to this, but I think these pretty well represent current state.  Not all of the use cases above are implemented in the cleanest way at the moment, so feel free to suggest tweaks as we revisit this.  As we improve this model to satisfy the requirements you've eluded below, I'd also like to have us think about revisions/authority.  Ex. apis rarely provide enough information about location.  How can we smartly weave in additional metadata which may come from another api (ex. cloudharmony) or static reference data (ex. json balls)?

Thanks again for the help!
-Adrian

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.

[Andrew Phillips]

> 1. collocation: generally, with the same Location, resources are on the same
> network and use private IP addresses to communicate, less implied cost, etc.
> 2. correlation to a service provider: blobstore and compute are provided by
> the same service (accounts are correlated, etc.)
> 3. compatible scopes: we use the same vocabulary across vcloud, ec2, etc. to
> describe that one scope contains another
> 4. assignability: out of the universe of locations in a provider, which are
> available and assignable
>
> There are more aspects to this, but I think these pretty well represent
> current state.

I think this is an important consideration at this stage - the fact
that there unfortunately isn't enough good-quality data available yet
to support the very valid use cases (certainly from an enterprise
perspective!) Alex mentioned.

Cloudharmony and others are trying to provide some of this, and of
course we should try to use these sources in a smart way, as Adrian
said. Still, I can't see this being more than a short-term fix:
merging, translating, reconciling, checking for staleness etc. of this
data is always a huge challenge.

Put another way, until this data is largely made available by the
providers themselves, it's going to be tough.

Is there any way to start a dialogue amongst the Amazons, Rackspaces
etc. to see what's cooking here? Perhaps something for OpenStack [1]
or SINA [2]?

ap

[1] http://www.openstack.org/
[2] http://www.snia.org/cloud

[Alex Heneveld]

Hi Andrew,

Pleased to have your support. I don't usually subscribe to the "build
it and they will come" school of thought but in this case I think it
just might work.

On the content side, to begin with for sure we will have to seed this,
using information from providers where they will tell us (it is worth
asking, we've already done so with a few) -- and otherwise either
leaving it blank or limited to information we can deduce with
certainty. In other words, the easy path. (Which within Cloudsoft
we've done for a few and will contribute.)

For me the more interesting part -- initially -- is getting a mechanism
by which this can easily be both described and consumed. Have OpenStack
or CloudHarmony (or SNIA) done anything on this? I haven't seen
anything deep or "machine readable".

I agree completely, by the way, that in the longer term this information
has to come from the providers. It should be in their interest to do
so, once someone (like us) has a good mechanism. As a practical
implementation would it make sense for the location metadata API to
support references to URLs, ie to allow a static location file to refer
to providers to get full/updated info?

Best,
Alex

[Adrian Cole]

FYI, all.

Had a great response to the london cloudcamp presentation on the topic tonight:
    http://www.slideshare.net/jclouds/location-matters-even-in-the-cloud-cloudcamp-london

In case you are curious, you can follow alex's proof of concept on his fork here:
    https://github.com/ahgittin/jclouds

Any feedback so far?

Cheers,
-Adrian

[Andrew Phillips]

> Any feedback so far?

I haven't looked at the branch yet but would initially think that it'd
be good to make this an external (REST?) service which jclouds
consumes. That way it should be easier to merge in new providers' data
without touching jclouds and hopefully, in the medium-to-long term,
hand this over to providers to maintain...

Also a good opportunity to get involved with other potential consumers
of this data who are not in the Java space, perhaps?

ap

[Adrian Cole]

I think REST will be a natural outcome, but I'd like to avoid a service dependency while we're working on the model.

lean and mean for now?
-A

[Andrew Phillips]

> I think REST will be a natural outcome, but I'd like to avoid a service
> dependency while we're working on the model.
>
> lean and mean for now?

I see your point about the external dependency. I think what this
ultimately boils down to is whether we consider *collating* location
information from providers functionality that jclouds provides. As
opposed to simply making location information available that is
provided by a service.

Basically, I guess what I'm saying is that if the providers offered
good location info through their APIs we'd simply be exposing it in
jclouds. And while they're not we're trying to supplement their info
with more stuff from an additional source. But I'm not sure whether
jclouds *should* be that source.

For now? Fat and mean is fine by me ;-)

ap

[Alex Heneveld]

+1

Plump in the right places! When providers expose the data is the right
time to shed the weight.
Am working on JSON support in my branch ATM -- so REST is natural extension.

--A

[Adrian Cole]

I think I get you, Andrew.

Yeah, there will be some extension tooling to do mixins and such while we get this right, but in the end what is in jclouds core will be a rest-agnostic set of interfaces and classes. 

I think it will not only be a useful goal, but also fun, to standup a reference rest service that can show how this data can be fed from third parties.

Cheers,
-A

ap

[Adrian Cole]

P.S. 

Is there anyone that would like to simultaneously work on a REST mockup as we go along?  It will be shifting sands for sure, but if you're ok with that, let's try!

-Adrian

[Alex Heneveld]

Update: this work is now being tracked as Issue 418 at
> http://code.google.com/p/jclouds/issues/detail?id=418
and there is a wiki at
> http://code.google.com/p/jclouds/wiki/LocationMetadataDesign

The wiki includes several questions which we could use some help on!

--Alex

> <mailto:jclou...@googlegroups.com>.

> To unsubscribe from this group, send email to
> jclouds-dev...@googlegroups.com

> <mailto:jclouds-dev%2Bunsu...@googlegroups.com>.

[Andrew Phillips]

> Update: this work is now being tracked as Issue 418 at
>> http://code.google.com/p/jclouds/issues/detail?id=418
> and there is a wiki at
>> http://code.google.com/p/jclouds/wiki/LocationMetadataDesign
>
> The wiki includes several questions which we could use some help on!

Perhaps a "Motivations" section would be useful? If we can describe
the use cases we're interested in this should give some guidance as to
what types of data will be required. Especially if we can prioritize
the use cases in some way ;-)

ap

[Adrian Cole]

Very good point, Andrew.

Alex,

This wiki does relay a plan of action, but doesn't do justice to
introducing the topic and why.

What do you think about posting the latest version presentation on
motivations and background we reviewed at the riviera jug, cloudcamp
london, and with terremark?

Here's the last one I've updated:
http://www.slideshare.net/jclouds/location-matters-even-in-the-cloud-cloudcamp-london

Since a lot of folks are remote, maybe even a recorded screencast
could be useful (just an ask.. I'm sure you're busy).

Cheers, and keep the feedback running!
-Adrian

> --
> You received this message because you are subscribed to the Google Groups
> "jclouds-dev" group.

> To post to this group, send email to jclou...@googlegroups.com.

> To unsubscribe from this group, send email to

> jclouds-dev...@googlegroups.com.

[Adrian Cole]

Hi, Team.

This morning, Alex from Cloudsoft and I had a chat with Richard from ElasticHosts/ElasticStack.

Richard's company runs compute cloud services in their own and partner datacenters as well licenses their stack under the brand ElasticStack.  We are integrating with ElasticStack api via issue 412. 

The purpose of the call was to catchup with integration, and also help validate our new Location model tracked in issue 418 and on the jclouds-dev group.

Richard has the following suggestions that would help make the API relevant:

1.  expose location from legal purposes.  Ex know that one is in UK, not just in EU-WEST.  This could be exposed via ISO-3166

2. expose means of measuring latency.  For example, expose a representative edge ip address or hostname for measuring tools such as http://www.ris.ripe.net/cgi-bin/lg/index.cgi

3.  expose a normalized price per hour model. ex VM with 2 cores at 2GHz each, 2GB RAM, 100GB disk.

4. expose SLA normalized in results basis.  ex. what percentage return will one receive based on a standard VM being down for a number of hours.


Examples in discussion included the need to be near stock exchange from a <5ms latency perspective.  Near a datacenter from a physical drive-time perspective.  Examples from Elastic hosts practice included pricing/sla models such as The credit is 100x if the incident is over 15 minutes, so for 2 hours on a 50 cent VM it would be 2*0.5*100 = $10

http://www.elastichosts.com/cloud-hosting/terms-of-service
http://www.elastichosts.com/cloud-hosting/pricing

Sorry for being a bit disorganized about scheduling calls.  These are of course open to anyone to join in.  Our next chats will be announced in advance so that you can have a chance to listen-in and ask your own questions.

Cheers,
-Adrian

[Andrew Phillips]

> Richard has the following suggestions that would help make the API relevant:
>
> 1. expose location from legal purposes. Ex know that one is in UK, not
> just in EU-WEST. This could be exposed via ISO-3166
>
> 2. expose means of measuring latency. For example, expose a
> representative edge ip address or hostname for measuring tools such as
> http://www.ris.ripe.net/cgi-bin/lg/index.cgi
>
> 3. expose a normalized price per hour model. ex VM with 2 cores at 2GHz
> each, 2GB RAM, 100GB disk.
>
> 4. expose SLA normalized in results basis. ex. what percentage return
> will one receive based on a standard VM being down for a number of hours.

2-4. are certainly important from a business perspective but (unless I'm
missing something) have nothing to do with location. If we include these
kind of things I have a feeling what we're getting into is no longer a
location API but an attempt to try to "normalize" all business-relevant
aspects of different cloud providers' offerings.

Personally, I definitely see the need for this but feel that it would be
a couple of bridges too far at this point - certainly in the context of
issue 418.

Of course, businesses need to weigh lots of different concerns when
choosing a cloud provider. However, I think that, until the providers
themselves come up with a common service model description, a tradeoff
will remain between the ability to seamlessly move your application
across multiple clouds and the guarantees you can expect with regards to
performance SLAs etc.
Currently, if you have strict business needs in terms of location, SLA,
price etc. you already have the option of determining which provider(s)
meet those criteria (also factoring in human judgements like trust which
are tricky to automate). With jclouds you could then decide to launch on
only those providers.

> Examples in discussion included the need to be near stock exchange from
> a <5ms latency perspective. Near a datacenter from a physical
> drive-time perspective.

"Pinning down the cloud: real-life considerations in virtualized
environments" - I can almost see the article in front of me ;-) Jokes
aside, these examples made me take a step back and think: isn't part of
the concept of many cloud providers that they are able to deliver their
services precisely *because* they don't restrict themselves in this way?
How would EC2 look if Amazon *guaranteed* to have your AMI hosted within
so-and-so many minutes of a certain physical location? Of course, they
already offer you a regional choice, but how much more granular are they
likely to get?

Getting back to this issue, I think there are two things worth taking
into account:
- we should aim to build the location model to include mainly data that
cloud providers could/would feasibly provide themselves. That implies
some tough second-guessing, but thankfully the right people are involved ;-)
- physical contraints could be modelled with some probabalistic
component, i.e. not "the server must be within X km of this point" but
"it must be within X km Y percent of the time" or "the chance of it
being within X km should be at least Y percent". Setting Y to 100
recovers the simple "yes/no" case, too.

ap

[Adrian Cole]

Hi, Andrew.

Thanks for the feedback.  The last message was really a summary of requirements and example implementations we've gleaned from a conversation with a leading cloud software provider (ElasticStack).  There are other sets of conversations to have, and probably a great deal of these will be common enough to place into the location api. some others may be deferred to custom properties.  Our next skype call will be with Bradford Stephens (@LuciousPear) this Monday at noon PST.  Anyone should feel free to contact me (adr...@jclouds.org) and I'll add you to the call.  Hopefully, Bradford will have some interesting views from a big data/hbase perspective. 

Other feedback sessions will happen, and I'll post them here.  I'll also try to be more clear about the difference between decisions and just presenting feedback, as I understand it is probably confusing. 

All that said, I've put some comments below with my opinion on some of this, for whatever that's worth.

On Sat, Dec 4, 2010 at 9:30 AM, Andrew Phillips <aphi...@qrmedia.com> wrote:

Richard has the following suggestions that would help make the API relevant:

1.  expose location from legal purposes.  Ex know that one is in UK, not just in EU-WEST.  This could be exposed via ISO-3166

2. expose means of measuring latency.  For example, expose a representative edge ip address or hostname for measuring tools such as http://www.ris.ripe.net/cgi-bin/lg/index.cgi

3.  expose a normalized price per hour model. ex VM with 2 cores at 2GHz each, 2GB RAM, 100GB disk.

4. expose SLA normalized in results basis.  ex. what percentage return will one receive based on a standard VM being down for a number of hours.

2-4. are certainly important from a business perspective but (unless I'm missing something) have nothing to do with location. If we include these kind of things I have a feeling what we're getting into is no longer a location API but an attempt to try to "normalize" all business-relevant aspects of different cloud providers' offerings.

Personally, I definitely see the need for this but feel that it would be a couple of bridges too far at this point - certainly in the context of issue 418.

I agree these are business relevant contexts, but I'm sure we agree business context is cloud relevant.   Right now, people have to read wikis and twitter feeds to decide whether or not there are service problems in a location, etc.  Being able to monitor health of a location in general, via  a ping destination, is going to help reduce reliance on manual info.  Pricing models vary, too.  Especially with things like spot pricing, where the whole point is programmatic access to price drifting products.  I personally think price is going to be a tough one to model, but if you have programmatic access to price, provisioning decisions are more meaningful: ex. you can make a realistic comparison whether or not to spot for example.  Regardless, you can make tools with guis that allow more interactive decisions based on human choice.  Finally, on topic 4, this is admittedly hard to normalize.  Clouds like IBM dev has this in their api, where you can buy 3 levels of support for a specific offering.  From a location perspective, it is probably more relevant to a provider type of location than say, a  rack.  I'm not sure if we'll be able to work a reasonable model with SLA in it, but jotting down the requirement gives us context.  Perhaps we can.. who knows :)

Of course, businesses need to weigh lots of different concerns when choosing a cloud provider. However, I think that, until the providers themselves come up with a common service model description, a tradeoff will remain between the ability to seamlessly move your application across multiple clouds and the guarantees you can expect with regards to performance SLAs etc.
Currently, if you have strict business needs in terms of location, SLA, price etc. you already have the option of determining which provider(s) meet those criteria (also factoring in human judgements like trust which are tricky to automate). With jclouds you could then decide to launch on only those providers.

This is valid, but introduces a chicken-or-egg.  Most people I've met only know about EC2 vs datacenter.  Exposing more metadata about services in the api will allow folks to choose a service on a provider that makes sense for them.  Without this information, they'd have already needed to know there was a provider that can help them.  That wittles down the population to the ones tracking all services, which is a very small one.
 

Examples in discussion included the need to be near stock exchange from a <5ms latency perspective.  Near a datacenter from a physical drive-time perspective.  

"Pinning down the cloud: real-life considerations in virtualized environments" - I can almost see the article in front of me ;-) Jokes aside, these examples made me take a step back and think: isn't part of the concept of many cloud providers that they are able to deliver their services precisely *because* they don't restrict themselves in this way? How would EC2 look if Amazon *guaranteed* to have your AMI hosted within so-and-so many minutes of a certain physical location? Of course, they already offer you a regional choice, but how much more granular are they likely to get?

I think that we shouldn't settle for amazon's location concepts.  They do have region/availability zone/placement group, but these aren't portable unless we try and model some characteristics.  We may find that in future feedback sessions, no-one else cares about network locality, but I'd hold the guns for a bit.  I've noticed quite a few caring about this, and even things like L2 distances make HPC work vs not work.
 

Getting back to this issue, I think there are two things worth taking into account:
- we should aim to build the location model to include mainly data that cloud providers could/would feasibly provide themselves. That implies some tough second-guessing, but thankfully the right people are involved ;-)

+1 so far, we've spoken to terremark and elasticstack about this in this context.  we'll try to get more feedback from those who would provide this data.

- physical contraints could be modelled with some probabalistic component, i.e. not "the server must be within X km of this point" but "it must be within X km Y percent of the time" or "the chance of it being within X km should be at least Y percent". Setting Y to 100 recovers the simple "yes/no" case, too.

very interesting, especially in platform perspective.  I really like the idea.

[Adrian Cole]

Moving from reporter to implementer :)

I've thought quite a bit about what Andrew raised, specifically where to draw lines about what use cases we are looking to support in the location api.  We do want to focus on location sensitive issues, without weighing it down.  Some issues are location sensitive, but better performed elsewhere. For example, pricing of resources is actually practically useless at a location level [1]  Since TemplateBuilder already supports specifying a location id, we can defer responsibility of reasoning about whether a link to a network is free or not (ex. ec2 vs s3) to an external lib.  This bandwidth pricing problem is fairly isolated to AWS and won't become more or less difficult if we address it later.

Further reasoning about the nature of a location are also valid, including compliance, sla, etc.  However, it seems the cloudaudit group are far along the way of addressing these concerns.  It may be better to provide an integration model with cloudaudit than duplicate it here. [2]

What if we take a minimalist approach similar to Andrew's thoughts?  This doesn't mean dismiss use cases, rather document approaches we suggest to deal with ones that aren't hard coded, and file issues for those that may not be practical to implement as yet.

Here's my take on the minimal changes needed:

  * Extend Location model to support ISO 3166-1:2006, 3166-2:2007 geography statements most agree with.
  * Add in a URI to the cloud audit root where further assertions such as compliance and SLA can be made.
  * Add a Map<String, Object> metadata element to location so that any as yet untyped metadata can be specified.

I think taking a few small steps forward will allow us to practice the api while we get more feedback.

What do you think?

Cheers,
-Adrian

P.S. call with bradford is tomorrow 12pst. If anyone is interested, ping me.  Also Teich from Heroku will comment on the 16th.  More interviews will be posted as I get around to scheduling them :)

[1] location is a very small factor on price and seldom used.  If we are to price, it may be technically more portable to price a template rather than a vm or a location independently.
[2] http://wiki.cloudaudit.org/working-groups/standards

[Mark A. Carlson]

This is the right approach, and one that CDMI takes for the metadata
called cdmi_geographic_placement :

"JSON array of JSON strings, each containing a geopolitical identifier specifying a region where the object is permitted to or not permitted to be stored.

Geopolitical boundaries are a list of ISO-3166 country codes.

A "!" in front of a country code excludes that country from the previous list of geopolitical boundaries."

-- mark

On Tue, Dec 7, 2010 at 9:25 AM, Adrian Cole <adrian...@gmail.com> wrote:

...

[Adrian Cole]

Thanks for pitching in and highlighting the CDMI usage.  The helps, especially as we look at PaaS implications where a service can span multiple geographies (ex. BlobStore).

Cheers,
-Adrian

--

[Alex Heneveld]

+1 on:

- ISO 3166-1 and -2 alpha codes for country and subdivision ("region")
- cloudaudit repository root URL field (cloudaudit.url ?)
- Map<String,Object> properties space, though i would call it "custom"
or customProperties

most of this is in ahgittin/jclouds at present (not cloudaudit url yet
but that will come soon).

FYI it also contains:

- postal address
- coordinates (latitude and longitude)

should these be hard-baked at present or left as experimental? or
changed or junked altogether?
(for cloudsoft's purposes, coordinates and region have been the two most
useful.)

//follow up on wider discussion of purpose of location API to follow

--A

[Alex Heneveld]

excellent question about what information is appropriate to have contained in the location API.

for background -- we started with it containing only details about where it is situated, but we actually found that to be an unnatural segmentation of information.  for example "region" (e.g. US-VA), from how we wanted to consume it, was really just a type of "jurisdiction" and we also care about others such as EU or EEA or even NATO; and the question we were asking when we cared about region was "where am i allowed to put my service" in which case the answer also depends on physical security, audit information, etc (none of which strictly speakig answer the question "where").

a different use case is teased out from why we cared about latitude and longitude.  the main reason was to show various regions on a map (and only secondarily to compute distances, though other's mileage will no doubt vary [ahem]).  the generalised question was "how should i display this site", and the answers which come back could include logos, names, pantones, etc.

so what we actually wanted from the "location api" was information _about different locations_ where we might run things -- effectively information about different cloud service provider locations (or cloud service provider instances -- and perhaps "location" is misleading though personally i think it fits as it speaks to the fact that these are real tangible places and that's why we are discriminating!).  from that point of view richard's suggestions fit nicely, and links to other references such as a cloudaudit site worth including.

i hope this history is useful.  i will add some use-cases in the wiki to discuss, and also upload some illustrative slides.

my final thought is that the most important things to expose at this point in a location api are:

- a robust custom properties scheme (Map<String,Object> feels good)
- a sufficiently rich initial set of supported core properties to be useful (e.g. those listed above)
- the ability to define and *share* ad hoc extensions (using the robust custom properties scheme but affording some easy way for clients to access those properties)
- some illustrative/experimental custom properties/extensions provided out of the box (i am thinking of richard's SLA benchmark and andrew's probabilistic statements) which if feedback is positive may move in to core, and which will in future absorb other ad hoc custom properties which gain currency

comments?

best,
alex

On 07/12/2010 16:25, Adrian Cole wrote:

[Fred Zappert]

Hi,

Just out of curiosity, having worked recently on a location-based service, what's the standard representation for geocoded coordinates.

I've seen lat/long expressed as degrees in floating point (double) and as integer degrees and minutes, with the seconds taking a fractional representation.

Fred.

[Alex Heneveld]

Most places I've seen use degrees as double under the covers with N and E being positive (eg http://code.google.com/apis/maps/documentation/geocoding/ ).  For any computation this is obviously converted to radians pretty quickly but that's not very intuitive to look at.

The jclouds location work stores the direction with degrees as non-neg double -- for clarity and to avoid allegations of Eurasian bias.  The builder API at present requires a direction and accepts any of the following (longitude omitted for brevity):

        withLatitude(double latitude, LatitudeDirection latitudeDirection);
        withLatitude(int latitudeDegrees, double latitudeMinutes, LatitudeDirection latitudeDirection);
        withLatitude(int latitudeDegrees, int latitudeMinutes, double latitudeSeconds, LatitudeDirection latitudeDirection);

Does this look reasonable?

--A

[Adrian Cole]

Trouble with this stuff really happens at the PaaS level, since as
Mark from CDMI noted, services can spread across multiple geographies.
From an IaaS perspective, and in certain scopes (esp RACK, HOST) it
can work.

On geocoordinates as a top-level item in location [1], should we allow
multiple geocodes?

gut feeling is that postcode is a bit less available and generalizable
across cloud services such as the above.

I'm wondering if the universe of properties are really bound to a
specific scope, as this stuff makes more sense when in a context such
as datacenter

thoughts?

-Adrian

[1] Typically, inside jclouds, construction of things like Location,
Node, etc. are not exposed to the users. I appreciate the Builder
stuff though :)

[Adrian Cole]

All, I've heavily updated the location wiki including interviews with
Richard, Bradford and Jason:

http://code.google.com/p/jclouds/wiki/LocationMetadataDesign

If you have strong opinions on items in the Open Questions section,
please make note, or let me know, if you don't have write access.

Also, please let us know who else you'd like to involve!

Cheers,
-Adrian

[Andy Edmonds]

Hey all,

From what I've extracted from this conversation, there's a general idea of interacting with both functional (e.g. cpu speed) and non-functional (e.g. location). Focusing on Location is an excellent use case and using that to drive aspects related to it in jClouds. There also exists the potential to make these discussions more general purpose. I've seen references to SLAs mentioned and really when talking about service offerings and referring to functional and non-functional characteristics of a service it is the SLA that can bind these all together. Would it be of benefit to consider such a SLA-based model here? If so, I can bring plenty of existing work (including code), from SLA@SOI [1], to the table here so as to compliment and augment jClouds in this respect. SLA@SOI has spent the last 2+ years in tackling such and area and the vast majority of its outputs are licensed liberally (content CC, code BSD). Having such a model would allow service providers not only express characteristics like Location but others. Would people here be interested in this?

Now with my OCCI [2] hat on - a while back we talked about how we would expose the location of a compute resource through the API. It was agreed then that the use of an OCCI Category would for the simple cases suffice. Where a provider offered different locations to deploy resources at, those would be discoverable through the Query Interface as Categories. For a resource to be provisioned at a specified location, the client would supply an additional Category to the request. To query for a user's resources at a location would be simply retrieving back a list of resources pointed to the location Category.

HTH,

Andy
andy.edmonds.be

[1] http://sla-at-soi.eu

[2] http://occi-wg.org

PS: @Adrian - sorry for my delayed response.

[Alex Heneveld]

Hi Andy,

Sounds like our work should support references to SLA@SOI / OCCI descriptions at the very least.
What should the format for this be?  Would it make sense to point at URLs as we are considering doing
for CloudAudit?

cloudaudit.root.url="http://..."

Cheers
Alex

[Adrian Cole]

Hi, Andy.

Thanks for the offer wrt SLA. Certainly a lot of folks are interested
in this. I'm also keenly interested in the OCCI translation of some
of this work.

Based on cumulative feedback, I'm starting to feel we should take care
to support (i.e. make visible and/or possible to integrate) things
like Jurisdiction and SLA as top-tier concerns, but not necessarily
couple them to Location. I'm readying a summary from Rich Miller on
the topic, as I think it has helped me understand a need to not
conflate concepts.

So, in short, we have a universe of concepts important to decision
making including audit results, pricing, jurisdiction, stewardship,
sla, catalogue. Some are quite large and probably better pushed
outside jclouds, yet with clear integration paths from within it.
Some are also directly derived from location, and others location is
just a piece of input data. This continues to be quite an exciting
discussion.

Cheers,
-Adrian
P.s. do you have time interview on the concepts that interest you, so
that we have better context of SLA and location? You're clearly a
good source here.

[Andrew Phillips]

> withLatitude(double latitude, LatitudeDirection latitudeDirection);
> withLatitude(int latitudeDegrees, double latitudeMinutes,
> LatitudeDirection latitudeDirection);
> withLatitude(int latitudeDegrees, int latitudeMinutes, double
> latitudeSeconds, LatitudeDirection latitudeDirection);
>
> Does this look reasonable?

Sorry, coming in a bit late on this one - just catching up with jclouds
developments after a long week ;-)

From what I remember we have a couple of main use cases:
- displaying the location of servers to users, e.g. in a Google Maps mashup
- using the location data to perform calculations such as "how far is
this server from me/a given point/city etc."

That would seem to imply that we would want to be able to *get* the
location in
- formats that are convenient to calculate with (e.g. radians)
- formats that popular map display APIs accept

From the point of view of how to *set* the data (i.e. Alex' examples)
the only thing I can think of is that we probably want to support what
e.g. Google Maps comes up with, such as "55.944584,-3.18701" [1].
As far as I can see that's not yet covered by the proposal (I mean, the
latitude part 55.944584, of course), but maybe I'm missing something.

ap

[1] http://maps.google.com/maps?q=55.944584%2C-3.18701

[Alex Heneveld]

Hi Andrew-

Comments in-line.

On 11/12/2010 17:39, Andrew Phillips wrote:
> From what I remember we have a couple of main use cases:
> - displaying the location of servers to users, e.g. in a Google Maps mashup
> - using the location data to perform calculations such as "how far is
> this server from me/a given point/city etc."
>
> That would seem to imply that we would want to be able to *get* the
> location in
> - formats that are convenient to calculate with (e.g. radians)

Is there anything convenient about spherical coordinates? :)
I favour convenience methods for things like distance (and I think there
are only a handful at most)
rather than radians on the API.

> - formats that popular map display APIs accept

I think it is worth having an API method to return _signed_ latitude
(e.g. negative)
is good for Google compat etc. Any other APIs or thoughts?
Currently on the getter side it then looks like this:

double getLatitudeMagnitude() //always non-negative
enum getLatitudeDirection() // N or S
double getLatitudeNorth() // may be negative

> From the point of view of how to *set* the data (i.e. Alex' examples)
> the only thing I can think of is that we probably want to support what
> e.g. Google Maps comes up with, such as "55.944584,-3.18701" [1].
> As far as I can see that's not yet covered by the proposal (I mean, the
> latitude part 55.944584, of course), but maybe I'm missing something.

You can say withLatitude(55.944, NORTH).withLongitude(-3.187, EAST)
currently.
There isn't anything like withLatitudeLongitude(55.94, -3.187), whas
that what you were suggesting?

--A

[Andrew Phillips]

>> - formats that are convenient to calculate with (e.g. radians)
>
> Is there anything convenient about spherical coordinates? :)
> I favour convenience methods for things like distance (and I think
> there are only a handful at most) rather than radians on the API.

Oh, sure - I was just referring to a previous comment in this thread
that you'd probably want to go to radians for calculations, but maybe
I didn't recall that correctly.

> You can say withLatitude(55.944, NORTH).withLongitude(-3.187, EAST)
> currently.
> There isn't anything like withLatitudeLongitude(55.94, -3.187), whas
> that what you were suggesting?

Maybe ;-) Or perhaps even withLatLong(String lat, String long)...I
don't really know. I'm just thinking that, if common APIs return
Strings and everyone will have to convert them to doubles, checking
for exceptions etc. and catching them, we could also offer that in the
API.
But perhaps a helper utility would indeed be a better place for that
sort of convenience methods....

ap

[Adrian Cole]

Hi, Team.

Thanks to Rich Miller from Telematica, Inc [1]. The interview with
him last week definitely helped clarify concepts, and we are indebted
to him for typing up his notes in such an easy to understand way.
Please have a look at the updated link, as it really shows Location in
context in a rather clear way. If you feel otherwise, do speak up!

http://code.google.com/p/jclouds/wiki/LocationMetadataDesign#Rich_Miller,_Telematica_Inc.

Cheers,
-Adrian
[1] http://www.telematica.com/about-rhm/

[Andy Edmonds]

Hey Alex,

So OCCI allows the root URL of the service be defined by the provider. From that root URL all other resources are discoverable through the query interface, otherwise directly with a priori knowledge. There's more details of all this in the current set of specs that are currently out for public comment [1].

For SLA@SOI, it defines a model whereby it allows the representation of service characteristics and attributes (be they static or dynamic). Within the model, a service's endpoint (e.g. root URL) is defined. If you wish to dig into the model, I believe that the Y2 deliverable [2] on the model describes this. There is also a paper on the model published at Grid 2010 [3] (presentation here [4]). If you have difficulties in sourcing the paper grab me offlist.

HTH,

Andy
andy.edmonds.be

[1] http://ogf.org/gf/docs/?public_comment

[2] http://sla-at-soi.eu/wp-content/uploads/2009/07/D.A5a-M26-SLAManagementFoundations.pdf

[3] http://www.sla4d-grid.de/?q=node/30

[4] http://www.sla4d-grid.de/sites/default/files/SLAWS_Grid-2010_SLAatSOI_Model.pdf

[Andy Edmonds]

No problem and +1 to what you say! :-)

RE: interview, I think that should be sorted out with next week.

Andy
andy.edmonds.be

[Adrian Cole]

Many thanks to Oren Teich from Heroku, who gave us his perspective.  An interesting read:

http://code.google.com/p/jclouds/wiki/LocationMetadataDesign#Oren_Teich,_Product_Guy_at_Heroku

Best,
-Adrian

[Adrian Cole]

One more in an typed up!  Here's what the cloudswitch guys have to say:

http://code.google.com/p/jclouds/wiki/LocationMetadataDesign#John_Considine_and_Seshu_Pasam,_CloudSwitch

Cheers!
-Adrian