An Open Future for CCIF

[Reuven Cohen]

Dear Friends,

It is with an eye toward an open future that we address the many apt criticisms levied at the Cloud Computing Interoperability Forum (CCIF) and the difficult circumstance in which this community finds itself.

As the organizers of the community, we would like to make our intentions clear. The following letter is not an edict or decree. It is a heartfelt attempt to reach out to our fellow community members so we might begin to move past recent events and together, discuss our options.

 An Apology

 While sifting through this week's enthusiastic and well argued posts, one issue rose to painful clarity: There is not and has never been an agreed upon definition of the CCIF. As organizers we have “announced,” at various times, conflicting statements on how “our members” should view this Forum. These definitions range from “cloud advocacy group,” which implies membership and organized offline activity, to the much narrower “email discussion group.” Due to our failure to better define our project each community member has been left to his or her own devices, latching onto any number of definitions.

At some point over the last few months, the community began to feel a sense of ownership of and membership in the entity CCIF. Until this week, we had not fully appreciated that the CCIF had become the de facto membership organization for interoperability stakeholders. Under this new premise, it is clear that our direct and private engagement, in the name of the CCIF, vis a vis the Open Cloud Manifesto may be viewed as a breech of this community's norms. For this oversight, we take full responsibility.

Open Cloud Manifesto

To this end, when the Open Cloud Manifesto is officially released on Monday, March 30, the CCIF's name will not appear as a signatory. This decision comes with great pain as we fully endorse the document's contents and its principals of a truly open cloud. However, this community has issued a mandate of openness and fair process, loudly and clearly, and so the CCIF can not in good faith endorse this document.

Knowing what we know now, we certainly would have lobbied harder to open the document to the forum before this uproar ensued.

Governance and the Future of the CCIF

Therein lies the problem. Consider this: even if we had secured the OK to open the Manifesto for discussion before signing in the name of CCIF, there would have been no mechanism by which to formally make changes or give approval. This is, or at least in our opinion ought to be, unacceptable to most of the community.

Therefore, though this is simply a proposal to get us started considering next steps, we feel that it is time for some degree of formalization. This means governance and, of course, some or all of the following components:

    1. Formal mission statement, laws and articles
    2. Formal membership structure
    3. A board or other defined leadership structure
    4. Formal decision making mechanism
    5. Committees and/or formal interest groups
    6. Goals, deliverables and activities
    7. Wikis, websites and other properties governed by our laws and articles
    8. Financial backing and/or formal associations with industry

If the community coalesces around formalization, CCIF's organizers will go to the greatest possible lengths to ensure the process unfolds openly and in the best interests of the cloud computing community at large, not for the benefit or self-aggrandizement of any specific member or interest group.

Regarding the specifics of the outcome, we are not prepared to propose or oppose any plan. If and when the time is right, we will create a wiki or other mechanism to hash out details. For now, let's start discussing whether this is the right direction for the CCIF.

Thank you and best wishes to all,

Sam Charrington, Reuven Cohen, Dave Niesen, Jesse Silver (alphabetical)

[Alexis Richardson]

Ruv (et al., alphabetical),

Thank-you for making this statement. I am not sure an apology was
called for by anyone, but well done for demonstrating humility in the
face of some occasionally ugly criticism. A clean slate is the smart
course.

Let's move on.

You instigated CCIF. What do you think it is for? What do others
think it is for?

To date the "forum" has been characterised by various people as any
one of the following:

1. a mailing list
2. an event or 'meet up'
3. a community, organized and otherwise
4. a standards organization - delivering variously, a taxonomy, an interface
5. a set of community leaders
6. a common voice
7. a manifesto or 'position'
8. a wiki
9. a talking shop
10 a farce ... (see
http://en.wikipedia.org/wiki/A_Funny_Thing_Happened_on_the_Way_to_the_Forum)

Which if any of the above do people want?

alexis

[Mark Masterson]

I know what I want. If the WS-Deathstar wars taught us anything, it's
a distaste for vendor-driven "standards". There is a different
approach that has been emerging in the cloud space -- evolutionary
competition of various open sourced ideas. Enomalism is an example of
one such, as is Eucalyptus, GoGrid's and Elastic Hosts APIs, the new
Sun API, but also things like Nimbus and the latest efforts of the
OGF. There are many more.

Nobody *decides* which distro of Linux "wins" -- they compete, and the
market chooses. I like that mechanism. I'd like that to be the
mechanism that chooses the dominant cloud computing technologies. I'd
also like the "winner" to be open source. That open source and *be* a
winner is a demonstrated fact. Emerging de facto standards like AWS
don't need any help. But open source efforts are, by their nature,
often fragmented and disparate, making them less effective in
competing. A useful thing would be an effort to provide a place to
sort that out.

So what would that make this community? One which reviews, criticises
and publicises these competing open source efforts. A "Cloudforge" --
a one stop shop that collates all of the various disparate open source
efforts, provides expert (community) commentary on them, and makes it
easy to find and get access to them.

Just my .02

[Alexis Richardson]

Mark,

Thanks for putting a marker down.

You raise several interesting points which I would like to reply to below.

On Mon, Mar 30, 2009 at 11:02 AM, Mark Masterson
<m.mas...@computer.org> wrote:
>
> I know what I want.  If the WS-Deathstar wars taught us anything, it's
> a distaste for vendor-driven "standards".

Right, it could be as dangerous for a group of elite vendors to lock
down the interop discussion, as it could be to have no interop at all.
The voices of the innovating smaller vendor, end user, and other
community members should be equally important.

For this reason I think a property of any optimal "open cloud and
interop effort" should be that it:

* Not impose high membership fees
* Have simple, clear short term deliverables, so that you don't need
deep pockets to stay at the table
* Have open meetings and wikis so that anyone can get up to speed quickly

I feel that CCIF is heading in this direction and let's keep it that
way. It's certainly a goal of the OCCI which achieves the second goal
by narrowing focus right down to APIs around existing clouds/vdc.

> There is a different
> approach that has been emerging in the cloud space -- evolutionary
> competition of various open sourced ideas.  Enomalism is an example of
> one such, as is Eucalyptus, GoGrid's and Elastic Hosts APIs, the new
> Sun API, but also things like Nimbus and the latest efforts of the
> OGF.  There are many more.

So actually there are three themes here:

1. Voice of the end user
2. Open source code
3. Open source style governance

On item (1), I see a great role for CCIF. I have had some experience
in the last few years with developing a standard for messaging called
AMQP. One factor that has helped AMQP is that it has a working group
that encourages user participation and more recently has begun to
explicitly seek feedback from the wider community of interested
parties and end users of the existing AMQP products (RabbitMQ, OpenAMQ
and Qpid/MRG).

So I would like to see CCIF acting as a user and community sounding
board for people who want to use the cloud, discussing use cases,
economic wins, technical gotchas, and referring their experience to
users and vendors who deliver technology spec. I also think CCIF
could organise events, leveraging the marketing skills of the CCIF
evangelists who drafted the manifesto(s). At these events, people who
work on tech specs like OCCI could meet with end users who don't have
time to pay full attention to OCCI, and get feedback.

On item (2), open code, this is in two parts:

2a) Open source reference implementations. Today we don't develop
standards without at least a toy model that runs in code and is open
source. Nuff said.

2b) Open source production implementations. Yes, it is great that
service providers are choosing to be more open with APIs and open
source code. But they are under no obligation to be so, and I do not
think they should feel obliged. What *is* important is for end users
to have *cloud choice* in a *cloud market* where core offerings can
compete on easily comparable metrics, and preferably become fungible
commodities in time. Again this is something that CCIF can work
towards encouraging -- I am not sure how. I think Simon Wardley from
Canonical is a leading exponent of this view.

On item (3), independent of open vs closed code, I do think interop
forums and standards efforts should try to learn from open source
projects, in how they are run (Public mailing list, bug list, wiki,
meritocratic).

> Nobody *decides* which distro of Linux "wins" -- they compete, and the
> market chooses.  I like that mechanism.  I'd like that to be the
> mechanism that chooses the dominant cloud computing technologies.

Exactly! And this is the point that Simon Wardley argues in his case
for theme 2b above.

> I'd
> also like the "winner" to be open source.  That open source and *be* a
> winner is a demonstrated fact.

Yes that would be good but I take it you agree this cannot and should
not in some sense be 'mandated'. It has to emerge from competition
and effort.

>  Emerging de facto standards like AWS
> don't need any help.  But open source efforts are, by their nature,
> often fragmented and disparate, making them less effective in
> competing.  A useful thing would be an effort to provide a place to
> sort that out.

What does that mean? What kind of 'place'? A marketplace? Or more
of a 'forge' as you say below.

> So what would that make this community? One which reviews, criticises
> and publicises these competing open source efforts.  A "Cloudforge" --
> a one stop shop that collates all of the various disparate open source
> efforts, provides expert (community) commentary on them, and makes it
> easy to find and get access to them.

Bringing users and vendors together is good.

I would love to see the community listing projects in a wiki and
keeping the info about them up to date.

alexis




> Just my .02
> >
>

[Mark Masterson]

> Yes that would be good but I take it you agree this cannot and should
> not in some sense be 'mandated'.  It has to emerge from competition
> and effort.

My verb was "like", not "demand". ;) So, yes. Of course. (And that
2nd sentence should read "That open source can *be* a winner..." Typo,
sorry.

> What does that mean?  What kind of 'place'?  A marketplace?  Or more
> of a 'forge' as you say below.
>
> > So what would that make this community? One which reviews, criticises
> > and publicises these competing open source efforts.  A "Cloudforge" --
> > a one stop shop that collates all of the various disparate open source
> > efforts, provides expert (community) commentary on them, and makes it
> > easy to find and get access to them.
>
> Bringing users and vendors together is good.
>
> I would love to see the community listing projects in a wiki and
> keeping the info about them up to date.

Not a marketplace in a commercial sense, no. But also much richer
than just a list of known projects. More like a 'forge environment,
with some Reddit or Digg-like functionality. A place where the
uninitiated could profit from the collective wisdom of this group, as
well as place for this group to debate (and thus form) that same
wisdom. A wiki with a space for each offering deemed worthy of
discussion would suffice, if it had some snazzy little feature to
aggregate ratings feedback (perhaps as simple as a poll). There'd be
a page for Enomalism, one for Eucalyptus, one for Nimbus, one for
GoGrid's API, one for AWS's APIs, one for GAE, one for Heroku, one for
Elastic Hosts APIs, ad infinitum, perhaps some industrious soul would
do a spreadsheet comparing and contrasting APIs, etc. -- all in one
place.

"But what about setting standards?! Getting a consensus on a
taxonomy? Etc.?" some might cry. Well, I would say, those would
*emerge* from the community, as it discussed and rated and reviewed
the alternatives, wouldn't it? As that began to happen, this
community would be a natural place, then, to document that emerging
consensus. No need to force it, in a heavy handed, top down manner.

[Sam Johnston]

On Mon, Mar 30, 2009 at 12:41 PM, Alexis Richardson <alexis.r...@gmail.com> wrote:

> I know what I want.  If the WS-Deathstar wars taught us anything, it's
> a distaste for vendor-driven "standards".

Right, it could be as dangerous for a group of elite vendors to lock
down the interop discussion, as it could be to have no interop at all.

Agreed.
 

The voices of the innovating smaller vendor, end user, and other
community members should be equally important.

Agreed, this should be a key tenet of the forum.
 

For this reason I think a property of any optimal "open cloud and
interop effort" should be that it:

* Not impose high membership fees

...or no fees at all unless there is extremely strong justification for them (it's a form of discrimination after all, proportional to the size of the fee). The (currently event based?) sponsorship program could be significantly enhanced to include individuals through bigco's and I for one would be likely to contribute individually and/or professionally as a "founding" sponsor. I'll point at LOPSA for one multi-tier model we could follow and it's negligible cost to set up a 503(c)(3) org for a minimal tax/admin function.

* Have simple, clear short term deliverables, so that you don't need
deep pockets to stay at the table

See above re: deep pocket requirement (e.g. they shouldn't be required to get to the table in the first place, especially if we are to represent end users). Otherwise +1.
 

* Have open meetings and wikis so that anyone can get up to speed quickly

+1
 

I feel that CCIF is heading in this direction and let's keep it that
way.  It's certainly a goal of the OCCI which achieves the second goal
by narrowing focus right down to APIs around existing clouds/vdc.

Which reminds me, yet another point I failed to mention in my earlier posts is that OCCI appears to be complementary to UCI (which so far as I can tell is more about discovery).
 

> There is a different
> approach that has been emerging in the cloud space -- evolutionary
> competition of various open sourced ideas.  Enomalism is an example of
> one such, as is Eucalyptus, GoGrid's and Elastic Hosts APIs, the new
> Sun API, but also things like Nimbus and the latest efforts of the
> OGF.  There are many more.

So actually there are three themes here:

1. Voice of the end user
2. Open source code
3. Open source style governance

On item (1), I see a great role for CCIF.  I have had some experience
in the last few years with developing a standard for messaging called
AMQP.  One factor that has helped AMQP is that it has a working group
that encourages user participation and more recently has begun to
explicitly seek feedback from the wider community of interested
parties and end users of the existing AMQP products (RabbitMQ, OpenAMQ
and Qpid/MRG).

Queues will play a critical role in terms of "cloud as an operating environment", allowing loosely coupled components to come up and down as necessary while tolerating intermittent connectivity. Perhaps another standard to "bless".

So I would like to see CCIF acting as a user and community sounding
board for people who want to use the cloud, discussing use cases,
economic wins, technical gotchas, and referring their experience to
users and vendors who deliver technology spec.  I also think CCIF
could organise events, leveraging the marketing skills of the CCIF
evangelists who drafted the manifesto(s).  At these events, people who
work on tech specs like OCCI could meet with end users who don't have
time to pay full attention to OCCI, and get feedback.

Equally important for projects like OCCI is building momentum in the vendor/implementor/FOSS camp, which is something I've been doing myself last few days. I'd rather have been able to spend that time on the spec though.

+1 for "voice of the end user". The forum should be helping users answer questions like "How would I move from Google Docs to Zoho or vice versa", not just focusing on infrastructure.
 

On item (2), open code, this is in two parts:

2a) Open source reference implementations.  Today we don't develop
standards without at least a toy model that runs in code and is open
source.  Nuff said.

+1. Time spent helping FOSS implementations support cloud standards is time well spent - there are many ways we can assist here and we already have the makings of such a function courtesy UCI.
 

2b) Open source production implementations.  Yes, it is great that
service providers are choosing to be more open with APIs and open
source code.  But they are under no obligation to be so, and I do not
think they should feel obliged.  What *is* important is for end users
to have *cloud choice* in a *cloud market* where core offerings can
compete on easily comparable metrics, and preferably become fungible
commodities in time.  Again this is something that CCIF can work
towards encouraging -- I am not sure how.  I think Simon Wardley from
Canonical is a leading exponent of this view.

On item (3), independent of open vs closed code, I do think interop
forums and standards efforts should try to learn from open source
projects, in how they are run (Public mailing list, bug list, wiki,
meritocratic).

+1 for open source processes. The flux of forum participants should not be impaired by barriers to entry/exit and decisions should be made on the strength of arguments rather than power games.
 

> Nobody *decides* which distro of Linux "wins" -- they compete, and the
> market chooses.  I like that mechanism.  I'd like that to be the
> mechanism that chooses the dominant cloud computing technologies.

Exactly!  And this is the point that Simon Wardley argues in his case
for theme 2b above.

That doesn't preclude us from giving guidance/recommendations however, for example by certifying compliance with Open Cloud Principles or similar.
 

> I'd
> also like the "winner" to be open source.  That open source and *be* a
> winner is a demonstrated fact.

Yes that would be good but I take it you agree this cannot and should
not in some sense be 'mandated'.  It has to emerge from competition
and effort.

Agreed. Proprietary alternatives are well represented here and the two can happily coexist - no point alienating one by over-serving the other.
 

>  Emerging de facto standards like AWS
> don't need any help.  But open source efforts are, by their nature,
> often fragmented and disparate, making them less effective in
> competing.  A useful thing would be an effort to provide a place to
> sort that out.

What does that mean?  What kind of 'place'?  A marketplace?  Or more
of a 'forge' as you say below.

Interesting ideas. Project hosting sites (ala CloudForge) are a dime a dozen, and they're non-trivial to run - at least in part because the software's forked. Helping existing projects work better in cloud environments and promoting those products that do already (e.g. Hadoop, RabbitMQ, Enomaly ECP, etc.) sounds like a useful function however. Perhaps something more like freshmeat.net, or even just a section of the wiki dedicated to FOSS?
 

> So what would that make this community? One which reviews, criticises
> and publicises these competing open source efforts.  A "Cloudforge" --
> a one stop shop that collates all of the various disparate open source
> efforts, provides expert (community) commentary on them, and makes it
> easy to find and get access to them.

Bringing users and vendors together is good.

I would love to see the community listing projects in a wiki and
keeping the info about them up to date.

+1 - low cost, high value.

Sam