Hadoop buzz continues to excite the cloud

[sbinsider]

Hadoop is the popular open-source implementation of MapReduce, a
powerful tool designed for deep analysis and transformation of very
large data sets. This is a CNET QA with Cloudera founder Christophe
Bisciglia about the upcoming Hadoop World event that is taking place
in New York City on October 2nd. QA is here – http://bit.ly/loH4h

[Sam Johnston]

And this has what to do with interoperability?

Please respect list charters when advertising your clients' events.

Sam

 

[Fred Zappert]

Sam,

Hadoop does appear to be the leading the charge for the standard for cloud storage using HDFS. There are a number of interesting extensions to it, including Hive, Pig, HBase, as well as MapReduce. It's support by Yahoo is important.

While Hadoop is an open source Apache project, Cloudera is adopting the now time-honored position of  providing fee-based support and training.

I don't know for a fact if the AWS MapReduce service  is based on Hadoop, but that would be my presumption.

Most position descriptions  for cloud development engineers and architects (public, private or hybrid) now expect working experience with Hadoop and MapReduce.

Regards,
Fred.
PS: In terms of cloud storage interoperability, stay tuned for JClouds

[sbinsider]

Apologies, Sam. Not my intent to spam. My bad. Thought it would be
useful info for group.

Best,
Ray George

[Scott Jordan]

On Tue, Sep 8, 2009 at 11:52 AM, sbinsider <r...@pageonepr.com> wrote:
>
> Apologies, Sam. Not my intent to spam. My bad. Thought it would be
> useful info for group.
>
> Best,
> Ray George

IMHO it was a worthwhile post.

Hadoop and its tributaries (including HDFS and HBASE) should
definitely be considered germane to this group, and Hadoop-centric
meetings such as the one spotlighted in the article are things I
appreciate knowing about.

Just my inflation-adjusted $0.02.

--Scott

[Paul Strong]

All,

On the subject of Hadoop cloud interoperability, we (eBay Research Labs) hope to be demo-ing a standards based (OGF HPC Basic Profile - http://www.ogf.org/hpc_profile/) Hadoop as a Service test implementation at OGF27 in Banff, Canada next month.  NOTE that this is not a product, but is a demonstrator to show how some existing standards can be used to deliver interoperability in this space.  This exposes Hadoop clusters as HPCBP compliant services and allows one to use a standards compliant meta-scheduler (think MS Windows HPC Server 2008, Platform LSF et al) to schedule, stage (via ftp. GridFTP and scp), run and monitor Hadoop jobs on multiple local and/or remote Hadoop clusters.  We (OGF/eBay) demo-ed an earlier version of this at SC'08.

Cheers
Paul