Re: my first successful example in bloom and some questions about bloom

[Joe Hellerstein]

Dillon, sounds great!

Your question is a very good one.  I believe that Bloom is a reasonable language to consider for a big-data scenario like Storm.  It remains to be seen if the current Bud prototype is up to the job though.  Keep in mind that Bud was not tuned for high-bandwidth data transfer.  My advice would be to use Bud for orchestrating the flows, but plan on eventually using low-level code (maybe even C++ rather than Ruby) for high-bandwidth data movement.

Keep us posted!

Joe

On Dec 3, 2012, at 5:36 PM, Dillon Peng <pengc...@gmail.com> wrote:

hi, all

    Thanks all! 

    Up to now, I completely implemented "3.4 A Robust Ring" of p2's user guide pdf  in bloom, 

next, I will implement a Chord refer to Implementing Declarative Overlays . 

    Of course, I will commit them to my github after finishing Chord!

    But I knew little about bloom's real capability, so I have a question about bloom: (I am sorry if my question is stupid ^_^)

 Bloom can build big data resolution, especially like Storm ?

[Neil Conway]

On Tue, Dec 4, 2012 at 12:58 AM, Joe Hellerstein
<helle...@cs.berkeley.edu> wrote:
> Your question is a very good one. I believe that Bloom is a reasonable
> language to consider for a big-data scenario like Storm. It remains to be
> seen if the current Bud prototype is up to the job though. Keep in mind
> that Bud was not tuned for high-bandwidth data transfer. My advice would be
> to use Bud for orchestrating the flows, but plan on eventually using
> low-level code (maybe even C++ rather than Ruby) for high-bandwidth data
> movement.

Agreed. Ruby wouldn't be my first choice for building something like
Storm, but a version of Bloom that ran on top of another runtime might
well be appropriate.

With respect to separating management/orchestration from data
movement, the BOOM Analytics paper gives an example of how to do this:
http://db.cs.berkeley.edu/papers/eurosys10-boom.pdf

Neil

[Dillon Peng]

hi, 

        Thanks Joe, Neil!

[Joe Hellerstein]

Hi Dillon:
    Thanks for reaching out.  I think a HOP-style implementation with coordination in Bloom and execution in Java makes sense.  The Bloom example code that I remember is something I hacked together very quickly one day, and I'm not sure it makes a great starting point.

If you decide to pursue this direction, do stay in touch.

Joe

 

On 3/26/2013 7:29 AM, Dillon Peng wrote:

hi, all

   Thank all!

    I have already implemented a Chord (based on Implementing Declarative Overlays ), next I will upload the source code to my github.

    As I mentioned in previous post, I thought whether I can design a Stream Processing Engine like storm, but,  after reading and practicing some storm, and HOP,

I have a new idea that whether I can use Bloom and Java (for plan) to implement a real-time MapReduce( as title described ) based on the Bloom's example MapReduce, Or said whether I can re-implement HOP and modify HOP to process external real-time data stream (not depending HDFS like stock Hadoop).?

   Maybe my idea is crazy, any advice will be appreciated!  Thanks a lot in advance!

BEST REGARDS

PengCZ

--
You received this message because you are subscribed to the Google Groups "Bloom Language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bloom-lang+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Dillon Peng]

hi, Joe

    Thank you very much,     Your reply gave me encouragement and confidence!

     I will post my plan recently. Any one who interested in this will be welcomed!

BEST REGARDS

PengCZ