Re: graphlab v2

[Danny Bickson]

Hi Steve!

We would love having you write an R interface for GraphLab. 

GraphLab v2 is almost ready and currently it is going some extensive debugging. The answer to your question can be divided into two. Do you want to create an R interface to some of our popular algorithms like matrix factorization? In that case there will be no observable change between v1

and v2. If you like to write an interface to the update function itself, then

there will be some changes. But I guess it will not be too much work to port

between v1 and v2. I will ask Joey to summarize the changes he is heading 

in v2 so you can get a better idea of it. v2 should be ready in a couple of months the latest.

If you have some spare time, we would really appreciate your great work as a collaborator. Having an R wrapper for GraphLab will be really cool.

On Wed, Nov 30, 2011 at 10:26 PM, Steve Lianoglou <lia...@cbio.mskcc.org> wrote:

Hey Danny,

I figure I'd just ask you directly instead of having a convo in your blogspot comments :-)

Regarding the R/graphlab thing ... let's assume for the moment that I'm whipping that together more because I think it's fun rather than it being a critical something I need for my work. Do you reckon I should just rather wait until graphlab v2 is a bit more baked?

I think you mentioned somewhere (blog or ML, not sure) that v2 would make graphlab easier to program against, so ... if that's the case, perhaps it'd make an R <--> graphlab bridge  a bit easier, or .. ?

Do you folks have an ETA as to when you expect v2 to be a bit more mature (if not exactly "stable")? With respect to an R library, I'd most likely be primarily concerned with the single machine/multicore scenario. I know  (think) you mentioned that a more robust distributed graphlab is a big aspect of v2 also, so maybe "just" the multicore version might be coming along at a quicker pace than the distributed version, but then again ... maybe not.

Thanks,
-steve

[Joseph Gonzalez]

Hi Steve,

Most of the shared memory design of GraphLab v2 is actually close to completion internally but we may be making minor changes as we move forward. We will also need to spend some time documenting v2 and developing the distributed version.  If you would like to start working on the v2 code now it is available by checking out the code from google code and then running:

   hg up -r v2

The big changes in v2 are:

   

 a) Update functors (State full update functions)

     * These enable something like active messaging with programmable task de-duplication 

 b) Decomposable Update Functions

     * These break the classic update function into 3 phases: a parallel gather, 

        a central update, and then a parallel scatter.

 

 c) Unified design for the sync mechanism

      * sync is modeled after update functors 

 d) Simplified interface with the engine from within the new update functors

      * The scheduler and scope have been unified into a context object 

        which provides a much wider range of functionality including access

        to local variables.   

 e) Changes to several of the core components of GraphLab:

     * Simplified schedulers, cleaned up engine, improvements in the graph 

        design (still underway)

As always, let us know what you think?

Joey