TBB in HipHop

[ZaneA]

Hello HipHop users and developers!

I am sending this message on behalf of OpenParallel, a group that is
working to further multi-core processing into the mainstream. I am
here today to announce our HipHop branch/extension that aims to push
the integration of Intel’s Threading Building Blocks library into
HipHop beyond the already utilized data structures (such as the
concurrent_hash_map).
In particular we are looking to get some parallel functionality
exposed to PHP itself through this extension.

We have already had some initial success with a parallel_for
implementation that can be used from PHP scripts using a familiar API
(Aligned with the C++ API but slightly modified to better suit PHP
style).

Our repository can be found at http://github.com/OpenParallel/hiphop-php,
the extension is simply named tbb, and can be found in the usual paths
in the source tree.

Is there any work under way already to extend the usage of TBB in
HipHop? Are there any reasons why only the data structures have been
used so far?


Zane Ashby,
OpenParallel

[Scott MacVicar]

Hey Zane,

Thanks for the first contribution, do you have a test script and some benchmarks that you've used this with?

We've looked at adding a call_user_func_async() that would do a limited threading within HipHop.

- Scott

[ZaneA]

Hey Scott,

We have a small php test script on a private Dropbox account that we
have been using for testing, which will be moved to GitHub some time
soon.
The API is pretty straightforward, an example (This is basically our
test script):
<?php
function test_callback($indices) {
foreach ($indices as $idx) {
echo "{$idx}\n";
flush();
}
}

// Parameters are, (start index, length, function, optional grain
size)
parallel_for(0, 1000, 'test_callback');

// Will print from 0 to 1000 using multiple cores
?>

We have no benchmarks at this stage, although we plan on integrating
this with a custom Wordpress fork which we will most definitely
benchmark in the future.

Cool, call_user_func_async() and the associated memory copying/merging
is something I will be looking at a bit more closely soon.

Cheers,
Zane

On Jul 27, 2:55 pm, Scott MacVicar <macvi...@facebook.com> wrote:
> Hey Zane,
>
> Thanks for the first contribution, do you have a test script and some benchmarks that you've used this with?
>
> We've looked at adding a call_user_func_async() that would do a limited threading within HipHop.
>
> - Scott
>
> On Jul 26, 2010, at 4:56 PM, ZaneA wrote:
>
>
>
> > Hello HipHop users and developers!
>
> > I am sending this message on behalf of OpenParallel, a group that is
> > working to further multi-core processing into the mainstream. I am
> > here today to announce our HipHop branch/extension that aims to push
> > the integration of Intel’s Threading Building Blocks library into
> > HipHop beyond the already utilized data structures (such as the
> > concurrent_hash_map).
> > In particular we are looking to get some parallel functionality
> > exposed to PHP itself through this extension.
>
> > We have already had some initial success with a parallel_for
> > implementation that can be used from PHP scripts using a familiar API
> > (Aligned with the C++ API but slightly modified to better suit PHP
> > style).
>

> > Our repository can be found athttp://github.com/OpenParallel/hiphop-php,

[Marcel Esser]

The one thing that I would find incredibly useful day to day is the
ability to launch worker threads inside the apps. Yes, I realize this
violates share-nothing, but that's been out the window for a long
while anyway. It would be massively useful to have a persistent thread
pool to work with across requests. A dispatcher/worker architecture
with gearman, worker processes, etc, works well, but it's a lot of
stuff for something that could be a lot simpler (especially when
talking about small jobs rather than large ones, like thumb generation
vs video encoding).

Just my two cents on the topic.

- M.

[ZaneA]

Hi Marcel,

With the TBB integration, a thread pool is indeed one of the things
that will become available to the end user. I agree, for smaller tasks
such as thumb generation, it is important to have access to a
lightweight method of running code asynchronously, and it should be
much cleaner than using worker processes or using Gearman.

Cheers,
Zane

[Marcel Esser]

Your intentions, good sir, are full of win.

[ZaneA]

You can see some of our progress at the link below,

http://openparallel.wordpress.com/2010/09/08/parallel_for-implementation-project-tbb-hiphop/

Cheers,
Zane

[Haiping Zhao]

Wow, this is really interesting to me. Anywhere I can read more more details
about it? How it works and how to turn it on? Thanks! Excellent work. I'll
read the web site more for sure.

-Haiping

[Scott MacVicar]

Haiping its a separate extension and just opens part of TBB which is already present.

http://jfkbits.blogspot.com/2007/12/tbbs-parallelfor.html

- S