distributed Watchmaker on linux

[Jose Aguerre]

Hi to everyone!! I'm new to watchmaker and I'm developing a Genetic Algorithm to extract a partiture from a song using audio fingerprints. It's kind of cool how it is turning out.

The thing is that the fitness evaluation takes considerable time because it needs a linux service that is not paralelizable (transforming midi to wav). If I could use more linux pcs the algorithm would be much more efficient.

So, here comes the question:

Has anyone tried some of the Watchmaker distributed techniques?? Any examples or tutorials available? I've searched all the web and had no luck....

Cheers and thank you guys in advance!!

José.

[Klaas Hölscher]

Hey there,

Dan posted https://gist.github.com/dwdyer/3961650 two years ago for use with terracotta. I remember trying to move an example from the framework to hadoop/mahout some years ago but i did not get very far ;-)

A far more simple solution would be running multiple services as in one per cpu core (on an eight-core machine or so), if thats possible somehow and pool it for the Evaluators in your application.

Maybe someone here has already implemented distributed evaluation, but i cant help you with that

Regards,

Klaas

--
You received this message because you are subscribed to the Google Groups "Watchmaker Framework for Evolutionary Computation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to watchmaker+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Dyer]

On Thu, 17 Jul 2014 13:30:10 +0100, Klaas Hölscher
<hoelsch...@gmail.com> wrote:

> Hey there,
>
> Dan posted https://gist.github.com/dwdyer/3961650 two years ago for use
> with terracotta. I remember trying to move an example from the framework
> to
> hadoop/mahout some years ago but i did not get very far ;-)

Yes, that's about as far as I got. The Terracotta example worked but with
the example I chose it was slower than running it standalone. It would
work well if each fitness evaluation was long-lived; less well if there
were just lots of them.

Mahout might be the better option but I've never tried using it myself.

Another thing to consider, though it's not supported out-of-the box, is to
use the support for island models and have each "island" running on a
different machine. There would be less communications overhead between
nodes then.

Dan.

--
Daniel Dyer