Everest (was CloudHaskell-OTP) Suggestions

[Dylan Lukes]

I'm finding that providing a cogent layer on top of ProcessM that is
user friendly and what-not is requiring re-exporting almost everything
from Remote.

So, in that light, I'm refocusing CloudHaskell-OTP into an abstracted
layer on top of Remote (which should work nicely given Remote is in
active development), reexporting *everything*, while keeping the
original goals of including OTP-like constructs (Supervisors,
GenServers, etc).

As a result, I've renamed CloudHaskell-OTP to Everest. Since it's
(harr harr), on top of the clouds.

The end goal is that users of Everest should not have to import
Remote.

I would love to hear any suggestions or ideas. Obviously Haskell is
not Erlang, so Everest Supervisors and Servers and so forth will share
only the same roles as their Erlang counterparts. Any ideas for how
they would be most cleanly implemented would be wonderful.

[Ryan N]

Would you mind catching some of us up with the packaging decisions thus far?  When we saw that the cabal file was named "remote" and the github repository "CloudHaskell" we assumed that the package was ultimately going to be called CloudHaskell.  Or is it the other way around?

And what's OTP ;-)?

-Ryan

[Dylan Lukes]

I'm not involved in CloudHaskell itself. As far as I can tell, Epstein named the cabal file/package `remote'. I don't agree with the use of a top level (Remote) name for it... I personally think Remote.CloudHaskell.* would be a better package namespace.

As for it eventually being named CloudHaskell in the end, I have no idea. Not my project/domain at all.

I'm just building my project on top of it. OTP is the Open Telecom Platform in Erlang. It's a sort of common set of patterns you can build just about any application from which have been standardized and thoroughly tested, comprising supervisors, finite state machines, generalized servers, etc.

- Dylan

[Ryan N]

Thanks Dylan.

As far as naming goes I'd be interested to hear from Jeff.

I notice that the MPI bindings on hackage occupy Control.Parallel.MPI.  It may have been better to put that in "Control.Distributed.MPI".  The CloudHaskell package could go in one of those spots.  Or I suppose it could pay explicit homage to Erlang and be "Control.Distributed.Erlang".

-Ryan

[Ryan Newton]

By the way, speaking of MPI, we would like to run some CloudHaskell tests on an infiniband-based cluster in the near future.

Is anyone working on an MPI (or Portals or IBV/verbs) backend for CloudHaskell yet?  Peter Braam mentioned he was going to hire someone to do this but I have no idea what the time frame for that would be.

Thanks,

  -Ryan

[Jeff Epstein]

Hi,

The module is so named because the working title for the project was
Remote Haskell, and none of the existing module hierarchies seemed to
add any semantic value, while nevertheless being verbose.

There has been talk of an MPI backend. I'm not sure that it can be
done cleanly; MPI likes to manage its own workers, and as far as I
know there is no standardized way for MPI applications to spawn new
processes, recover cleanly from hardware failure, etc. So it would
probably mean either tailoring the backend to a particular
implementation of MPI, or forgoing some useful CH features, which may
result in a dramatically changed interface. I don't know much about
MPI, though, and would love to hear from someone with more experience.

Jeff

[Simon Peyton-Jones]

Dylan and others

Thank you Ryan for setting up this list. It's exciting how much energy there seems to be in the Cloud-Haskell area. I hope we can work together to build something (or a collection of compatible pieces) that will really bring Haskell into a entirely new area.

I think it'd be really helpful to have an overview of the various layers involved here. Perhaps it'd be useful to start a Wiki page on haskell.org to describe them?

For example, you are building a layer on top of ProcessM. An overview of
* What the layers are
* Roughly what API they support
* A pointer to the package or whatnot that implements them

would be fantastically helpful. For example, Jeff has a "Task" layer that he builds on top of ProcessM too, but it's probably different to your OTP layer.

And maybe others have different ideas again. For example, we probably want a communication/deployment layer *below* ProcessM to abstract away from the details of whether the underlying infrastructure is Infiniband or MPI or Portals or what.

Simon

[Dylan Lukes]

It definitely seems very very exciting :).

Well, I'm essentially just trying to put together a layer that better suits the OTP paradigm than Erlang processes themselves do. The problems I see to address are manifold.

OTP is implemented as a set of behaviours (i.e, interfaces for modules, though the requirement is less strict). In Haskell, rather than use modules, we can fill in a record with the callback closures and initialization state instead. In Erlang, each callback for the OTP components (supervisor, gen_serv, gen_fsm, ...) all take a State parameter as their first argument. In Haskell it seems more natural to instead lift all callbacks with StateT. However merely using StateT ProcessM () is somewhat nasty, so I'm trying to reexport and reinstantiate my own Process with StateT built in (I may add more later).

I may also try to add some different sorts of Processes, i.e, pure and impure ones, ones with different specializations/features, and so forth. I'm not sure yet whether this will be beneficial, but I think there are some interesting possibilities.

Wrapping remote itself also lets me undo some of its unfortunate naming conventions...

In short, my "OTP" layer is just meant to be a slightly more restricted, but easier to work with layer. The OTP philosophy is to have a core set of tools that can be composed to do their job well. My API is currently very unstable, and I'm still experimenting. However, once it evens out I'll certainly do that. I'm certainly nowhere near an expert or even relatively knowledgeable in this field, I'm just trying to hit the ground running, learn as much as I can, and hopefully make a valuable contribution.

I was wondering about backend. If I recall correctly from the whitepaper, the current remote package's closure implementation is something of a stopgap hack. From the source code it certainly seemed so. It would probably be best to allow for alternative transport layers too...

- Dylan

[Simon Peyton-Jones]

The root is here: http://haskell.org/haskellwiki/GHC

under “Collaborative documentation”

 

See for example links to “Data parallel Haskell” or “Concurrency”

 

Perhaps add a link to GHC/CloudHaskell, and then a sub-page GHC/CloudHaskell/Transport about transport layers?

 

S

 

From: Peter Braam [mailto:peter...@parsci.com]
Sent: 03 October 2011 13:54
To: Simon Peyton-Jones
Cc: cloudh...@googlegroups.com; Geoffrey Mainland
Subject: Re: Everest (was CloudHaskell-OTP) Suggestions

 

Hi -

 

I would be glad to write a few pages about network libraries that I'm familiar with or studying.   Could someone tell me where to create a page about this, then I can get going.   I'm not sufficiently familiar with the layout of haskell.org to choose myself I think.

 

Thanks!

 

Peter 

[Peter Braam]

Hello -

I wrote down my thoughts about HPC / Cloud frameworks for execution of parallel programs - see

http://haskell.org/haskellwiki/GHC/CouldAndHPCHaskell

More about networking will come during the coming days.

Peter

[Ryan Newton]

Peter, 

Regarding your wiki article, I was very interested in the fact that you see the future in software platforms that explicitly model the hardware as a data-structure.  In the literature I've seen many systems that do this in one way or another, and their particular schema for representing machines always feels a bit ad hoc.  I'm not sure what the Right Thing is.  Do you have any particular systems that you think are exemplary for doing this modeling in the right way?

Are you thinking of, for example, the coarse level of detail provided by "lstopo"?

Cheers,

  -Ryan

[Ryan Newton]

Hi Geoff -- small world ;-),

I think the key to getting a set of components that work together will be to hash out a set of type classes that encapsulate, on the one hand, basic scheduling and parallel data structures (Par monad), and communication and fault-handling libraries (CloudHaskell).

Jeff can moderate the discussion on the Cloud Haskell front ;-). Questions that have already emerged include:

• Whether to include collective communication (bcast, multicast)
• Whether to target other network APIs (CCI, Portals) -- I think we agree here: "yes"
• Whether to do an MPI backend, at the possible expense of making the "monitors" fault-tolerance layer an only optionally provided by the API (i.e. not required of all backends)

My own interest is in seeing a set of common APIs for scaling monad-par to the distributed context (one such API is in Patrick's IFL paper).  Then at least we can share applications, whether or not we share implementations!

The problem of course is that there are quite a few things we might imagine supporting.  The feature set is pretty wild right now.  We've got a draft of a class hierarchy for Par-like monads in the newClassAPI branch of the monad-par github repository.

Also, talking with Phil, Patrick, Hans-Wolfgang, and Prabhat another (pervasive) issue came up -- better concurrent data structures for Haskell!  For example, we're using a non-ideal deque type for monad-par.  Wrapping a few of the TBB data-structures may be something worth looking into.

Cheers,

  -Ryan

[Rob Stewart]

Hi Ryan,

On 11 October 2011 06:49, Ryan Newton <rrne...@gmail.com> wrote:

> Whether to do an MPI backend, at the possible expense of making the
> "monitors" fault-tolerance layer an only optionally provided by the API
> (i.e. not required of all backends)

Perhaps I can offer a glimmer of hope in this regard. I am working
with Patrick on HdpH, and my PhD is looking at fault tolerant
distributed Haskell. As MPI is the de-facto backend for (probably) all
HPC systems, it seemed sensible to focus on the feasibility of fault
tolerant MPI implementations.

Any yes - such things do exist! I'm aware of 2 MPI implementations
that handle node failure, that is - jobs are not killed given the
termination of the process on a node included in mpiexec. I am using
one of these implementations on Heriot Watt's cluster with distributed
Haskell, and preliminary results are promising. They do appear to be
adhering to the MPI standard too, using MPI_RETURN_ALL, and perhaps
one has to speculate that other fault tolerant MPI implementations are
to follow.

--
Rob Stewart
Heriot Watt University

[Ryan Newton]

distributed Haskell. As MPI is the de-facto backend for (probably) all

 

Actually, I seem to be running into more people that use infiniband verbs (IBV)!  For example, around here at IU we develop the parallel Boost graph library, which is currently based on IBV.  But I think supporting MPI too is great. 
 

Any yes - such things do exist! I'm aware of 2 MPI implementations
that handle node failure, that is - jobs are not killed given the
termination of the process on a node included in mpiexec. I am using
one of these implementations on Heriot Watt's cluster with distributed
Haskell, and preliminary results are promising. They do appear to be
adhering to the MPI standard too, using MPI_RETURN_ALL, and perhaps
one has to speculate that other fault tolerant MPI implementations are
to follow.

Great, so does that give you confidence that the CloudHaskell API could be implemented verbatim with an MPI backend?  (Jeff was doubtful but only cited the fault tolerance issue when I talked with him.)  Alternatively, the functionality could be layered and not required of all backends, much as we've been experimenting with for monad-par.

Cheers,

  -Ryan

[Simon Peyton-Jones]

Phil,

Is your paper Patrick Maier, Phil Trinder and Hans-Wolfgang Loidl, Implementing a High-level Distributed-Memory Parallel Haskell in Haskell online yet? If not, could it be. It would give us a better idea of what you are up to. Thanks!

Could you add the link to http://www.haskell.org/haskellwiki/GHC/CloudAndHPCHaskell

Simon

| -----Original Message-----
| From: cloudh...@googlegroups.com [mailto:cloudh...@googlegroups.com]
| On Behalf Of Rob Stewart
| Sent: 11 October 2011 07:28
| To: cloudh...@googlegroups.com
| Cc: Peter Braam; Geoffrey Mainland
| Subject: Re: Everest (was CloudHaskell-OTP) Suggestions
|

[Peter Braam]

Hi

I think that supporting MPI is a good idea.  It is the standard now, but the landscape is shifting in different directions as it is not convenient for the combination of distributed systems, many cores (here openmp rules) and GPU's.   Something new is going to emerge, many efforts in that space. 

As for verbs, they are supported and fast on IB and not on other networks, and other networks ae becoming more prominent again.  MPI loses 2x over verbs, something like CCI is a sliver that can run over many different networks, much more portable, and doesn't have major performance implications at the moment.   The majority of systems outside the very very alrge ones are still running on IP networks and will rarely support verbs.

We will be pairing some more information about low level networking APIs in Haskell in support of CCI and other low level layers soon.

Regards

Peter

Sent from my iPad

[Dylan Lukes]

This is all extremely interesting, but this is perhaps not the right thread for all this discussion. I intended it just for suggestions for my small project... Not to be the main discussion thread :).

- Dylan

[Trinder, Philip W]

Simon and All,

With agreement of my co-authors we've added the draft HdpH paper to
http://www.haskell.org/haskellwiki/GHC/CloudAndHPCHaskell

Also added some brief thoughts in response to Peter's on abstract
language models of large systems, with a reference to another draft
paper.

Cheers,

Phil

[Alberto G. Corona]

I need to use haskell in an environment where different users add and remove dynamically nodes to/from the cloud.

A question that seems not to be explicitly considered (there are too many three-letter achronimous here) is dynamic configuration. Allthoug I saw something in this direction in the description of CIEL. 

 What to do for adding a new node to the cloud?. How to notice and make use of it?. How to distribute workload taking into account not only static but dynamic conditions: for example a task is doing something with a blob of data in one machine but later it has to do it with a remote blob. It is better to migrate the process/state? it is better to spawn a subprocess? it is better to move te blob to the local machine instead? all this depend not only on static but on dynamic conditions: local CPU load, length of the blob, available bandwith. All these details ideally should not be specified in a high level, declarative, task oriented language runing over the services of cloud Haskell.  The scheduler should not be hyper intelligent, it can just execute rules defined by the programmer.  

2011/10/11 Ryan Newton <rrne...@gmail.com>

[Simon Peyton-Jones]

Thanks

[Trinder, Philip W]

What I omitted in my rush to collect our daughter from school was that
HdpH is Patrick Maier's brain child.

Phil

> -----Original Message-----
> From: cloudh...@googlegroups.com
> [mailto:cloudh...@googlegroups.com] On Behalf Of Simon Peyton-Jones
> Sent: 12 October 2011 22:26
> To: cloudh...@googlegroups.com
> Subject: RE: Everest (was CloudHaskell-OTP) Suggestions
>

--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.

Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012

[Trinder, Philip W]

Great, so does that give you confidence that the CloudHaskell API

 

HdpH, which looks a lot like Cloud Haskell,  is implemented with an MPI backend as we target HPC platforms that mandate MPI.

 

We’ve even used a fault tolerant MPI, so we can kill nodes.

 

Best Regards

 

Phil

 

 

From: cloudh...@googlegroups.com [mailto:cloudh...@googlegroups.com] On Behalf Of Ryan Newton

Sent: 11 October 2011 17:48
To: cloudh...@googlegroups.com

Cc: Peter Braam; Geoffrey Mainland
Subject: Re: Everest (was CloudHaskell-OTP) Suggestions

 

distributed Haskell. As MPI is the de-facto backend for (probably) all

[Kirill Zaborsky]

Dylan, what did happen to your project?

Are there any chance to see OTP in Haskell other than to implement it?

Kind regards,

Kirill Zaborsky

четверг, 29 сентября 2011 г. 22:29:32 UTC+4 пользователь Dylan Lukes написал: