[Haskell-cafe] LFM for Haskell bittorrent effort

[Mateusz Kowalczyk]

Hi,

For this or another reason, no bittorrent libraries/clients available
today seem to suit my needs: they are either too slow, too memory
inefficient, bundled with a client which often can't run in a headless
environment or broken in some other way or don't implement all the
common BEPs. I came to the conclusion that I'll probably have to write
one. Happily, whenever I mentioned the topic in #haskell, there was
quite a bit of interest each time so I'm looking to recruit people to
join me in this effort.

First a brief overlook of what's already available in the Haskell
environment:

* Combinatorrent[1] – a rather old client, no longer maintained with the
last Hackage upload in 2010. Only supports few BEPs and was not able to
load in the .torrent file provided by Ubuntu. I have spoken to jlouis
who wrote this, you can read the conversation at [2]. He himself says
that nowadays you can do better performance wise with available
libraries. Additionally, this comes with a client on top rather than
just being a library.

* conjure – not touched since 2008

* bittorrent – seems to be the closest to what I'd like but it is still
incomplete and the library is in limbo. There is a lot of stuff that
does not work or is commented out and for various reasons the maintainer
is unavailable to help out. I have spoken to the maintainer and he
himself would like to redesign the library.

That's about it. There seem to be a few related tools for bencoding &c
but nothing huge. I think the best way to go about this is to learn from
Combinatorrent and bittorrent and start something fresh.

I'll outline the goals I personally am looking for even though they
might be obvious:

* efficient, scalable – the library needs to be able to handle tens of
thousands of active (announcing, not necessarily downloading) torrents
at once. For reference, libtorrent-rakshasa + rtorrent is able to run
about ~10k torrents in about 1GB of memory with low CPU usage. The
downside is that the library crashes like no tomorrow and the client
itself sucks. A slow library is useless to me but so is a fast but
unstable one.

* implements all the common BEPs – no use for a library if it doesn't
work for a lot of cases. It needs to implement everything from [3] maybe
except BEP 8.

* sane interface – in the end we need to be able to build clients on
top. Fortunately we can leverage Haskell's types to do this whereas the
existing C/C++ bittorrent libraries seems to suffer badly here.

* resilient – if I wanted a fast library which crashes every ten minutes
I'd stick with libtorrent

Basically, I'm looking to build a fully usable bittorrent library.
Notably, I'm not looking for people to help to write a client (yet!).

I plan to start as soon as time allows but it'd be great if there was
more than just myself on board. Let me know one way another if you're
interested in contributing. If you're just interested in the final
product then that's fine too, I could always use more motivation!

[1]: https://github.com/jlouis/combinatorrent
[2]:
http://ircbrowse.net/browse/haskell?id=18701612&timestamp=1406738464#t1406738464
[3]: http://www.bittorrent.org/beps/bep_0000.html

--
Mateusz K.
_______________________________________________
Haskell-Cafe mailing list
Haskel...@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Mateusz Kowalczyk]

On 08/31/2014 09:04 PM, Mateusz Kowalczyk wrote:
> Hi,
>
> For this or another reason, no bittorrent libraries/clients available
> today seem to suit my needs: they are either too slow, too memory
> inefficient, bundled with a client which often can't run in a headless
> environment or broken in some other way or don't implement all the
> common BEPs. I came to the conclusion that I'll probably have to write
> one. Happily, whenever I mentioned the topic in #haskell, there was
> quite a bit of interest each time so I'm looking to recruit people to
> join me in this effort.
>

> [snip]

I had a few replies already which is great news for me. If you're
interested in contributing then send me your GitHub username (or SSH key
if you don't like GitHub) and I'll create an organisation later.

There is a #haskell-bittorrent channel that was previously used for the
‘bittorrent’ library which I think we can use, if you use IRC of course.