A&O Alan McGovern MonoTorrent/Monsoon - Final Report
23 views
Skip to first unread message
Alan
Aug 15, 2008, 12:19:03 PM8/15/08
to Mono Summer of Code 2008
Hi,
Here's my final report. I've just detailed what I've accomplished over
the summer and what I have left to work on. Firstly, my SoC tasks:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
MonoTorrent:
Completed:
* DHT support
The DHT module is pretty much complete, but as I haven't had regular
internet this summer it's relatively untested 'in the wild'. I'd
estimate at most 2 weeks of testing/usage in the real world before it
can be considered complete. It passes all 34 NUnit tests so far, so
i'm relatively confident it should be fine. The biggest problem is
likely to be issues with other non-compliant clients.
* Manual IP banning - Integration with existing IP range ban lists
This has been completed and has NUnit tests attached. It parses the
three most popular banlists as well as allows the user of the library
to ban any IPAddress they want.
* Extended Messaging Protocol (Azureus and LibTorrent)
It was decided that supporting the Azureus protocol was a waste of
time. Only the LibTorrent protocol is supported, and support for that
is complete.
* DBus service
This is still more of a proof of concept than a solid usable item.
Whilst it is possible to use the service to download torrents and
retrieve various statistics from those torrents, it is not advanced
enough to use as the default backend of an advanced torrent client.
* Banshee plugin to download torrents
A basic banshee plugin has been created to allow banshee to download
torrent based podcasts. I submitted this work to Gabriel Burt a few
days ago.
* Implement Http Seeding (Getright).
Support for this has been nearly completed. The only thing needing to
be added is support for multi-file torrents. This is a fairly trivial
thing to add - the hard thing was making the WebSeeding code use
standard BitTorrent messages when integrating with the library.
Basically, whenever the engine sends a requests a block of data to a
'WebSeeder', this request is translated into a standard HttpRequest.
When the data has been downloaded from the web server, a fake
'PieceMessage' (the bittorrent message which contains the block of
data) is returned to the engine. This way a Http server can be treated
*exactly* the same way as a regular bittorrent peer, meaning all the
existing infrastructure to deal with piece reading/writing, timing out
and verification can be used. I'm quite proud of this hack :)
* Implement peer exchange support.
This was completed solely by Olivier Defour, i did nothing for this
except 1 or 2 small bugfixes.
Not started:
* HTTP/SOCKS Proxy support was never started. I received an email
after blogging about it from a guy who did implement this support in
MonoTorrent. I aim to merge those changes into svn when i get back to
Ireland. It was too big a thing to take on at the moment.
* Expose the WebUI over the DBus service
The WebUI that was originally being worked on is no longer maintained.
I don't think I have the time to maintain it, so i've let this item
slide for the moment. If someone were to maintain the WebUI, it'd be
fairly easy to hook it into the DBus service.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
Monsoon:
Completed:
* NAT indicator - Display NAT status in a clear and understandable way
There is now a little indicator with 3 colours, red, yellow and green.
This colours indicate whether a upnp/nat-pmp capable router could be
found along with whether or not an incoming connection has been
detected.
Not Complete:
* Integrate the WebUI
As before, the WebUI isn't maintained, so i didn't integrate it.
* Port Monsoon to the DBus service
The DBus service isn't suitable (yet) to be used by Monsoon, so this
hasn't been completed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
Here are the major changes to MonoTorrent itself which have been
committed between 5 May and now. That's roughly when I started the SoC
work:
* Completed support for the Extended Messaging Protocol (LibTorrent).
Olivier submitted a few patches for this.
* Faster SHA1 Algorithm used by default. This decreased the time spent
hashing a large torrent by a significant amount of time (iirc about
30%). Some of these changes were rolled back into Mono.
* Fixed several bugs and race conditions in the Encryption code and
cleaned it up hugely. Initially this code had no NUnit tests, there
are now 15 NUnit tests.
* Initial implementation of WebSeeding completed by Olivier Defour.
I've extended this implementation and added several NUnit tests. This
is still disabled by default as support for multi-file torrents is
still lacking. The current NUnit tests fail because of this missing
support. However WebSeeding support is very close to be enabled by
default. I should even by able to do this with only a few hours of
work. Multi-file support would probably be the easiest feature to
code.
* Peer Exchange support completed by Olivier Defour and enabled by
default.
* Completely refactored the threading api to make it impossible to
deadlock and easier to extend. Everything now runs on a main loop,
similar to winforms/gtk. This was a fairly major change and took quite
a while, but things have become *drastically* simpler with this API.
It used to be difficult to add new functionality to anything which
touched the socket code. Previously, adding what I thought was a
simple check to abort async connections ended up deadlocking the
library on two separate occasions. With the new API, it's next to
impossible to deadlock the library - a big win.
* Implemented a way to cancel asyncronous connect requests as
sometimes they can take over 3 minutes to fail
* Added ability to change the listen port on the fly and be able to
gracefully handle when a port is busy.
* Added a whole new debug GUI written by Karthik Kailish and David
Sanghera.
* Committed a PiecePicker written by Karthik and David that allows you
to efficiently 'stream' a torrent.
* Implemented the necessary API to allow you to ban IP address ranges.
Can parse the three most popular banlist formats: emule, peerguardian
and safepeer
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
Other points of interest: Originally there were 55 NUnit tests in the
MonoTorrent testsuite. There are now 110 NUnit tests for MonoTorrent
and 34 NUnit tests for the DHT module. That's nearly triple the
tests :)
Finally, thanks for the opportunity to take part as a student again.
It was fun. I'm looking forward to enabling all these new features by
default for the next release :)
Alan.
Here's my final report. I've just detailed what I've accomplished over
the summer and what I have left to work on. Firstly, my SoC tasks:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
MonoTorrent:
Completed:
* DHT support
The DHT module is pretty much complete, but as I haven't had regular
internet this summer it's relatively untested 'in the wild'. I'd
estimate at most 2 weeks of testing/usage in the real world before it
can be considered complete. It passes all 34 NUnit tests so far, so
i'm relatively confident it should be fine. The biggest problem is
likely to be issues with other non-compliant clients.
* Manual IP banning - Integration with existing IP range ban lists
This has been completed and has NUnit tests attached. It parses the
three most popular banlists as well as allows the user of the library
to ban any IPAddress they want.
* Extended Messaging Protocol (Azureus and LibTorrent)
It was decided that supporting the Azureus protocol was a waste of
time. Only the LibTorrent protocol is supported, and support for that
is complete.
* DBus service
This is still more of a proof of concept than a solid usable item.
Whilst it is possible to use the service to download torrents and
retrieve various statistics from those torrents, it is not advanced
enough to use as the default backend of an advanced torrent client.
* Banshee plugin to download torrents
A basic banshee plugin has been created to allow banshee to download
torrent based podcasts. I submitted this work to Gabriel Burt a few
days ago.
* Implement Http Seeding (Getright).
Support for this has been nearly completed. The only thing needing to
be added is support for multi-file torrents. This is a fairly trivial
thing to add - the hard thing was making the WebSeeding code use
standard BitTorrent messages when integrating with the library.
Basically, whenever the engine sends a requests a block of data to a
'WebSeeder', this request is translated into a standard HttpRequest.
When the data has been downloaded from the web server, a fake
'PieceMessage' (the bittorrent message which contains the block of
data) is returned to the engine. This way a Http server can be treated
*exactly* the same way as a regular bittorrent peer, meaning all the
existing infrastructure to deal with piece reading/writing, timing out
and verification can be used. I'm quite proud of this hack :)
* Implement peer exchange support.
This was completed solely by Olivier Defour, i did nothing for this
except 1 or 2 small bugfixes.
Not started:
* HTTP/SOCKS Proxy support was never started. I received an email
after blogging about it from a guy who did implement this support in
MonoTorrent. I aim to merge those changes into svn when i get back to
Ireland. It was too big a thing to take on at the moment.
* Expose the WebUI over the DBus service
The WebUI that was originally being worked on is no longer maintained.
I don't think I have the time to maintain it, so i've let this item
slide for the moment. If someone were to maintain the WebUI, it'd be
fairly easy to hook it into the DBus service.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
Monsoon:
Completed:
* NAT indicator - Display NAT status in a clear and understandable way
There is now a little indicator with 3 colours, red, yellow and green.
This colours indicate whether a upnp/nat-pmp capable router could be
found along with whether or not an incoming connection has been
detected.
Not Complete:
* Integrate the WebUI
As before, the WebUI isn't maintained, so i didn't integrate it.
* Port Monsoon to the DBus service
The DBus service isn't suitable (yet) to be used by Monsoon, so this
hasn't been completed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
Here are the major changes to MonoTorrent itself which have been
committed between 5 May and now. That's roughly when I started the SoC
work:
* Completed support for the Extended Messaging Protocol (LibTorrent).
Olivier submitted a few patches for this.
* Faster SHA1 Algorithm used by default. This decreased the time spent
hashing a large torrent by a significant amount of time (iirc about
30%). Some of these changes were rolled back into Mono.
* Fixed several bugs and race conditions in the Encryption code and
cleaned it up hugely. Initially this code had no NUnit tests, there
are now 15 NUnit tests.
* Initial implementation of WebSeeding completed by Olivier Defour.
I've extended this implementation and added several NUnit tests. This
is still disabled by default as support for multi-file torrents is
still lacking. The current NUnit tests fail because of this missing
support. However WebSeeding support is very close to be enabled by
default. I should even by able to do this with only a few hours of
work. Multi-file support would probably be the easiest feature to
code.
* Peer Exchange support completed by Olivier Defour and enabled by
default.
* Completely refactored the threading api to make it impossible to
deadlock and easier to extend. Everything now runs on a main loop,
similar to winforms/gtk. This was a fairly major change and took quite
a while, but things have become *drastically* simpler with this API.
It used to be difficult to add new functionality to anything which
touched the socket code. Previously, adding what I thought was a
simple check to abort async connections ended up deadlocking the
library on two separate occasions. With the new API, it's next to
impossible to deadlock the library - a big win.
* Implemented a way to cancel asyncronous connect requests as
sometimes they can take over 3 minutes to fail
* Added ability to change the listen port on the fly and be able to
gracefully handle when a port is busy.
* Added a whole new debug GUI written by Karthik Kailish and David
Sanghera.
* Committed a PiecePicker written by Karthik and David that allows you
to efficiently 'stream' a torrent.
* Implemented the necessary API to allow you to ban IP address ranges.
Can parse the three most popular banlist formats: emule, peerguardian
and safepeer
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
Other points of interest: Originally there were 55 NUnit tests in the
MonoTorrent testsuite. There are now 110 NUnit tests for MonoTorrent
and 34 NUnit tests for the DHT module. That's nearly triple the
tests :)
Finally, thanks for the opportunity to take part as a student again.
It was fun. I'm looking forward to enabling all these new features by
default for the next release :)
Alan.
Alan
Aug 16, 2008, 12:33:17 PM8/16/08
to Mono Summer of Code 2008
Just an update:
HttpSeeding (a.k.a. WebSeeding) is now enabled by default. I'm
awaiting verification that it performs as it's supposed to when tested
on a live http server. My NUnit tests all pass which is always a good
thing ;)
Alan.
HttpSeeding (a.k.a. WebSeeding) is now enabled by default. I'm
awaiting verification that it performs as it's supposed to when tested
on a live http server. My NUnit tests all pass which is always a good
thing ;)
Alan.
Reply all
Reply to author
Forward
0 new messages