File Chunking in SL Client / WCF Host

[Philipp Sumi]

Hi group

For my current pet project (which I hope to release in early January), I need to transmit files over the wire, and I want this to be possible with all sorts of clients - including Silverlight. I happily started my implementation using streaming and MessageContracts, but unfortunately, this is a dead end, even with SL4: There's neither support for message headers, nor streamed transfers (which I found quite surprising - SL is IMO a first class candidate for all kinds of streaming scenarios, more than ever with OOB and local file access). However, I'm really not the SL expert here, so maybe I missed a few bits.

As far as I can tell, the way to go is splitting files into parts, and sending them to the server (upload) or client (download) chunk by chunk. Apart from ensuring compatibility with SL, this will also simplify progress indication on the client and possible file transfer resumes (I'll probably save the latter for a later version). So I guess I have another side project on my hands :). However: I've seen quite a few examples on the web, but none has fit my needs so far. However, I don't really want to reinvent the wheel, so before I start writing my own: Did any of you work on a similar project and have a good starting point?

Cheers,
Philipp

--
code hard - http://www.hardcodet.net

[Sacha Barber]

Yeah I looked into this, and we did actually get the chunking approach working at work, but I can not share that with you. But as you say it started off by google, and we ended up with something that you could pass all you bytes to, and it did a recursive call to WCF sending new chunks each time, this did require us to persist the bytes read so far to disk though to allow us to reassemble them on each call.

We were using Instance PerCall WCF setting so had no choice but to persist bytes to disk between calls, included in the call to WCF was total byte number so we knew when we were done.

Streaming also worked well for us, but that was proper stuff, none of this SL jargon. Sorry group, I just dont dig SL, but you lot do, so I will just steal all this groups good work and use it.

MSMQ has also worked quite well for us, though I do not know if that streams? Cant quite remember.

Anyway that was my 2 cents.

--
Sacha Barber
sacha....@gmail.com

[Philipp Sumi]

Thanks Sacha - it's always good to know somebody did something similar and may know some of the hidden trap doors! I'm currently thinking about delegating all the responsibility for the transfers to a dedicated and reusable helper library (responsible for chunking/reassembling, maintaining transfer queues and cleanup), so both WCF services and SL/CLR clients would just forward the received bits in order not to be dependent on service instantiation strategies. I'll look into this.

                                                                                                                                                                                                                                                        

About SL: Yeah, I keep running into dead ends in areas where I really wouldn't expect them. Like yesterday when I discovered that RIA services don't support the DateTimeOffset properties of my data contracts. However, I guess what makes it annoying is also because SL already brings so much to the table that it nearly feels like working with the full .NET set. That's a good thing I guess :)

[Sacha Barber]

Philip this is pretty much the approach we took, its in our codebase, but I can not share it do to IP rights.

But you are not too far off what we did. I would love to see what you come up with, let us know when its done.

--
Sacha Barber
sacha....@gmail.com

[Philipp Sumi]

I'll have to decide whether to implement chunking, or take a RESTful route to get my data. I'll post back as soon as I have something to show for.

[Laurent Bugnion, GalaSoft]

Hey,

I needed something similar some time ago, and found this:
http://www.codeplex.com/SilverlightFileUpld

This project implements chunking on the client and on the server. It also
supports resuming an upload. I found it a good approach, however eventually
I went another way so I didn't end up using it.

Note that WCF services are not the best for file upload in chunks, because
of the overhead: Each call to the WCF service causes a SOAP response to be
sent and this causes delays. The project referenced above uses an ASHX
generic handler instead, which is the fastest possible service in .NET. I
would recommend this way for your implementation, should you decide to go
with your own.

Have fun!
Laurent

-----Original Message-----
From: wpf-di...@googlegroups.com [mailto:wpf-di...@googlegroups.com]

[Philipp Sumi]

Thanks for the link, Laurent! The library looks promising - haven't discovered that one so far.

Regarding WCF and perf: A REST based interface might actually be a serious alternative to web handlers, especially if I streamed the file data. But then, implementing a complementary ASHX handler isn't too much overhead either. I guess I'll have quite a bit of comparing to do in order to find the best solution :)