Intercept file transfer with dummy file and save original file locally - possible?

[Felix Nielsen]

hi

I have a client server system that transfer files over TCP, I am trying figure out a way to transfer the file over UDP instead (due to bad latency and dealing with very big files)

I am not sure how to do it - if possible at all - but I am thinking if I could save the file locally and only transfer a single byte to the server - then the operation completes rather quickly in the client. I could the transfer the actual file with some UDP transfer tool and on the server "update" the file with the proper content.

Would this be possible with a Mallory plugin?

Thanks

[Rajendra Umadas]

I am a bit confused by the problem. Some more details may help. 

Raj

--
You received this message because you are subscribed to the Google Groups "Mallory Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mallory-prox...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Felix Nielsen]

Hi Raj,

Yes of course and thanks for replying :)

This is the challange :

I have a scm client server system where the client is on a very bad latency link and they need to submit/upload some very large files. The system is using TCP/IP and it takes for ever to submit/upload the file.

So I am thinking that if I could somehow transfer the files behind the "scenes" with an UDP based transfer protocol, then it should be much faster.

If that is the case, I need to be able to intercept the original submit/upload process - so I have the original file for the alternative transfer - and that the file is actually registered in the original client/server system.

So if possible at all - intercept file, and maybe replace all incomming packages with a dummy/smaller package or even a whole other file size.

After the alternative transfer, I can "update" the original file with the actual file.

Does it make sense at all :)

Thanks

Felix

[Felix Nielsen]

Hi Raj - any feedback?

thx

[Rajendra Umadas]

Sorry... Been playing in the snow. :) Ill type up a response tonight. 

On Tue, Jan 21, 2014 at 5:54 PM, Felix Nielsen <felix....@gmail.com> wrote:

Hi Raj - any feedback?

thx

[Rajendra Umadas]

So i feel that a mallory plugin could do this. However, using mallory for this process would be a bad idea. It is really meant for debugging and testing. Long term use of mallory might not be robust enough for production things. 

Alternatively, there may be better iptables/python tricks to make this happen without using mallory. The exact proxying part of mallory is only like 30 lines of code. Making a 1 off to accomplish this task shouldnt be hard. With that said, this solution to the problem seems like it will lead to trouble. 

Im assuming there is no way to configure the client to use UDP. Also, do you feel that moving over to a UDP link will actually speed things up AND reliably send over the file? Have you considered local storage of the large files on the clients into a directory that uses rsync to xfer files. That may be a more elegant solution. 

There are just some thoughts, what do you think?

Raj

[Felix Nielsen]

Hi Raj,

Thanks for your reply, yes the client can't use UDP. The UDP approach - with a sync system that can handle errors etc - should make a big difference in transferring lots of data over a high latency line.

I know this could lead to trouble, but I am seeking out different options :)

Thanks again :)

Felix

[Rajendra Umadas]

Check into rsync