Response Stream Control

[Aldo Bassanini]

Hello everyone.
Is there any way to control the final status of the response.stream function?

I need to serve a small file to a client application (the client app uses wget, not a browser), but I wish to know if the client successfully downloaded the file, in order to update some server-side data.

The way that I am thinking to achieve this, is programming the client application (a shell script) to GET a second URL (sending a session-id or something like that) from the server, after downloading the file, but this means, that I have to develop some file validation logic to verifiy the correct downloading (md5suming? or alikes) in the client, that I prefer to avoid, since the client has a very small environment (OpenWrt's Busybox Shell) and very small CPU and RAM.


Thank's in advance and any suggestions will be appreciated

[Leonel Câmara]

Can't you just check what wget returns for errors? That said, the only way to be sure there was no corruption is using the hash method, I don't think that's too much for even that machine to do to be honest, you can use cksum instead of md5sum if md5sum is too resource intensive. If this is still too resource intensive you can make a small python script that calculates the hash in very small chunks this way your memory usage will be low for sure.

[Derek]

Well, if this is http, there really is no way to tell how much of a file was downloaded by a client, unless you write the webserver part yourself. Web2py never sees if a file is completely downloaded or not. Why are you using HTTP if you need to be absolutely sure that the file transferred? Have you looked at Zmodem or Kermit?

[Ricardo Pedroso]

If I understood correctly, you only need to know if the file was
successfuly downloaded
or not on the server side.

Don't know if there are better ways in web2py but...
...this code, that goes into a controller, should do the trick:

import os, sys

def abort(status_code, message=None):
raise HTTP(404, message)

def index():
filename = request.args(0) or abort(404)
fullpath = os.path.join(request.folder, 'uploads', filename)

def stream():
try:
with open(fullpath, 'rb') as f:
while True:
data = f.read(1024)
if not data:
break
yield data
except:
# with any error, assume file not fully downloaded
print >>sys.stderr, 'failed'
else:
# if you reach this point the file was successfully
# downloaded. Do what you need on the backend side.
print >>sys.stderr, 'done'

headers = {'Content-Type': 'application/octet-stream'}
raise HTTP(200, stream(), **headers)


PS: You dont need to do any checksum anywhere. TCP is reliable.
It guarantees that everything that is transmitted is error free and ordered.


Ricardo

[Derek]

TCP is not as reliable as you think. A half open connection can leave both sides confused, and then you have no way of knowing if your transfer got through or not. 

Generally, it does a good job, but when you absolutely need to know if a file has transferred successfully or not, you need something better than TCP.

[Leonel Câmara]

Even if TCP doesn't fail there could be corruption copying the file to disk. Hell a cosmic ray may flip a bit, it can happen. The only way to be sure is to verify.