Best practice for server-generated downloads?

[DJ-Tom]

Hi,

I need to create database report downloads in Excel format (via xlsxwriter) and I'm wondering if there is any standard or best practice as to where those downloads should be located.

Especially helpful would be if there was a portable way of managing the file system location and web request url in a way so that I don't have to change settings between the development and production server.

Is this -> https://docs.djangoproject.com/en/1.5/topics/files/ the way to go?

Any ideas?

Thomas

[Russell Keith-Magee]

If they're sever generated, why do they need to hit the file system at all? 

The following example in the docs:

https://docs.djangoproject.com/en/1.5/howto/outputting-pdf/

shows how you can stream a report directly to the end user. The example uses reportlab to produce a PDF, but the same approach will work for a tool writing to a different format.

Yours,

Russ Magee %-)

 

[DJ-Tom]

Hi,

this is quite easy to answer - those reports need to be archived - users must be able to download them days or weeks later.

So I will create a record in a database for each report that is created so users can access it later - maybe this can be seen as "uploading" a file to the server that acts as a document repository, with the only difference that the file is not uploaded, but produced by the server.

I don't see how I could use the xlsxwriter object in the way it is described here https://docs.djangoproject.com/en/1.5/howto/outputting-pdf/ - how could I pass the httpresponse object to xlsxWriter?
(Maybe I have not yet found how this might work - but it is not what I need anyways...)

Thomas

[Russell Keith-Magee]

On Wed, Oct 2, 2013 at 2:33 PM, DJ-Tom <event...@gmail.com> wrote:

Hi,

this is quite easy to answer - those reports need to be archived - users must be able to download them days or weeks later.

So I will create a record in a database for each report that is created so users can access it later - maybe this can be seen as "uploading" a file to the server that acts as a document repository, with the only difference that the file is not uploaded, but produced by the server.

Well, if you need archival, there are two alternatives:

1. Write the file to disk at the same time it is being generated. There's no reason you have to serve the file from where it was generated -- generate it, write it, and serve it.

2. Make sure the report will always be generated the same way. An analog here -- if you request a page from a web server, it probably isn't saved on disk like that (unless it's a genuinely static page) -- the server knows how to reproduce the same page every time you request a specific URL. Make the reports the same -- if you request /report/September-2013, you generate the same report every time. 

The second approach depends on whether you have well time-bucketed data, but it's certainly possible to do.

I don't see how I could use the xlsxwriter object in the way it is described here https://docs.djangoproject.com/en/1.5/howto/outputting-pdf/ - how could I pass the httpresponse object to xlsxWriter?
(Maybe I have not yet found how this might work - but it is not what I need anyways...)

I haven't used xlsxwriter myself, but the key part of the PDF example is that StringIO is an object that adheres to the python File API, but doesn't actually involve a file. So, you open a StringIO object, "write" to it, then dump the contents as the HTTP response. 

So - whatever API endpoint on xlsxwriter lets you pass in a file object -- pass in a StringIO instance instead.

Yours,

Russ Magee %-)

Thomas

Am Mittwoch, 2. Oktober 2013 02:11:21 UTC+2 schrieb Russell Keith-Magee:

On Tue, Oct 1, 2013 at 9:49 PM, DJ-Tom <event...@gmail.com> wrote:

Hi,

I need to create database report downloads in Excel format (via xlsxwriter) and I'm wondering if there is any standard or best practice as to where those downloads should be located.

Especially helpful would be if there was a portable way of managing the file system location and web request url in a way so that I don't have to change settings between the development and production server.

Is this -> https://docs.djangoproject.com/en/1.5/topics/files/ the way to go?

If they're sever generated, why do they need to hit the file system at all? 

The following example in the docs:

https://docs.djangoproject.com/en/1.5/howto/outputting-pdf/

shows how you can stream a report directly to the end user. The example uses reportlab to produce a PDF, but the same approach will work for a tool writing to a different format.

Yours,

Russ Magee %-)

 

--

You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/93413618-01d4-4fc0-a4d0-94fa67efe371%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[Jirka Vejrazka]

Hi Thomas,

 I'm doing exactly this - allowing users to download (graphically simple) XLSx files from my web app, it also needs to be available for download weeks later.

  As Russell pointed out, you don't need to store data on this if you have a way of getting the same set of data later, you can always regenerate the same Excel file. You don't even need to use StringIO for that.

 I'm using the save_virtual_notebook() in openpyxl package to generate the Excel file straight from database upon each (infrequent) request.

  HTH

    Jirka

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJxq84-kJ%2BNhxt43b_ighenWZGErpHvPRE3q53BGrDczckFPyg%40mail.gmail.com.

[François Schiettecatte]

Hi

+1 on this except that I use xlrd and xlwt to read and write Excel files. To generate the excel file I just generate a list of lists (an array effectively) and pass that through a function which walks the list and generates the excel worksheet that I write to a StringIO so it can be returned to the client.

François

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFhEBEARVBt%3DQMavyq3gt8XaVXvmmeAH_CKT3jcyPS2e8h8MTQ%40mail.gmail.com.

[Timothy W. Cook]

I tried to return a generated file but it seems that the Admin interface (or middleware?) is capturing my response. 

I get a brief blank page and then the Admin interface again without eve getting the download prompt. 

My last attempt looked like this:

    fname = 'ccd-'+self.ct_id+'.xsd'

    response = HttpResponse(ccd_str, content_type='application/xml')

    response['Content-Disposition'] = 'attachment; ' + fname

    return response

Where ccd_str contains the text (an XML Schema) which is also written to the database.  I did try a few other approaches according to the documentation and entries on StackOverflow.  

Can this be dome using the Admin interface?  Any ideas?

Thanks,

Tim

--
MLHIM VIP Signup: http://goo.gl/22B0U
============================================
Timothy Cook, MSc           +55 21 94711995
MLHIM http://www.mlhim.org
Like Us on FB: https://www.facebook.com/mlhim2
Circle us on G+: http://goo.gl/44EV5
Google Scholar: http://goo.gl/MMZ1o
LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook

[John McNamara]

Hi,

Here is an example of using XlsxWriter from SimpleHTTPServer or Django. It probably doesn't do 100% of what you are looking for but it may help anyway.

    https://xlsxwriter.readthedocs.org/en/latest/example_http_server.html

John

[DJ-Tom]

The report can't be reporduced later on - as the data is constantly changing, so I need to store the result.

Storing the data on which the report is based is very complex and would be too expensive regarding storage space just for this purpose.

But I also have now found how to use StringIO and xlsxWriter here:

http://stackoverflow.com/questions/16393242/xlsxwriter-object-save-as-http-response-to-create-download-in-django

So most likely I will do it like in this sample, plus saving the report for later retrieval and always serving through httpresponse, so I don't need to expose the actual storage location via the web server.

[DJ-Tom]

I have now created a test view that basically does what I want:

def reporting_new(request):

    output = StringIO.StringIO()

    book = Workbook(output)
    sheet = book.add_worksheet('test')
    sheet.write(0, 0, 'Hello, world!')
    book.close()

    # construct response
    output.seek(0)
    response = HttpResponse(output.read(), mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
    response['Content-Disposition'] = "attachment; filename=test.xlsx"

    return response

I could now use output:getvalue() to write it to disk - but how do I redirect the browser to the next page after the download?

Thomas

[DJ-Tom]

Thanks - that's what I was looking for.

But how do I send the browser back to a specific page after serving the download?