Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
How to create and serve a .zip file using data from database?
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  4 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
mrtn  
View profile  
 More options Oct 7 2012, 1:32 pm
From: mrtn <mrtnl...@gmail.com>
Date: Sun, 7 Oct 2012 10:32:01 -0700 (PDT)
Local: Sun, Oct 7 2012 1:32 pm
Subject: How to create and serve a .zip file using data from database?

Inside the get method of a handler, I would like to do the following:

1. query some data store, and get the data
2. create a bunch of .csv files using the data
3. create a .zip file containing all those .csv files
4. serve this .zip file to the client

In addition, I have no need to store .csv or .zip files, so would prefer
that by the end of the request there is no file left on the server.

I wonder how should I optimally achieve steps 2, 3, 4 here in Tornado?
Thanks!


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
A. Jesse Jiryu Davis  
View profile  
 More options Oct 7 2012, 8:02 pm
From: "A. Jesse Jiryu Davis" <ajesseda...@gmail.com>
Date: Sun, 7 Oct 2012 17:02:41 -0700 (PDT)
Local: Sun, Oct 7 2012 8:02 pm
Subject: Re: How to create and serve a .zip file using data from database?

While it may not be the most efficient implementation, I think a good start
is to create a StringIO for each file and pass it as the file object to
csv.writer():

http://docs.python.org/library/csv.html#csv.writer

Once you've written all the CSVs to StringIOs, create a ZipFile with
another StringIO as its output file:

output = String()
zip = zipfile.ZipFile(output, 'w')

and write each CSV to the zip archive with writestr(my_stringio.getvalue(),
...):

http://docs.python.org/library/zipfile.html#zipfile.ZipFile.writestr

And finalize the ZipFile with close(). Finally:

self.set_header('Content-Type, 'application/zip')
self.write(output.getvalue())
self.finish()

Obviously, this requires having several copies of your data in memory at
once. If this isn't a problem then don't sweat it. If it is, you could
experiment with streaming results into the zipfile, which would save some
overhead, and more interestingly you could try streaming results *out* by
passing self.connection.stream into the ZipFile constructor as its output
file. If that works then you wouldn't have to keep much data in memory at
all.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
mrtn  
View profile  
 More options Oct 8 2012, 7:44 am
From: mrtn <mrtnl...@gmail.com>
Date: Mon, 8 Oct 2012 04:44:46 -0700 (PDT)
Local: Mon, Oct 8 2012 7:44 am
Subject: Re: How to create and serve a .zip file using data from database?

Thanks! That's more or less what I've ended up with.

The only difference is that I don't create a StringIO object for each .csv
file, but use a single StringIO object for the zipfile object (passing it
into the constructor), and then I just do:

zf.writestr('file1.csv', content1)
zf.writestr('file2.csv', content2)
zf.writestr('file3.csv', content3)
...

The self.connection.stream approach sounds interesting, but what would you
pass into self.write() method when you flush the data to the browser client?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
A. Jesse Jiryu Davis  
View profile  
 More options Oct 9 2012, 10:45 am
From: "A. Jesse Jiryu Davis" <ajesseda...@gmail.com>
Date: Tue, 9 Oct 2012 07:45:59 -0700 (PDT)
Local: Tues, Oct 9 2012 10:45 am
Subject: Re: How to create and serve a .zip file using data from database?

The idea is you don't call self.write, but instead let ZipFile do the writing. On second thought this isn't going to work because you need to set a content-length header before you can start writing, so I'm not sure the best way to do what I'm thinking of.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »