serving up generated images

[erowan]

Hello,

I have a web app that dynamically generates images (png files). The
main html response page has links to these files

i.e.
<a href="/tmp/demand.png" title="/tmp/demand">
<img id="/tmp/demand" src="/tmp/demand.png" width="300"
height="200" />

I created the tmp dir under public and while running pylons in
development this worked ok. Basically these filepaths resolved to the
same location.

1. os.path.join(config['app_conf']['public_dir'], 'tmp', filename)
2. graph_file_name = h.url_for('/tmp/' + str(graph_file_name))

However, after packaging, install & deploying the app with the same
config file dev.ini

python setup.py bdist_egg
easy_install bmra_web-0.1.0dev-py2.5.egg
cd some_deployment_dir
paster serve dev.ini

The dirs are not the same. 1 is but 2 resolves under the ~site-
packages/bmra_web-0.1.0dev-py2.5.egg installation dir. I am not sure
how to handle this? Should I be using another StaticURLParser
instance for this tmp directory combined with Casade and maybe name it
_tmp (does this bypass the routes dispatching?).

I am a bit confused as to how to handle this. Thanks.

[erowan]

[jerry]

Hi,

You should find this thread helpful --

http://groups.google.com/group/pylons-discuss/browse_thread/thread/70fb565a909c72f0/aae0f836d23e62a2?lnk=gst&q=dataapp#aae0f836d23e62a2

Jerry

[Gael Pasgrimaud]

Hi,

You can also have a look at this: http://pypi.python.org/pypi/iw.thumbs

--
Gael

[erowan]

thanks, I'll take a look at these. I worked around it by creating a
soft link from by deploy dir to the install dir but this is not ideal.

bash-3.00$ pwd
/opt/energy/emd/bmra/web/bmraweb
bash-3.00$ ls -lt
total 4
lrwxrwxrwx 1 energy asgdev 102 Dec 8 13:40 public -> /san/apps/
energy/root/usr/local/lib/python2.5/site-packages/bmra_web-0.1.0dev-
py2.5.egg/bmraweb/public

On Dec 8, 10:36 pm, "Gael Pasgrimaud" <g...@gawel.org> wrote:
> Hi,
>
> You can also have a look at this:http://pypi.python.org/pypi/iw.thumbs
>
> --
> Gael
>

> On Mon, Dec 8, 2008 at 11:23 PM, jerry <jerryji1...@gmail.com> wrote:
>
> > Hi,
>
> > You should find this thread helpful --
>

> >http://groups.google.com/group/pylons-discuss/browse_thread/thread/70...

[Matthew Zwier]

Hello,

Do you need the graphics files to remain around for a while, or are
they dynamically generated for one request? If the former, the
approach being discussed here makes a lot of sense. The files you
create will be around until you delete them yourself. If the latter
(read data from constantly-updated source, make graph, put on page --
that sort of thing), you may be better suited rendering the graphics
into a temporary file (as created with tempfile.NamedTemporaryFile or
whatever) and streaming it to the client yourself.

# at the top of the controller file
import tempfile

class ChunkedFileIter(object):
"""An iterator interface which reads chunks of a given size from a file
until the file ends.
"""

def __init__(self, fileobj, blocksize = 8192):
"""Initialize this object.

:Parameters:
fileobj : object supporting ``read(blocksize)``
File object from which to read
blocksize : integer
Number of bytes to read in each iteration
"""
self.fileobj = fileobj
self.blocksize = blocksize

def __iter__(self):
return self

def next(self):
"""Read ``self.blocksize`` bytes from ``self.fileobj``. Raises
``StopIteration`` when the file ends."""
dat = self.fileobj.read(self.blocksize)
if not dat:
raise StopIteration
return dat


# in your action:
tf = tempfile.NamedTemporaryFile(suffix='.png')
# Write graphic into the file named by tf.name
makeGraphic(tf.name)
# tf.flush() # <-- may be necessary if you write the file yourself,
as opposed to an external program
tf.seek(0)
response.content_type='image/png'
response.app_iter = ChunkedFileIter(tf)

Using the NamedTemporaryFile ensures that the file is deleted after
the request ends. You can then instruct the browser to cache its copy
of the file (using etag_cache()) so you don't regenerate it unless the
source data has changed or the client requests a new copy explicitly.

That totally won't help if you need those PNGs laying around for
future use...but as it took about five minutes of cut-and-paste...if
it does help...great!

Cheers,
Matt Z.

[erowan]

Hello Matt,

Thanks for your response. The page I serve has a graph plus a link to
a csv file of the graph data so I can't dynamically return the data as
you suggest. The page looks like this.

<html><title>demand</title></head><body>

<a href="/tmp/demand.png" title="/tmp/demand">
<img id="/tmp/demand" src="/tmp/demand.png" width="300"
height="200" />

</a></br>
<a href="/tmp/demand__2008-10-29_2008-10-30.csv">csv</a></br>
</body></html>

Using tempfile.NamedTemporaryFile specifying this tmp directory could
be an improvement if the deletion works ok. If I don't explictly close
the file the docs say they will be closed and therefore deleted with
gc is run.

[Matthew Zwier]

Hi,

> Using tempfile.NamedTemporaryFile specifying this tmp directory could
> be an improvement if the deletion works ok. If I don't explictly close
> the file the docs say they will be closed and therefore deleted with
> gc is run.

Hmm...because gc can be run at any time, there's no guarantee the temp
files will still be around by the time the browser gets around to
retrieving them. It may work, but it sounds a bit fragile. Your idea
about StaticURLParser and Cascade sounds elegant, and it should bypass
routes dispatching unless you insert a route mapping which matches
/tmp or /_tmp or whatever you call it.

MZ