Cleaning up files generated for user session

[Greg Slodkowicz]

Hi all,
I'm working on a Pyramid application where I need to serve plots
generated on the fly with an external application (R). For now I have
R drop a png file with the plot to ./static but I'm having trouble
disposing of it once it's no longer needed. Is there any way to have a
session cleanup hook? I would guess the solution would involve event
handlers somehow but I couldn't find anything appropriate in the
documentation.

Best,
Greg

[Juliusz Gonera]

Do you need them deleted as soon as the session expires? If not, I
guess the easiest way would be to set up a cron job which would delete
all the files older than something every day/hour/whatever.
Or if it's a small app that won't have to scale, you could even do it
in some view callable (not very elegant though).

--
Juliusz Gonera

[Jonathan Vanasco]

I've done something similar in the past.

1. I suggest expanding the logic a bit- log the filename , along with
timestamp and session id
2. have a cronjob that deletes files associated with sessions that
were last used outside a given window

session cleanups within an app tend to only happen if someone
explicitly clicks "logout", and thats rarely even a single digit
percentage of most apps. the overwhelming majority of people just
don't 'continue' using the session -- so you need logic to handle that.

[Greg Slodkowicz]

Oh, I see. For no good reason I assumed the cleanup should be done by
Pyramid but a cronjob obviously also works. Thanks to both of you.

-Greg

[Wyatt Baldwin]

Do you need to save temp files at all? Are the PNGs expensive to generate in R? Can you request raw bytes from R instead of saving to file?

I have a scenario that's similar to yours (not using R, though; I'm generating audio files), and I stream the bytes generated by the back end directly to the client with the appropriate content-type header.

You could also cache the results using Beaker if generating the PNGs is expensive.

[Greg Slodkowicz]

> Do you need to save temp files at all? Are the PNGs expensive to generate in
> R? Can you request raw bytes from R instead of saving to file?

> I have a scenario that's similar to yours (not using R, though; I'm
> generating audio files), and I stream the bytes generated by the back end
> directly to the client with the appropriate content-type header.

I don't need to save them and I don't think I need caching. It should
be possible to make are write to a memory buffer but I'm not sure if I
see how to send the data directly to the client. Would I need an AJAX
call for that?

Best,
Greg

[Mike Orr]

No. Just make an ordinary view that returns the image from a string
(rather than from a file). You don't need to "stream" it in the sense
of sending part of it at a time, you can just send the whole thing at
once.

This is really the same as "Registering a View Callable to Serve a
'Static' Asset" in the manual.

http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/narr/assets.html#registering-a-view-callable-to-serve-a-static-asset

===
import os
from pyramid.response import Response

def favicon_view(request):
here = os.path.dirname(__file__)
icon = open(os.path.join(here, 'static', 'favicon.ico'))
return Response(content_type='image/x-icon', app_iter=icon)
===

You can modify this to read a temporary file stored outside 'static',
or to read an image string stored in the session. The content type
would be "image/png".

You can also use Python's "mimetypes" module to guess the type from
the content, although the API requires a filename rather than a
string. (You could pass a StringIO object.)

Serving static files via ordinary views is also useful when you need
to check permissions; i.e., when only certain users should see certain
files.

--
Mike Orr <slugg...@gmail.com>