Link to files stored in data directory

[Joel]

Hey Pylons people,

I'm trying to add file upload functionality to a Pylons project and
I'm following the Pylons Book example in chapter 6. Uploading the
file to a subdirectory ('uploads') of the data directory works fine.
Following the example, I set the permanent_store variable in
development.ini as:

permanent_store = %(here)s/data/uploads

and reference it in my create() action as:

config['app_conf']['permanent_store']

The problem arises when I want to reference my newly uploaded file in
a Mako template html <img> or <a> tag. If I use the value of
permanent_store as set in the config file, I get an absolute path and
that results in a 404 File Not Found error. That is, if I enter

<%! from os import sep, path %>
<% pathName = os.path.join(config['app_conf']['permanent_store'],
file.name) %>
<img src="${pathName}" />

the output is:

<img src="/home/MyApp/myapp/data/uploads/filename.png" />

which generates a 404 File Not Found error.

My solution is to move my uploads directory to the public directory,
i.e., by changing the line in the config file to:

permanent_store = %(here)s/myapp/public/uploads

and change my template code to:

<%! from os import sep, path %>
<% pathName = path.join(sep + config['app_conf']
['permanent_store'].split(sep)[-1], file.name) %>
<img src="${pathName}" />

This works. But I'm wondering whether I should be storing my user-
uploaded files in a subdirectory of the data directory for some
reason. And if I should be, how can I reference those files from my
Mako templates?

I remember from previous projects using PHP & Apache that storing
files in a subdirectory of public_html would render the files publicly
available. Access to my site will require authentication and I do not
want the files to be publicly accessible. Do I have to worry about
this if my uploaded files are stored in public/uploads?

Any help much appreciated,

Joel

[Jonathan Vanasco]

i store the files into a top-level directory called
"user_uploads" ( ie. on the same level as data and public )

i do this, only because i find it easier to handle backups /
redundancy

when the images are 'public', i have a symlink from public/_img/NAME
into that directory. just to note, i never have pylons serve these
files - i have nginx serve those directories itself.

and when files are not public, i can still access them with pylons too.

[Mike Orr]

On Sun, Mar 7, 2010 at 1:29 AM, Joel <jrwd...@gmail.com> wrote:
> This works.  But I'm wondering whether I should be storing my user-
> uploaded files in a subdirectory of the data directory for some
> reason.  And if I should be, how can I reference those files from my
> Mako templates?
>
> I remember from previous projects using PHP & Apache that storing
> files in a subdirectory of public_html would render the files publicly
> available.  Access to my site will require authentication and I do not
> want the files to be publicly accessible.  Do I have to worry about
> this if my uploaded files are stored in public/uploads?

Serve them from an action using FileApp or DirectoryApp.

# Route
map.connect("file", "/my_url/{path:.*}", controller="mycontroller",
action="my_action")

# Controller
from paste.fileapp import FileApp
from pylons.controllers.util import forward
def my_action(self, path, environ, start_response):
# Do authorization, abort(404) or abort(403) if disallowed.
path = os.path.join(config["permanent_store"], path)
app = FileApp(path)
return forward(app)

'app' is a WSGI application. 'forward' is a utility which delegates
to it. The ":.*" in the path variable matches slashes (which normally
aren't matched) if you're using subdirectories.

Don't put anything into the public directory unless it's truly public.
And I also wouldn't put user-uploaded material there because I think
of it as part of the application (i.e., unchanging, version
controlled).

Whether to put it inside the data directory depends. I do this in one
application. But you have to remember it's there, and that you can't
just blow away the data directory whenever you want to clear the
sessions/logs/compiled templates. If you don't have a better place on
your server for it, you can put it in the data directory.

--
Mike Orr <slugg...@gmail.com>

[Mike Orr]

On Sun, Mar 7, 2010 at 9:42 PM, Mike Orr <slugg...@gmail.com> wrote:
> def my_action(self, path, environ, start_response):

Ignore the last two arguments. They were needed in older versions of
Pylons before the 'forward' utility existed.

--
Mike Orr <slugg...@gmail.com>

[Joel]

Thanks Mike!

FileApp and forward did the trick. I ended up storing my user-
uploaded files in a subdirectory ("files") of my Pylons project root
directory. And I added my own authentication decorators to both the
view() and retrieve() actions of FileController so that
unauthenticated guests cannot access the files.

My one reservation is that I don't really understand the routes
configuration that you suggested:

# Route
map.connect("file", "/my_url/{path:.*}", controller="mycontroller",
action="my_action")

But that's understandable seeing as I haven't yet learned more than
the basics of the Routes system.

I ended up using the following, which seems to work fine for my
purposes (I guess because I am storing all my files in the "files"
directory without any subdirectories):

# Route
map.connect('/file/retrieve/{path}', controller='file',
action='retrieve')

And I reference my files from within my Mako templates via:

${h.url_for(controller='file', action='retrieve', path=fileName)}

Thanks again for the help,

Joel

On Mar 7, 9:47 pm, Mike Orr <sluggos...@gmail.com> wrote:

> On Sun, Mar 7, 2010 at 9:42 PM, Mike Orr <sluggos...@gmail.com> wrote:
> > def my_action(self, path, environ, start_response):
>
> Ignore the last two arguments. They were needed in older versions of
> Pylons before the 'forward' utility existed.
>
> --

> Mike Orr <sluggos...@gmail.com>

[Mike Orr]

On Tue, Mar 9, 2010 at 12:28 AM, Joel <jrwd...@gmail.com> wrote:
> My one reservation is that I don't really understand the routes
> configuration that you suggested:
>
>  # Route
>  map.connect("file", "/my_url/{path:.*}", controller="mycontroller",
> action="my_action")
>
> But that's understandable seeing as I haven't yet learned more than
> the basics of the Routes system.

The first argument is the route name, which can be used in generation.
It's optional. The second is the route path. They keyword args are
hardcoded variables attached to the route.

This route will match URLs like "/my_url/a.txt" and
"/my_url/subdir/a". The match dict returned to Pylons is {"path":
"a.txt", "controller": "mycontroller", "action": "my_action"}. Pylons
uses the controller and action variables to determine which class and
method to call. Pylons also examines the method's arguments, and
passes any routing variables that have the same name ('path' in this
case).

> I ended up using the following, which seems to work fine for my
> purposes (I guess because I am storing all my files in the "files"
> directory without any subdirectories):
>
>  # Route
>  map.connect('/file/retrieve/{path}', controller='file',
> action='retrieve')

This matches "a.txt" but not "subdir/a.txt", because the default regex
does not match a slash. The route also has no name.

> And I reference my files from within my Mako templates via:
>
>  ${h.url_for(controller='file', action='retrieve', path=fileName)}

That works but it's the old way of doing it. 'h.url_for' is being
phased out in favor of 'url'. If you're following the Pylons book it
still uses 'h.url_for' I think.

It's also generally better to generate routes by name. E.g., if the
route is named "file":
url("file", path=fileName)
This ensures you get exactly the route with that name, and not some
other route that happens to have compatible variables.

--
Mike Orr <slugg...@gmail.com>