seemingly inconsistent path resolution from <%inherit/> when running mako-render

[Christopher the Magnificent]

Hello everyone!

I've got Mako syntax highlighting up and running in Komodo Edit on my Mac and I'm excited to get some serious work done in Mako.

I imagine I'll be making the rendering calls to Mako from Python if I continue my plan to use Mako with Django or something.  For now, however, I've just been executing the mako-render command-line script utility to do my rendering.  In so doing, I seem to have stumbled upon some path resolution behavior in Mako that seems inconsistent from one file to another.  It may just be another documented behavior that I've overlooked, but it seems rather odd to me.

Here's the situation.

I have three Mako template files. all contained somewhere within a folder called "Mako".  The first file is called "base.mako" and is directly inside the Mako folder.  The second two files live in a folder called "login_base" which itself is directly inside the Mako folder.  Here's an ls -R output for you:

mako: christopher$ ls -R .

base.mako  login_base

./login_base:

login.mako      login_base.mako

The "login.mako" template is the starting point; it is not used inside of any other template.  I want it to inherit from "login_base.mako", and I want "login_base.mako" to inherit from "base.mako".

This way of organizing files hierarchically may not work for everyone, but it occurred to me this morning, and I liked it at the time, so I went with it.

Now, I want to run mako-render with "mako" as the current directory like this:

mako: christopher$ mako-render login_base/login.mako

I tried fussed and fussed with the <%inherit .../> tags until I finally got it to work (didn't get exceptions thrown because it couldn't find my files), and the result didn't make much sense to me.  This shows the inherit tags and file attributes I had to use to make it work:

"login.mako": 

<%inherit file="login_base/login_base.mako"/>

"login_base.mako":

<%inherit file="../base.mako"/>

"base.mako":

(doesn't inherit)

And it worked when I ran the above mako-render command from inside my "mako" folder.

To me, this says that the file attribute I specified in the <%inherit/> tag in "logo.mako" was interpreted as relative to the current working directory ("mako"), while the file attribute in the <%inherit/> tag inside "login_base.mako" was interpreted as relative to the directory containing "login_base.mako" ("login_base").

What is more, when I inserted some debug code into Mako itself into "/Library/Python/2.7/site-packages/Mako-0.7.2-py2.7.egg/mako/runtime.py" into the _lookup_template() function like this:

def _lookup_template(context, uri, relativeto):

    lookup = context._with_template.lookup

    # CTM mod:

    print "_lookup_template:", __file__

    print "    uri initial value is", repr(uri)

    print "    relativeto is", repr(relativeto)

    print "    lookup is", repr(lookup)

    print "    lookup.filesystem_checks is", repr(lookup.filesystem_checks)

    # end CTM mod

    if lookup is None:

        raise exceptions.TemplateLookupException(

                            "Template '%s' has no TemplateLookup associated" %

                            context._with_template.uri)

    uri = lookup.adjust_uri(uri, relativeto)

    try:

        return lookup.get_template(uri)

    except exceptions.TopLevelLookupException, e:

        raise exceptions.TemplateLookupException(str(e))

And when I ran "mako-render" as described above, I observed the following debug displayed above the output of my templates (color added):

_lookup_template: /Library/Python/2.7/site-packages/Mako-0.7.2-py2.7.egg/mako/runtime.py

    uri initial value is u'login_base/login_base.mako'

    relativeto is 'memory:0x1027ea9d0'

    lookup is <mako.lookup.TemplateLookup object at 0x1027ea950>

    lookup.filesystem_checks is True

_lookup_template: /Library/Python/2.7/site-packages/Mako-0.7.2-py2.7.egg/mako/runtime.py

    uri initial value is u'../base.mako'

    relativeto is u'login_base/login_base.mako'

    lookup is <mako.lookup.TemplateLookup object at 0x1027ea950>

    lookup.filesystem_checks is True

Is it wrong that on the first invokation of _lookup_template the argument "relativeto" is equal to the byte string 'memory:0x1027ea9d0', especially considered that on the next invocation it's a Unicode string instead?  That seems a bit strange to me.

So is this incorrect or buggy behavior, or did I miss something again?  :-)

Thanks,

Christopher

[Michael Bayer]

On Aug 17, 2012, at 4:40 PM, Christopher the Magnificent wrote:

I tried fussed and fussed with the <%inherit .../> tags until I finally got it to work (didn't get exceptions thrown because it couldn't find my files), and the result didn't make much sense to me.  This shows the inherit tags and file attributes I had to use to make it work:

"login.mako": 

<%inherit file="login_base/login_base.mako"/>

"login_base.mako":

<%inherit file="../base.mako"/>

"base.mako":

(doesn't inherit)

And it worked when I ran the above mako-render command from inside my "mako" folder.

here's what is cleaner, forgetting about mako-render for the moment:

login.mako:  <%inherit file="login_base.mako"/>

login_base.mako: <%inherit file="/base.mako"/>

The values inside of "file" are URIs.  These URIs serve as string keys in the TemplateLookup collection.  That they happen to line up with paths on your filesystem is an implementation detail of how the template was placed into the TemplateLookup.

If the path of the URI specified in "file" does not start with a slash, it is relative to the URI of that template.

If the path is absolute, then it is taken as is.

When the URI is converted to a filesystem path in order to locate a file to generate a new Template for that URI, the URI is tested against each entry in the "directories" argument of the TemplateLookup.   When you use the mako-render script, "directories" is passed as ["."].   That is, the current working directory.  The script would do well to allow this as a configurable option.

Is it wrong that on the first invokation of _lookup_template the argument "relativeto" is equal to the byte string 'memory:0x1027ea9d0',

Yeah probably, the filename isn't being passed into the Template() inside of mako-render so that top level Template isn't given context to know how to figure out its relative URI.

especially considered that on the next invocation it's a Unicode string instead?  That seems a bit strange to me.

So is this incorrect or buggy behavior, or did I miss something again?  :-)

both.  its a bug in mako-render and also I'd advise not stressing the mako-render script very much, as it's only a quick and dirty test script.  it's not used in framework integrations. the patch below should produce more predictable behavior.

--- a/scripts/mako-render Fri Aug 03 12:26:08 2012 -0400

+++ b/scripts/mako-render Fri Aug 17 17:04:14 2012 -0400

@@ -1,11 +1,11 @@

 #!/usr/bin/env python

 

-def render(data, kw):

+def render(data, filename, kw):

     from mako.template import Template

     from mako.lookup import TemplateLookup

 

     lookup = TemplateLookup(["."])

-    return Template(data, lookup=lookup).render(**kw)

+    return Template(data, filename=filename, lookup=lookup).render(**kw)

 

 def varsplit(var):

     if "=" not in var:

@@ -40,7 +40,7 @@

 

     kw = dict([varsplit(var) for var in opts.var])

     data = fo.read()

-    print render(data, kw)

+    print render(data, filename, kw)

 

 if __name__ == "__main__":

     main()

[Christopher Johnson]

That patch for mako_render worked like a charm, and now the paths seem to work consistently.  

I just want to confirm that I understand the part about absolute and relative paths.  If it doesn't start with a slash, it's relative to the path of the template file containing the reference.  Otherwise if it starts with a slash, it's searched in the list of directories, (including the current directory where mako_render was run, if Mako was invoked form mako_render).  Is this right?

Thanks for the help.

--Christopher

--
You received this message because you are subscribed to the Google Groups "Mako Templates for Python" group.
To post to this group, send email to mako-d...@googlegroups.com.
To unsubscribe from this group, send email to mako-discuss...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mako-discuss?hl=en.

[Michael Bayer]

On Aug 17, 2012, at 5:59 PM, Christopher Johnson wrote:

That patch for mako_render worked like a charm, and now the paths seem to work consistently.  

I just want to confirm that I understand the part about absolute and relative paths.  If it doesn't start with a slash, it's relative to the path of the template file containing the reference.  Otherwise if it starts with a slash, it's searched in the list of directories, (including the current directory where mako_render was run, if Mako was invoked form mako_render).  Is this right?

the given URI is always converted to an absolute URI first, using the source file as the source of what's "relative" (hence the relativeuri argument to _lookup_template).   then the absolute URI is tested against the filesystem paths in "directories".