Improving the template language

[Sam Minnee]

A number of people in the community have requested that we make the
templating language more powerful. My initial response to this has
been that we've deliberately kept the template language simple. We
want people to be able to download modules and be able to customise
the templates without knowing PHP.

However, perhaps it's time for a change. Restricting the template
language's syntax isn't the best way to ensure simple templates;
rigourous review would be of more benefit.

To move forward with this, I think we need to examine the kinds of
improvements that people thing would be warranted. We don't want to
build everything and the kitchen sink, but the template language would
probably benefit from the judicious inclusion of a few syntax
improvements.

So - what do people think we should add?

[Nicolaas Thiemen Francken]

Keep it simple Sam... The logic should be in the php - not in the templates!
This requires more skill from the developer but makes for better sites.

Cheers

Nicolaas Thiemen Francken
Director Sunny Side Up Ltd
skype: nicolaasthiemen
within NZ phone 0800 771 777
overseas call +64 274 771 777
n...@sunnysideup.co.nz
www.sunnysideup.co.nz

> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.503 / Virus Database: 269.16.10/1159 - Release
> Date: 29/11/2007 11:10 a.m.
>
>

No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.10/1159 - Release Date: 29/11/2007
11:10 a.m.

[Sam Minnee]

My thoughts:

* Fixing the template language so that
$Really.Long.And(Complex).Variables work
* $Up and $Top variables, to access parent contexts within a
template.
* <% if Var != Val %>, to go with <% if Var = Val %>, but stay away
from complex conditionals.
- Possibly <% if Var in Va1, Val2, Val3 %> and <% if Var not in
Va1, Val2, Val3 %>
- Possibly <% else if XXX %>
- Possibly <% if Var = $OtherVar %>

[Ingo Schommer]

I see "Syntax improvements" as the way to go (as opposed to new
capabilities).
We should support existing assumptions when using known syntax such as
"if".

Some examples:
<% if !$CurrentMember %>
<% if $CurrentMember.Groups.First %>
<% if Property < value %>
<% control MyThing(str1, str2, str3) %>

You can go quite a long way with controller-methods,
where you can do whatever PHP allows (in a far more descriptive/
versatile way).
E.g. <% if $CurrentMember.Groups.First %>
would be <% if $HasGroup %> and define "function HasGroup()" on the
controller.

I think an extension of the syntax (with the current regex-base)
would open a can of worms both in developer-expectations and
maintenance
(e.g. "Why can't I use dollar-variables in my orderby-snippet in the
template?")

[Ingo Schommer]

Oh god, parent context would be soo handy - this is the "most-wanted
item" for me *g

[Michael Gall]

I've attached a half finished patch to ticket 1934

--
--

http://wakeless.net

[Jamie Neil]

Nicolaas Thiemen Francken wrote:
> Keep it simple Sam... The logic should be in the php - not in the templates!
> This requires more skill from the developer but makes for better sites.

The problem is that at the moment there are certain things that should
be simple, but aren't. For example:

- $Title <% control Children %>$Outer.Title - $Title<% end_control %>

- $Author.FirstName.LowerCase

- <% if Colour != 'red' %>Not Red<% end_if %>

Jamie

[Lakshan Perera]

I think the ability to pass a variable inside a control is more important addition..

Lakshan

[Tim Copeland]

My biggest request here is to make the error messages more useful, for
example, the ability to tell me in which file I've stuffed up
something in. Currently it's difficult to pinpoint exactly where the
problem is, and the best approach I've found for dealing with finding
templating errors, is to progressively remove blocks of code untill
the page runs again, and trying to rebuild the page untill it works in
the way I had intended, which is very slow.

The types of error messages I'd like to see are things such as;

* You have a missing end_control statement in the file "/templates/
Includes/footer.ss"
* You have tried to use the method setWidth on an image $Logo, line
12 in the file "/templates/Includes/Menu1.ss". This method does not
exist, please check your spelling and review the available methods for
this field type at doc.silverstripe.com/blah/

[Sam Minnee]

To effectively to this, we would need to improve the SS->PHP convertor
to be able to detect syntax errors. This would be hard, but not
impossible. We would need to move right away from the regular
expression based parser and into some more sophisticated. Perhaps
even a context-free grammar parser or something. Alternatively, we'd
need to code up a system to check for matched pairs of <% control %>
and <% end_control %> blocks, and avoid things like

<% control XX %>
<% if YY %>
<% end_control %>
<% end_if %>

One potential approach with this would be to break the code into an
sequential array, each item of which would contain one of:
* control or end_control code
* if or end_if code
* $ variable
* HTML literal

It would then be relatively straightforward to look ahead in the array
and verify that there is a matching end_ code down the line.

Wakeless - how far towards this goal did you get in your upgraded
parser?

[Markus Lanthaler]

Have you ever considered using a 3rd party template engine instead?
For example Smarty (http://smarty.php.net/) supports all the features
requested above, it's really fast (compiles templates into PHP files),
supports caching and is easily extendable.

> > the way I had intended, which is very slow.- Hide quoted text -
>
> - Show quoted text -

[Sam Minnee]

Although we're open to the idea of letting people use another
templating language for work not hosted on our servers, we're not
going to move to Smarty, or another template language, for main
SilverStripe development in the foreseeable future.

SSViewer has been developed specifically for Sapphire, and Sapphire
has been developed specifically for SSViewer. Our templating language
benefits from mandating OO access to the template data, and thus our
control structures are more concise.

Migrating all of the current templates to another templating language
would be an enormous task, and running multiple templating languages
would be equally unsatisfactory. Neither of these alternatives is
more attractive than rolling up our sleeves and fixing a couple of
issues that are letting down an otherwise good template language.

Frankly, the "major shift" that needs to be made in our development is
not to switch to Smarty. It's to give more priority to SSViewer
development. It's been neglected for a long while, it's a vital part
of the whole system, and this needs to change. Solving some of the
most annoying problems, as discussed on this thread, is not going to
be too hard - we just need to actually do it!

[Markus Lanthaler]

> Although we're open to the idea of letting people use another
> templating language for work not hosted on our servers, we're not
> going to move to Smarty, or another template language, for main
> SilverStripe development in the foreseeable future.
>
> SSViewer has been developed specifically for Sapphire, and Sapphire
> has been developed specifically for SSViewer. Our templating language
> benefits from mandating OO access to the template data, and thus our
> control structures are more concise.

I understand that SSViewer has been developed specifically for
Sapphire. The "we make everthing ourselves" is unfortunately a common
phenomena in the PHP community. I suggested to give a look at for
example Smarty was because I think we could save an enormous
developing effort by using a 3rd party library. There is a whole
community around that project that concentrates solely on developing a
good template engine.

Of course it would have some cost to migrate the code but I think it
should be worth to consider it or at least to discuss objectively
about it.

What are the unique features of SSViewer? I used Smarty quite a lot
and I can't see much differences - at least no differences that
couldn't be implemented easily. Smarty is extremely modular. All the
logic in SSViewer::parseTemplateContent() could be easily be ported to
Smarty plugins (http://smarty.php.net/manual/en/plugins.php,
http://smarty.php.net/manual/en/plugins.functions.php,
http://smarty.php.net/manual/en/plugins.block.functions.php,
http://smarty.php.net/manual/en/plugins.compiler.functions.php). We
could even automatically convert legacy templates on-the-fly to new
Smarty templates with prefilters (http://smarty.php.net/manual/en/
advanced.features.prefilters.php).


What do you mean by "Our templating language benefits from mandating

OO access to the template data, and thus our control structures are

more concise"?
You can also pass objects to Smarty templates and call functions on
those objects. You can even restrict the methods and properties that
can be accessed. Take a short look at those docs:
* http://smarty.php.net/manual/en/language.variables.php
* http://smarty.php.net/manual/en/advanced.features.php#advanced.features.objects

Also Smarty's default delimiter "{" and "}" can be changed to "<%" and
"%>" (http://smarty.php.net/manual/en/language.escaping.php)

> Frankly, the "major shift" that needs to be made in our development is
> not to switch to Smarty. It's to give more priority to SSViewer
> development. It's been neglected for a long while, it's a vital part
> of the whole system, and this needs to change. Solving some of the
> most annoying problems, as discussed on this thread, is not going to
> be too hard - we just need to actually do it!

Please don't get me wrong! I'm not against enhancing SSViewer but I
think we should definitively consider to use more 3rd party libraries.
We could save a lot of man-power that we could use to improve
SilverStripe's uniqueness and add new features instead of ("wasting"
it by) focusing on such a low-level task like template parsing.

[Ingo Schommer]

thanks for starting this interesting discussion markus!

i've had a quick look at smarty's object handling (its been a while
since i last used smarty),
they seem like somewhat "second class citizens", while SSViewer is
built around OOP.

E.g. how would you achieve this construct in smarty:
$myViewableData->customise(array(
'MyCustomProperty' => 'foo',
'MyCustomObject' => $manySubproperties
))->renderWith('LayoutTempate','MainTemplate');

i'm not sure about the performance-implications of registering objects
with the templating engine,
and if it supports nesting them - any ideas?

don't forget that we're also building a "low-level" framework
(sapphire), and bringing on external dependencies where we can't
control release cycles and contributions limits the flexibility of
what we can do with it. generally i'm open for the idea of reducing
custom code and by that reducing maintenance cost, but i'm skeptical
if the templating language is the right candidate for it.

On Dec 4, 5:47 am, Markus Lanthaler <mark_lantha...@gmx.net> wrote:
> > Although we're open to the idea of letting people use another
> > templating language for work not hosted on our servers, we're not
> > going to move to Smarty, or another template language, for main
> > SilverStripe development in the foreseeable future.
>
> > SSViewer has been developed specifically for Sapphire, and Sapphire
> > has been developed specifically for SSViewer. Our templating language
> > benefits from mandating OO access to the template data, and thus our
> > control structures are more concise.
>
> I understand that SSViewer has been developed specifically for
> Sapphire. The "we make everthing ourselves" is unfortunately a common
> phenomena in the PHP community. I suggested to give a look at for
> example Smarty was because I think we could save an enormous
> developing effort by using a 3rd party library. There is a whole
> community around that project that concentrates solely on developing a
> good template engine.
>
> Of course it would have some cost to migrate the code but I think it
> should be worth to consider it or at least to discuss objectively
> about it.
>
> What are the unique features of SSViewer? I used Smarty quite a lot
> and I can't see much differences - at least no differences that
> couldn't be implemented easily. Smarty is extremely modular. All the
> logic in SSViewer::parseTemplateContent() could be easily be ported to

> Smarty plugins (http://smarty.php.net/manual/en/plugins.php,http://smarty.php.net/manual/en/plugins.functions.php,http://smarty.php.net/manual/en/plugins.block.functions.php,http://smarty.php.net/manual/en/plugins.compiler.functions.php). We

> could even automatically convert legacy templates on-the-fly to new
> Smarty templates with prefilters (http://smarty.php.net/manual/en/
> advanced.features.prefilters.php).
>
> What do you mean by "Our templating language benefits from mandating
> OO access to the template data, and thus our control structures are
> more concise"?
> You can also pass objects to Smarty templates and call functions on
> those objects. You can even restrict the methods and properties that
> can be accessed. Take a short look at those docs:
> *http://smarty.php.net/manual/en/language.variables.php

> *http://smarty.php.net/manual/en/advanced.features.php#advanced.featur...

[xeraa]

I'm joining Markus' point of view that considering other templating
engines would be a good idea.
I've also been using Smarty for some projects, but I'm not totally
convinced it's the best necessarily the best one for SS. Maybe it's a
bit "too much" and the future development is not yet clear to me (for
more than a year there have been talks about a PHP5 only version, but
obviously they cannot really decide on anything and start off).

Maybe it would be a good idea to take a better look at available
options (there are heaps of them, some are listed at
http://www.whenpenguinsattack.com/2006/07/19/php-template-engine-roundup/
for example) and try to find viable alternatives.
Could be better than creating templating engine number 1538.

But of course this decision also includes quite a lot of risks and
problems ;-)

> > Smarty plugins (http://smarty.php.net/manual/en/plugins.php,http://smarty.php.net/man...). We

[Sam Minnee]

I think that we would want to restrict our analysis to templating
engines that have active development going on, and are preferably
quite well-known. It's hard to tell which of those listed meet those
criteria, other than Smarty.

Judging whether Smarter or SSViewer is going to be more appropriate
will be an ominous task in itself. Trying to compare 25 different
templating engines will be next to impossible. I think we establish a
maximum of 2 or 3 potential replacements to SSViewer and then look at
their suitability.

On Dec 4, 1:11 pm, xeraa <P.Xe...@gmail.com> wrote:
> I'm joining Markus' point of view that considering other templating
> engines would be a good idea.
> I've also been using Smarty for some projects, but I'm not totally
> convinced it's the best necessarily the best one for SS. Maybe it's a
> bit "too much" and the future development is not yet clear to me (for
> more than a year there have been talks about a PHP5 only version, but
> obviously they cannot really decide on anything and start off).
>
> Maybe it would be a good idea to take a better look at available

> options (there are heaps of them, some are listed athttp://www.whenpenguinsattack.com/2006/07/19/php-template-engine-roun...

[Sigurd Magnusson]

Without trying to engage in the conversation heavily, if it turns out
that many of SSViewer's key benefits reside in sapphire, and that we
stand to loose a lot by adopting an external templating system, then we
should look to creating a community of people around ours (both for
technical enhancements, support and to continually increase adoption),
which may suggest refactoring it for seperate use, etc.

[Sam Minnee]

Thanks for the documentation references, Markus - Smarty is certainly
very powerful!

To answer the question - what does SSViewer have that Smarty doesn't,
it basically comes down to the following points:

* ManifestBuilder to automatically find templates no matter where
they are in the filesystem.
* Theme support - which is really part of the line above.
* Everything is automatcally passed through cachedCall(), obj(), and
XML_val() calls on ViewableData, which handle caching and format
conversion.
* Object method calls are first class, not second class. In SSViewer
you can write $SomeVariable, in Smarty, you would have to go { $item-
>SomeVariable }, or register every method on every object in the
template language, which isn't an option as it would completely
corrupt the flexibility of SilverStripe.

It's true that we could implement our own { control } and { if }
constructs as custom Smarty functions. However, then we would have to
implement our own Up and Top functions too, since Smarty gives you
access to parent contexts by explicitly labelling the contexts with
different variables, like you do inside a piece of PHP code.

If we're going to that, it begs the question - why are we using Smarty
if we're butchering it so dramatically?

SSViewer is what you could describe as a "strongly contexted templated
language" - every template code is a reference on the current
context. Smarty forces you to specify the context that you wish to
access. This gives it a certain power, but it also requires template
designers to be more verbose, and steps outside of the design goals of
SSViewer, which was to provide a really simple templating language for
writing CMS templates.

If there is a well-supported, actively-developed, "strongly contexted
template language", then it might be a more appropriate fit for
SilverStripe. Does anyone know of such a system?

On Dec 4, 5:47 am, Markus Lanthaler <mark_lantha...@gmx.net> wrote:

[Markus Lanthaler]

> i've had a quick look at smarty's object handling (its been a while
> since i last used smarty),
> they seem like somewhat "second class citizens", while SSViewer is
> built around OOP.

Well principally objects are threatened like normal variables in Smarty - I
don't understand exactly what you mean by "second class citizens".

> E.g. how would you achieve this construct in smarty:
> $myViewableData->customise(array(
> 'MyCustomProperty' => 'foo',
> 'MyCustomObject' => $manySubproperties
> ))->renderWith('LayoutTempate','MainTemplate');

Hmm... if I understand the code right, I does nothing more than registering
'foo' as $MyCustomProperty and the object/variable $manySubproperties as
$MyCustomObject for the template and render it then with the templates
'LayoutTempate' or 'MainTemplate', right!?
In Smarty you would do something like this

$smarty->assign('MyCustomProperty', 'foo');
$smarty->assign('MyCustomObject', $manySubproperties);

OR

$smarty->assign(array(
'MyCustomProperty' => 'foo',
'MyCustomObject' => $manySubproperties));

// check if LayoutTemplate exists
$smarty->fetch('LayoutTemplate');
// otherwise do

$smarty->fetch('MainTemplate');

So specifying more possible templates at once is not possible by default,
but we could easily reuse the existing code to implement this.

> i'm not sure about the performance-implications of registering objects
> with the templating engine,
> and if it supports nesting them - any ideas?

Registering variables or objects for the template does nothing else than
assigning them to some array ($smarty->vars['xyz'] = $value). So there
shouldn't be performance problems. What do you mean by nesting them?

[Markus Lanthaler]

To be honest I have not a clear overview of the current template system but
I appears to met that (parts of) SSViewer and ViewableData represent the
view in an MVC architecture. A template engine should not need to search for
the template itself for example.

How does caching work at the moment? Is there really any caching or is there
just a compilation of the templates? At least I cannot see anything else by
looking at "silverstripe-cache" directory. Smarty would support compilation
AND caching of whole documents which would result in a big performance boost
for SilverStripe powered sites (since most of the pages are static or change
seldom nevertheless).

I would suggest to outsource all the template parsing, compiling and caching
stuff to (for example) Smarty and leave the rest of the logic in SSViewer
and ViewableData. So we could combine the power of two worlds. SilverStripes
features like templating and the parsing power of Smarty with it's
modularized architecture and extendability.
I haven looked at it in detail but I think it wouldn't be much work to
achieve this.

[Sam Minnee]

You're right that I was mashing together the concept of a "view" and
the template language that supports that view. It was another piece
of Rails inspiration :-P

There's currently no caching, although there is a static export. And
one of the GHOPers is writing a patch that provides generic "cache the
result of this function" support that could be used for a number of
things including templates: http://open.silverstripe.com/ticket/1954

I think that in some ways caching belongs more comfortably outside of
the template parser, since the question of when a cache item should be
expires depends so heavily on the model & controller layers. Saving
the result of a template execution to a file for later usage is the
trivial part of the problem that #1954 addresses.

You've got a lot more knowledge of Smarty than I do. It would be good
if you could have a think about how you could incorporate the "default
context" concept into Smarty, to see whether its worthwhile.

In the meantime, I'm going to investigate how complex it would be to
improve the current SSViewer parser, and perhaps look at separating
the template parser from the MVC-specific aspects, to ease the
transition of templating languages should we decide that its
appropriate.

[Sam Minnee]

As I understand it, in smarty, you could set up your template render
as follows:
* Register $page
* Put { $page->FirstName } { $page->Surname } in your template

This is too verbose for my liking: The template language should know
that you're talking about the page. This is what Ingo means by
objects being second class citizens. SSViewer makes every template
code into a method call on a specific object - the current context.
For me, this is a show-stopper; concise templates were a major design
goal of SilverStripe and it's not worth giving that up for code reuse.

The other way you could do things in Smarty is like this:
* Register $page->Title as Title and $page->Content as Content
* Put { $Title } { $Content } in your template

Which is fine if you know exactly which fields are required by the
current template - but you don't. The whole point of SS is that you
can chuck <% control Children %> into your template and it will know
that it needs to request the children from the database. Again,
another show-stopper.

So - my question is basically: is there a 3rd way? Can we make every
top-level template code resolve to a method call on an object of our
choice?

[Nicolaas Thiemen Francken - Sunny Side Up]

just to add my five cents worth... I really feel that this logic belongs in PHP so that I can get an html builder to create the templates ... what Smarty can do, I would do within the PHP so that the templates are kept nice and simple... allowing a programmer to work side-by-side with a html/css specialist.

[Sam Minnee]

That's an important use-case for us as well, Nicolaas. Of course,
with an overly restrictive template language, things can be *harder*
for the HTML/CSS guys, rather than easier. Which is why we're looking
at extending the abilities of the template language, either by
enhancing SSViewer or making use of Smarty as the core.

On Dec 5, 4:01 pm, "Nicolaas Thiemen Francken - Sunny Side Up"

[Markus Lanthaler]

> I think that in some ways caching belongs more comfortably outside of
> the template parser, since the question of when a cache item should be
> expires depends so heavily on the model & controller layers. Saving
> the result of a template execution to a file for later usage is the
> trivial part of the problem that #1954 addresses.

Yes, I agree completely with you.. but it could help a lot if the
template engine supports that stuff. For example in Smarty you do it
like this:


if(!$smarty->is_cached('template.tpl',$my_cache_id)) {
// No cache available, do variable assignments here.
$contents = get_database_contents();
$smarty->assign($contents);
}

$smarty->display('template.tpl',$my_cache_id);

> As I understand it, in smarty, you could set up your template render
> as follows:
> * Register $page
> * Put { $page->FirstName } { $page->Surname } in your template
>

> The other way you could do things in Smarty is like this:
> * Register $page->Title as Title and $page->Content as Content
> * Put { $Title } { $Content } in your template

Right

> Which is fine if you know exactly which fields are required by the
> current template - but you don't.

Well, that's the question! Should the template designer and the
programmer not somehow negotiate which variables should be available
for a specific template? It's no problem if the programmer is the same
person as the template designer, but if we talk about creating generic
themes and stuff like that there should be some "contract" about what
variables should be available. What happens if the programmer renames
some variables or methods?
I saw the following question very often "What variables are available
for me as template designer?".
If we would have such a contract (just an array of variable names) it
would be trivial to set it up with Smarty and much easier for template
designers.

> You've got a lot more knowledge of Smarty than I do. It would be good
> if you could have a think about how you could incorporate the "default
> context" concept into Smarty, to see whether its worthwhile.

That means that all the variables like $Title or $Content should be
automatically translated into $page->Title or $page->Content, right?
This could be easily implemented with an pre-filter (which is executed
only once per template). Take a look at
http://smarty.php.net/manual/en/advanced.features.prefilters.php

[Sid Bachtiar]

Hi,

I think business logic should be out from template/interface, but not
all logic.

Personally I prefer using php alone for several reasons:
- It is already a templating engine
- Many IDEs support its syntax (Eclipse, Zend, Dreamweaver, etc)
- Already object oriented (PHP 5)
- No overhead of template engine
- No learning new syntax

Having said that I can see some benefits of using template engine on
top of php:
- Template caching
- Friendlier / less techie syntax
- Keeping business logic out of the template

Integrating Smarty has a lot of benefits:
- It is mature: been around for years, well and trully tested
- It is popular: lots of templates already and its syntax has been
proven to be popular with designers
- A significant leverage in terms of maintenance, tutorial,
documentation, etc
- Don't need to reinvent the wheel

I think Silverstripe should either go with mature templating engine
like Smarty, or just use php syntax, rather than reinventing its own
template engine with its own syntax. Just my 2c as someone who just
tried Silverstripe today :-)

Regards,

Sid

On Nov 30, 10:22 am, "Nicolaas Thiemen Francken" <nfranc...@gmail.com>
wrote:

[Akash Mehta]

I think Sid's got the right idea here (re: pure PHP templates), for a
number of reasons:

PHP is designed with a template-style approach to HTML generation.
With the new syntax (e.g. foreach (...): ... endforeach; ) it's got
pretty clean syntax that suits templates.

Extensions to the SS templating language are being discussed here; as
the language grows, the overhead of the parsing logic will increase,
and on large installations this could easily get out of hand, even
with caching. Pure PHP has none of this overhead.

A PHP-based template will make sense to PHP developers, arguably the
largest group of web developers out there. That's instant user base.
As a GHOP participant, I'm just getting my head around the SS template
system, but CakePHP templates (well, views) make perfect sense. If you
switch to pure PHP templates and SS becomes really popular, you'll
find you get many more open source designs converted to SS themes as a
result of the lower barrier to entry.

You could maintain the current (reasonably) designer-friendly style of
templating using some of the extended features of PHP syntax, e.g.
dynamic functions: $Control($ControlName), in a way that makes sense
to everyone.

Just my two cents as well :)

---
Akash

[Andre]

I agree - use PHP - if a designer has to learn a new template syntax
for every new development firm or contract that they work on, why not
learn one thats actually usefull PHP.

[marikas]

At the risk of getting flamed, I'll present my two cents on this in a
different way;
As a designer (as opposed to a php coder), let me just say that Joomla
is the easiest-to-design-for CMS. Creating templates for Joomla is a
breeze. Once upon a time, with nil php knowledge, it took me five
minutes of studying a Joomla template file to be able to build
complete, fully-functional templates. Whatever Joomla does to achieve
that, it works.

cheers
-noobestofdemall