Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Move to maketext: sample templates

4 views
Skip to first unread message

Vitaly Fedrushkov

unread,
Jan 9, 2009, 6:53:36 PM1/9/09
to local...@bugzilla.org
Good $daytime,

"The proof is in the pudding", says TPJ13. To get an idea on how
[un]readable templates would become under Maketext, I have converted
several of them:

https://bugzilla.mozilla.org/attachment.cgi?id=356254&action=edit

These contain no examples of numeric inflection, (which I guess would be
in favor of Maketext), but show at least some problems to be addressed
before we start converting all 33k LOCs of Bugzilla templates to Maketext.

Lessons learnt:

1. By far most annoying was the need for

[% l("text [_1]",value FILTER whatever,...) %]

I start thinking of unconditional 'FILTER html' to all args to maketext
calls. But there must be counterexamples.

2. Even if we do enforce such filtering, template security audit
(automated by t/008filter.t) will become harder and yield more
exceptions and even may be prone to subtle errors.

3. Large static texts look fine with {{ }} syntactic sugar. However,
stock xgettext.pl grabs them literally, leading to Lexicon keys with
lots of leading spaces, and ugly PO entries. As long as we're rendering
HTML, we may enforce 'FILTER collapse' on such keys both in xgettext.pl
and upon lookup.

4. Long texts as Lexicon keys are not so handy, with their multiple
explicit newlines, leading spaces, etc. Other projects using Maketext
and Template Toolkit also confront this:

Act! ended up with keys cut at first newline:

| msgid "This is your personal page."
| msgstr ""
| "This is your personal page.\n"
| "From here you can manage everything regarding your participation\n"

Request Tracker indeed uses long keys, up to 400 characters on a single
line. Not a problem with special PO editing tools, but their template
files look accordingly, i.e. do not wrap paragraphs at all:

http://svn.bestpractical.com/cgi-bin/index.cgi/bps/view/rt/3.8/trunk/share/html/Install/DatabaseDetails.html

(around line 55)


Ideas and criticism are welcome. Thanks in advance!

Regards,
Vitaly.

Vitaly Fedrushkov

unread,
Jan 17, 2009, 7:48:06 PM1/17/09
to local...@bugzilla.org
Good $daytime,

Last Bugzilla meeting saw an improvised brainstorming session on how to
keep template syntax easy when adding l10n support. Thanks to everyone
who contributed to discussion.

> 1. By far most annoying was the need for

> [% l("text [_1]",value FILTER whatever,...) %]

Example 1:
https://bugzilla.mozilla.org/attachment.cgi?id=356254&action=diff#bugzilla-tip/template/en/default/account/profile-activity.html.tmpl_sec3

As is:
<a href="editusers.cgi?action=edit&amp;userid=
[%- otheruser.id FILTER url_quote %]"
title="Edit user '[% otheruser.login FILTER html %]'">Edit this user</a>

Variant 1.1: dumb
[% otheruser_login = otheruser.login FILTER html %]
[% edit_url = BLOCK %]<a href="editusers.cgi?action=edit&amp;userid=
[%- otheruser.id FILTER url_quote %]"
title="[% l("Edit user '[_1]'", otheruser_login) %]">
[% END %]
[% |l(edit_url, "</a>") %]
[_1]Edit this user[_2]
[% END %]

Variant 1.2: introduce functions for 'FILTER html' and so on
[% |l("<a href=\"editusers.cgi?action=edit&amp;userid=" _
url_quote(otheruser.id) _ "\" title=\"" _
l("Edit user '[_1]'", html(otheruser.login)) _ "\">",
"</a>") %]
[_1]Edit this user[_2]
[% END %]

Variant 1.3: introduce scalar operators for the same:
[% |l("<a
href=\"editusers.cgi?action=edit&amp;userid=${otheruser.id.url_quote}\"
title=\"" _
l("Edit user '[_1]'", otheruser.login.html) _ "\">",
"</a>") %]
[_1]Edit this user[_2]
[% END %]

Variant 1.4: Leave basic HTML syntax within translated text:
[% |l("editusers.cgi?action=edit&amp;userid=${otheruser.id}"
l("Edit user '[_1]'", otheruser.login)) %]
<a href="[_1]" title="[_2]">Edit this user</a>
[% END %]

In the latter case, each parameter can be html-filtered within 'l'
filter implementation.

Will any of these improve readability and what option do you prefer?

Regards,
Vitaly.

Vitaly Fedrushkov

unread,
Jan 17, 2009, 8:27:30 PM1/17/09
to local...@bugzilla.org
Good $daytime,

Another option came from reading about Pylons and comparison of compiled
templates before and after maketext'izing.

We may try to hide maketext calls entirely to make it happen on precompilation,
not template parsing.

We still need to mark translation units with {{ }} or whatever. However,
instead of substituting them to l() calls on parse time:

1. When {{ is encountered, start new block (with my $output as usual)

2. Incoming literal text is added to $output as is.

3. Stash calls, filters and captures add "[_n]" to $output and evaluated with
results saved to @params list.

4. When }} is encountered, block is closed, returning l($output, @params).

Good news: almost no explicit l() calls

Bad news:

- xgettext complications: parse TT well or scan precompiled templates?

- @params are obfuscated by precompilation, nothing to put as PO comments

Is it practical? Suggestions and criticism are welcome.

Regards,
Vitaly.

Frédéric Buclin

unread,
Jan 18, 2009, 11:30:46 AM1/18/09
to devel...@bugzilla.org
Le 18. 01. 09 01:48, Vitaly Fedrushkov a écrit :
> Variant 1.1: dumb

This one is clearly ridiculous.


> Variant 1.2: introduce functions for 'FILTER html' and so on

So you will have to define one function for all possible FILTERs,
including those taking parameters?


> Variant 1.3: introduce scalar operators for the same:

I don't like this one. I'm not even sure this will work as foo.html will
be seen as html() being a method of the foo object, which is not the case.


> Variant 1.4: Leave basic HTML syntax within translated text:

No idea what you try to do here, but not all strings use FILTER html.


> Will any of these improve readability and what option do you prefer?

From a developer point of view, I like none of these alternatives. Look
at the developers/localizers ratio: there are hundreds of developers,
compared to maybe ten localizers or so. And you are making life really
difficult for developers to make life a *little* bit easier for a few
localizers. I don't see this as being an improvement.
Now from a reviewer point of view: it would be a real pain to have to
review patches with tens of such lines of code. It's really prone to errors.
So I'm still very septical with all this stuff.

LpSolit
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=dev-apps...@lists.mozilla.org>

Teemu Mannermaa

unread,
Jan 19, 2009, 5:43:31 AM1/19/09
to devel...@bugzilla.org
On 18.1.2009 2:48, Vitaly Fedrushkov wrote:
> <a href="editusers.cgi?action=edit&amp;userid=
> [%- otheruser.id FILTER url_quote %]"
> title="Edit user '[% otheruser.login FILTER html %]'">Edit this user</a>

From all the examples you show, I don't understand why you want to
translate the HTML link tag. Somebody really wants to translate the HTML
code and not just the english strings that are actually visible to user?

I would like to see something simple like this:

<a href="editusers.cgi?action=edit&amp;userid=
[%- otheruser.id FILTER url_quote %]"

title="[% l("Edit user '$1'", filter($otheruser.login, "html")) %]">[%
l("Edit this user") %]</a>

Could something like that work? It's similar to your variant 1.2 but
much more simplified. Also our filter test still needs to protect us
from unfiltered variables.

I'd think this syntax actually improves readability of templates.
Especially parameterized texts would be clearly visible. As are
translated text. Correct usage should also be simple to verify during a
review.

> [% |l("<a href=\"editusers.cgi?action=edit&amp;userid=" _

These quoted quotation marks are not a good idea in any case. I do hope
one can use apostrophes for the outermost quoting to get rid of these..

Otherwise I could pretty much echo what LpSolit said. :)
--
"Anything is possible but probabilities vary."
"Whatever. Life goes on." -Dark Angel

Vitaly Fedrushkov

unread,
Jan 19, 2009, 8:59:40 PM1/19/09
to local...@bugzilla.org
Teemu Mannermaa wrote:
> On 18.1.2009 2:48, Vitaly Fedrushkov wrote:
>> <a href="editusers.cgi?action=edit&amp;userid=
>> [%- otheruser.id FILTER url_quote %]"
>> title="Edit user '[% otheruser.login FILTER html %]'">Edit this user</a>

> From all the examples you show, I don't understand why you want to
> translate the HTML link tag. Somebody really wants to translate the HTML
> code and not just the english strings that are actually visible to user?

I really don't want to mix html into translation. Just tried to beautify the
ugly reality :-)

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla%2Fwebtools%2Fbugzilla%2Ftemplate%2Fen%2Fdefault%2Faccount%2Fprofile-activity.html.tmpl&rev=&cvsroot=%2Fcvsroot#77

> I would like to see something simple like this:

> <a href="editusers.cgi?action=edit&amp;userid=
> [%- otheruser.id FILTER url_quote %]"
> title="[% l("Edit user '$1'", filter($otheruser.login, "html")) %]">[%
> l("Edit this user") %]</a>

Natural language in real example consisted of

<>Edit this user<> or <>find other accounts<>
or go <>back to the user list<>

where <> are HTML markup parts and second line is conditional

Not all languages maintain progression similar to English so I doubt about
possibility to translate

[_1] or [_2] or go [_3]

to every language without complications, given that 1, 2 and 3 are phrases by
themselves, already passed through l().

> I'd think this syntax actually improves readability of templates.
> Especially parameterized texts would be clearly visible. As are
> translated text. Correct usage should also be simple to verify during a
> review.

Yes, we're trying to separate natural language and html markup better
(see https://bugzilla.mozilla.org/show_bug.cgi?id=367784)

>> [% |l("<a href=\"editusers.cgi?action=edit&amp;userid=" _

> These quoted quotation marks are not a good idea in any case. I do hope
> one can use apostrophes for the outermost quoting to get rid of these..

Double quotes do allow ${} interpolation within.

> Otherwise I could pretty much echo what LpSolit said. :)

Thanks for your feedback!

Regards,
Vitaly.

SnowyOwl

unread,
Jan 20, 2009, 8:24:34 AM1/20/09
to local...@bugzilla.org
> Natural language in real example consisted of

>    <>Edit this user<> or <>find other accounts<>
>    or go <>back to the user list<>

Another idea went in while reading this back:

implement FILTER linkify(anchor1, ... anchorN):

Instance of <N> within text is replaced to <a anchorN>, empty <> with
</a>. Would enhance readability and simplify separation of text and
markup. Optionally FILTER url_quote may be enforced on anchors, to
make the whole construct safe (anyone to provide counter-example when
we should not do so?)

Example:

[% FILTER linkify('href="editusers.cgi?action=edit&amp;userid=' _
otheruser.id _


" title="' _ l("Edit user '[_1]'",

otheruser.login) _ '"',
'title="' _ l("Search For Users") _ '"
href="editusers.cgi"') %]
{{<1>Edit this user<> or <2>find other accounts<>}}
[% END %]

Here 'linkify' and 'l' filters are orthogonal, and 'l' is inner.

   Regards,
   Vitaly.

Vitaly Fedrushkov

unread,
Mar 6, 2009, 12:23:47 AM3/6/09
to local...@bugzilla.org
Vitaly Fedrushkov wrote:
> https://bugzilla.mozilla.org/attachment.cgi?id=359941&action=diff

Per bug 481796 [1], adding more syntactic sugar to reduce template complexity
considered harmful, primarily from security standpoint.

Introduction of Maketext would inevitably reduce direct text rendering. Note
how other case (bug 367784 [2]) also led to introduction of virtual methods.

And FILTERs are not handy enough for tasks besides direct text rendering. Any
logic besides simple nesting results in excessive template complexity when
implemented with FILTERs.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=481796
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=367784#c3

Vitaly Fedrushkov

unread,
Mar 6, 2009, 11:28:35 PM3/6/09
to local...@bugzilla.org
Frédéric Buclin wrote:
>> Variant 1.1: dumb
> This one is clearly ridiculous.

>> Variant 1.3: introduce scalar operators for the same:


> I don't like this one. I'm not even sure this will work as foo.html will
> be seen as html() being a method of the foo object, which is not the case.

Bug 481796 landed WONTFIX, so this option is closed.

>> Variant 1.4: Leave basic HTML syntax within translated text:
> No idea what you try to do here, but not all strings use FILTER html.

This does not allow for proper separation of text and markup.

>> Variant 1.2: introduce functions for 'FILTER html' and so on

|| [% |l("<a href=\"editusers.cgi?action=edit&amp;userid=" _
|| url_quote(otheruser.id) _ "\" title=\"" _
|| l("Edit user '[_1]'", html(otheruser.login)) _ "\">",
|| "</a>") %]
|| [_1]Edit this user[_2]
|| [% END %]

> So you will have to define one function for all possible FILTERs,
> including those taking parameters?

So this option is left alone, unless we come up with some new and fully
orthogonal approach.

> From a developer point of view, I like none of these alternatives. Look
> at the developers/localizers ratio: there are hundreds of developers,
> compared to maybe ten localizers or so.

And this (lack of) instrumentation is exactly _why_ we have fewer successful
l10n efforts, compared to competing bug trackers. Many skilled people have
looked at it, tried but then never seen again.

> So I'm still very skeptical with all this stuff.

Dear localizers, your opinions are wanted. I'm (sort of) stuck with Maketext
implementation (bug 407752 and in particular bug 412161), mostly because we're
lacking a good syntax to keep templates both simple to developers, safe and
still localizable.

Not sure if there's something worth discussing at upcoming Bugzilla meeting,
besides freezing bug 407752 entirely (P1, but not on 4.0 roadmap anyway). At
the moment I'm very skeptical about gettext future...

Regards,
Vitaly.

Vitaly Fedrushkov

unread,
Mar 22, 2009, 12:35:02 AM3/22/09
to local...@bugzilla.org
Good $daytime,

> Variant 1.2: introduce functions for 'FILTER html' and so on
> [% |l("<a href=\"editusers.cgi?action=edit&amp;userid=" _
> url_quote(otheruser.id) _ "\" title=\"" _
> l("Edit user '[_1]'", html(otheruser.login)) _ "\">",
> "</a>") %]
> [_1]Edit this user[_2]
> [% END %]

Done: https://bugzilla.mozilla.org/attachment.cgi?id=368754&action=diff

Function is called FILTER_html(), just to make things obvious. Diffstat is the
same as before, but I hope readability is better and 008filter.t checking
wouldn't be too complex.

Again, your feedback is very appreciated.

Regards,
Vitaly.

0 new messages