i18n enhancements
Diez B. Roggisch
I've talked a few times about enhancing the TG i18n-support. Now I
finally had time to produce a first patch for the current 1.0-trunk. It
should apply to 1.0.2, too.
See ticket #1405 (http://trac.turbogears.org/attachment/ticket/1405) for
an overview & the necessary files/patches.
The enhancements/fixes are essenitally these:
1) make DateValidation take format as a callable. That was missing andp
revented locale-dependend date parsing/formatting
2) collect strings from KID-templates in the same way the
KID-i18n-filter treats them and thus make them subject to i18n.
3) collect strings from JS-files and create message-catalogs for JS
which are automatically included using a standard-widget.
Please review & discuss.
Regards,
Diez
Alberto Valverde
Thanks for the patch Diez!
I've tried it out in a freshly quickstarted app (attached at the
ticket) and looks great. These are the steps I took in case anyone
wants to try it out:
1) tg-admin quickstart
2) wrote a sample and dumb helloworld.js at ${package}/static/
javascript and placed a button at welcome.kid to trigger the function.
3) registered that directory with widgets.register_statict_directory
4) set tg.include_widget = ["turbogears.jsi18nwidget"] in app.cfg
5) tg-admin collect
6) generated po in the toolbox
7) tg-admin compile
8) start app and enjoy
Some comments & suggestions:
1) It took me some time to figure out 3. Couldn't the widget or
something be more clever and do this automatically. I guess it's kind
of unintuitive for those not using widgets to include their js files
at the template. Maybe the i18n widget should be just an
implementation detail and this functionallity work for those not
using widgets at all.
2) Do you have plans to make this useful for distributing 18n bundles
with packaged widgets? From the code I guess messages-X.js must live
inside a TG app but I might be wrong...
Thanks! :)
Alberto
Alberto Valverde
On Jun 10, 2007, at 5:17 PM, Alberto Valverde wrote:
> (...)
> 1) tg-admin quickstart
> 2) wrote a sample and dumb helloworld.js at ${package}/static/
> javascript and placed a button at welcome.kid to trigger the function.
> 3) registered that directory with widgets.register_statict_directory
> 4) set tg.include_widget = ["turbogears.jsi18nwidget"] in app.cfg
> 5) tg-admin collect
5) tg-admin i18n collect
(BTW: the toolbox didn't list js files as options of files to scan
for strings so I presume the collect step couldn't be done here... i
suppose this has an easy fix but not sure...)
> 6) generated po in the toolbox
> 7) tg-admin compile
7) tg-admin i18n compile
(toolbox can do this)
7.5) tg-admin i18n create_js_messages
(could toolbox do this too?)
> 8) start app and enjoy
(could toolbox write the app, given a written spec in english
(localizable), deploy and start it up automatically too? ;)
Alberto
Diez B. Roggisch
> ticket) and looks great. These are the steps I took in case anyone
> wants to try it out:
>
> 1) tg-admin quickstart
> 2) wrote a sample and dumb helloworld.js at ${package}/static/
> javascript and placed a button at welcome.kid to trigger the function.
> 3) registered that directory with widgets.register_statict_directory
> 4) set tg.include_widget = ["turbogears.jsi18nwidget"] in app.cfg
> 5) tg-admin collect
> 6) generated po in the toolbox
> 7) tg-admin compile
> 8) start app and enjoy
>
> Some comments & suggestions:
>
> 1) It took me some time to figure out 3. Couldn't the widget or
> something be more clever and do this automatically. I guess it's kind
> of unintuitive for those not using widgets to include their js files
> at the template. Maybe the i18n widget should be just an
> implementation detail and this functionallity work for those not
> using widgets at all.
I'm not sure I understand what you want here. I _do_ understand that the
registration step can be done - I just did it in my testing-app that
first didn't have any code in TG itself, so I just missed that things
don't work without it. I just added the necessary code and it works, I
replaced the patch in the trac.
What I don't understand is the rest. Everybody is free to include
js-files herself, but then she is on his/her own.
> 2) Do you have plans to make this useful for distributing 18n bundles
> with packaged widgets? From the code I guess messages-X.js must live
> inside a TG app but I might be wrong...
So far it has to live there, yes. I currently don't use packaged
widgets, but I can see it would be helpful to have this functionality
there.
The collect/merge/create-js-stuff should work the same way.
What is missing is the inclusion of the message-files with another
package-name - the packaged widget one.
I can thing of one of several ways to accomplish this. One would be to
factorize out the JSLink-creation facilities of the JSI18NWidget, giving
a packaged widget author a way to create the links with her own package
name as argument.
The other one would be a static registration of package names. Benefit
of the first approach: the amount of js-files included in each template
is reduced. Benefit of the second approach: the author has less to write.
Either or even both options could be included - but I need to get on
speed with packaged widgets, which I'm currently not - so I'm open for
help here. Well, I'm open for help in any case :)
Diez
Diez B. Roggisch
> 5) tg-admin i18n collect
>
> (BTW: the toolbox didn't list js files as options of files to scan
> for strings so I presume the collect step couldn't be done here... i
> suppose this has an easy fix but not sure...)
I'm not using the toolbox, I haven't had the most pleasant experiences
with it. But if the general policy is to include support for it and you
can give me some pointers on how to do so, I'm having a look of course.
>> 6) generated po in the toolbox
>
>> 7) tg-admin compile
>
> 7) tg-admin i18n compile
>
> (toolbox can do this)
>
> 7.5) tg-admin i18n create_js_messages
>
> (could toolbox do this too?)
>
>> 8) start app and enjoy
>
> (could toolbox write the app, given a written spec in english
> (localizable), deploy and start it up automatically too? ;)
I don't understand this.. :( Which is most probably due to my lack of
toolbox understanding in general.
Diez
Alberto Valverde
I think that all is needed is to register the project/static/
javascript so the messages-X.js file can be served.... Hmm, I see
that your latest patch does this... cool.
>
>> 2) Do you have plans to make this useful for distributing 18n bundles
>> with packaged widgets? From the code I guess messages-X.js must live
>> inside a TG app but I might be wrong...
>
> So far it has to live there, yes. I currently don't use packaged
> widgets, but I can see it would be helpful to have this functionality
> there.
>
>
> The collect/merge/create-js-stuff should work the same way.
>
> What is missing is the inclusion of the message-files with another
> package-name - the packaged widget one.
>
> I can thing of one of several ways to accomplish this. One would be to
> factorize out the JSLink-creation facilities of the JSI18NWidget,
> giving
> a packaged widget author a way to create the links with her own
> package
> name as argument.
>
> The other one would be a static registration of package names. Benefit
> of the first approach: the amount of js-files included in each
> template
> is reduced. Benefit of the second approach: the author has less to
> write.
Hmm, whatever you think it's best I think too since I'm not very much
into the details of your patch's code. However, the "the author has
less to write" part sounds appealing... :) Maybe a future improvement
of TG-Widgets/TW could be to concatenate, jsminify and gzip all js
files into a single file to optimize load time so having many js
files isn't too problematic after all.
>
> Either or even both options could be included - but I need to get on
> speed with packaged widgets, which I'm currently not - so I'm open for
> help here. Well, I'm open for help in any case :)
tg-admin quickstart -t tgwidget MyWidget
creates a bare-bones widget package. There's really not much more
into it... just keep in mind the neccesary entry point for the
toolbox to load them for the widget browser.
I can't help much more since my TG-widgets-foo is pretty rusty now
since I've been using TW since it was born :)
BTW, I finally implemented JS i18n with a similar approach as yours
in the (pylons) app I talked about so thanks for the inspiration :)
Some thinks I found which you might want to consider as ideas for the
patch:
1) pygettext.py can extract localizable strings from JS files. I'm
not sure how robust this would be in the long run or the degree of
"hackiness" it achieves, but it's certainly easier than the regexps
(however, now that they're already implemented I see no good reason
to change them)
2) A gettext.translations instance has a private attribute called
"_catalog" which is a dict with the translation mapping once the What
I did to create the translation mapping for the JS side is something
like:
import gettext
t = gettext.translation(domain, localedir, [langs])
js_mapping = "var MESSAGES = %s;" % simplejson.dumps(t._catalog)
and then serve this dynamically from a controller method (no need to
maintain the js files but certainly not as efficient)
Alberto
W-Mark Kubacki
>
> 3) collect strings from JS-files and create message-catalogs for JS
> which are automatically included using a standard-widget.
Why using message catalogs if gettext does already that job? Create a controller
which will serve the JS-file as plaintext, utilizing gettext. That way you can
still utilize TG's means as they already are. (Remember to set "Content-Type:
text/javascript".)
-- Mark
Diez B. Roggisch
How's that supposed to work? _('foo') needs to get interpreted in the
language the file is written in. So how do you utilize python's gettext
in a JS file?
Preprocessing could be done but must be implemented but is kinda flaky
without a proper JS parser.
And you'd not be allowed to refer to static JS directly. Maybe cherrypy
allows for a generalized filtering scheme based on content type, I lack
knowledge of it's capabilities there.
Diez
Diez B. Roggisch
>
> creates a bare-bones widget package. There's really not much more
> into it... just keep in mind the neccesary entry point for the
> toolbox to load them for the widget browser.
>
> I can't help much more since my TG-widgets-foo is pretty rusty now
> since I've been using TW since it was born :)
I created a new patch, and introduced the following functionality:
The widget now has a property, locale_catalog_providers. This is a list
of callables that are invoked with the current locale. They have to
return a list of JSLinks or similar resources.
Then there is a method, register_package_provider, that takes a package
name, a directory path relative to that package, and a domain. These
default to the current app's package, 'static/javascript' and the
configured i18n.domain or 'messages'.
The widget will by default register a package-provider that delivers
js-catalogs as before, but this can be configure away by setting
i18n.suppress_default_package_provider = True
in the app.cfg.
This scheme allows widget-authors now as well to register their
respective packages & static javascript-dirs, or even come up with a
totally new scheme.
> BTW, I finally implemented JS i18n with a similar approach as yours
> in the (pylons) app I talked about so thanks for the inspiration :)
> Some thinks I found which you might want to consider as ideas for the
> patch:
>
> 1) pygettext.py can extract localizable strings from JS files. I'm
> not sure how robust this would be in the long run or the degree of
> "hackiness" it achieves, but it's certainly easier than the regexps
> (however, now that they're already implemented I see no good reason
> to change them)
I will look into it, as my current implementation works but lacks e.g.
multiline-support.
>
> 2) A gettext.translations instance has a private attribute called
> "_catalog" which is a dict with the translation mapping once the What
> I did to create the translation mapping for the JS side is something
> like:
> import gettext
> t = gettext.translation(domain, localedir, [langs])
> js_mapping = "var MESSAGES = %s;" % simplejson.dumps(t._catalog)
>
> and then serve this dynamically from a controller method (no need to
> maintain the js files but certainly not as efficient)
I won't use this myself - but with the new functionality above + the
config-option, you (or someone) are free to implement that yourself if
you like, because the locale_catalog_provider concepts allows for that.
Hope this new scheme is flexible enough for most of the needs.
The only thing the current concept lacks is a way to deliver js based on
page-based context. By that I mean that if e.g. the application contains
20 js files that are i18n'ed, the messages will always be delivere
fully, regardless of the actual usage of only a subset of the js-files.
However, I don't consider this a major drawback - at least in my usage
scenarios, there are only a few i18n'ed messages in JS anyway, for e.g.
creating links or buttons.
Diez