Multiple questions about i18n / pluralization in Pyramid

[Laurent DAVERIO]

Hello list,

I'm in the process of internationalizing a Pyramid application, and I'm
currently stumbling on a few problems.

Basic setup as per the tutorial works fine: I can extract message
strings to catalog files using Babel+Lingua, fill-in the PO files,
compile them to MO files and display the result.

1/ After looking at the docs and the source code, I'm more and more
convinced that Babel can't handle pluralization. Do you have any hint on
the issue?

2/ Editing the PO/POT files manually to add pluralization is not
possible, because they are overwritten next time I extract the strings.

3/ Manually adding msgids/msgstrs to a PO file (for additional messages)
fails for the same reason: if it's not in the source code, it is deleted
from the PO file :-(

4/ A solution might be to use a separate PO file for additional
translations. This is the way I am exploring now. I've created an
extra.po file and compiled it to extra.mo, but it's not taken into
account. Yet.

5/ I've also to find out how to use multiple domains in my app (among
other problems, how to localize deform widgets - the tutorial didn't
work for me...)

Well, I think that's all for now ;-) Lot of questions, and I'd be very
grateful for all answers/pointers/hints/suggestions :-)

Many thanks in advance,

Laurent.

[Laurent DAVERIO]

Hello list,

I'm trying to create a scaffold inside a namespace package. The
namespace package is created correctly, but "pcreate -l" can't find the
scaffold:

From a clean virtualenv (with only pyramid in it), I create a package "foo":

pcreate -t starter foo

Then. I manually create a namespace package as follows:

foo.bar/
foo/
bar/
scaffolds/
__init__.py
foobar/
__init__.py_tmpl
__init__.py
__init__.py
foo.bar.egg-info/
setup.py


In order to make it foo.bar namespace package, foo.bar/foo/__init__.py
contains:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)


The scaffold is defined in foo.bar/setup.py :

[pyramid.scaffold]
foobar=foo.bar.scaffolds:FooBarTemplate


When I run "pcreate -l", I get the following message:

> Warning: could not load entry point foobar (ImportError: No module named bar.scaffolds)
> Available scaffolds:
> alchemy: Pyramid SQLAlchemy project using url dispatch
> starter: Pyramid starter project
> zodb: Pyramid ZODB project using traversal


Do you have any idea of what I'm doing wrong?

[Laurent DAVERIO]

> When I run "pcreate -l", I get the following message:
>
>> Warning: could not load entry point foobar (ImportError: No module named bar.scaffolds)

To be more precise : pcreate is looking for the scaffold in the main
package (foo), not in the namespace package (foo.bar). It can be
verified if you create the appropriate directories inside package foo:

foo/
foo/
bar/
scaffolds/
__init__.py
static/
templates/
...
foo.egg-info/
setup.py
...

A quick and dirty way for now could be to create a symlink:

foo/foo/bar -> foo.bar/foo/bar

Whether you use a symlink or physically duplicate the "bar/" directory,
a "pcreate -l" now returns:

> Available scaffolds:
> alchemy: Pyramid SQLAlchemy project using url dispatch

> foobar: foo.bar template

> starter: Pyramid starter project
> zodb: Pyramid ZODB project using traversal

Laurent.

[Laurent DAVERIO]

Hello again,

this is a follow-up to my own post. I'm slowly working my way towards
the light ;-)

> 1/ After looking at the docs and the source code, I'm more and more
> convinced that Babel can't handle pluralization. Do you have any hint on
> the issue?

All my apologies to Babel developers. Babel DOES handle pluralization
perfectly indeed.


The steps of the solution would be:

def aview(request):
_ = request.translate
_p = request.localizer.pluralize
print _(u"Hello, world")
print _p(u"Hello, ${n} world", u"Hello, ${n} worlds", 5,
domain="foo", mapping={'n': 5})


Then, extract messages using:

python setup.py extract_messages -k _p

The "-k _p" option instructs Babel to also extract messages from _p()
calls, in addition to _() calls.


Now, what remains to be done is let Babel know how to handle the
parameters of _p(). So, I'm now turning my attention to

babel/messages/extract.py

and:

lingua/extractors/python.py

> 4/ A solution might be to use a separate PO file for additional
> translations. This is the way I am exploring now. I've created an
> extra.po file and compiled it to extra.mo, but it's not taken into
> account. Yet.

That solution probably wouldn't work, because I understand the po.files
must be named after the translation domain (let's say, "foo"). So, the
"right way" would do to create a second locale directory beside the
first one, e.g. "locale_extra", initialize it properly, and declare it
in Pyramid'a configurator using:

config.add_translation_dirs('foo:locale/', 'foo:locale_extra/')

> 5/ I've also to find out how to use multiple domains in my app (among
> other problems, how to localize deform widgets - the tutorial didn't
> work for me...)

I must have bee very tired when I wrote this. Two days later, colander
and deform translations suddenly started working :D


Laurent.

[Laurent DAVERIO]

Hello again,

one final word, in case someone is interested (apart from me, I mean ;-))

> Then, extract messages using:
>
> python setup.py extract_messages -k _p

More precisely:

python setup.py extract_messages -k _p:1,2

The argument "_p:1,2" will tell Babel that arguments 1 and 2 of function
_p are translation strings.

> Now, what remains to be done is let Babel know how to handle the
> parameters of _p(). So, I'm now turning my attention to
>
> babel/messages/extract.py
>
> and:
>
> lingua/extractors/python.py

The python extractor in Lingua is hard-coded to consider arg1 as a
translation string, and arg2 as a default value (this is derived from
the signature of the TranslationString constructor).

I was able to work around the problem by subclassing the PythonExtractor
class, so that I'm using arg2 for plurals instead of default:

---------------------------------------------------------------------

import tokenize
from lingua.extractors.python import PythonExtractor as BasePythonExtractor


class PythonExtractor(BasePythonExtractor):

def stateWaiting(self, ttype, tstring, lineno):
if ttype == tokenize.NAME and tstring in self.keywords:
self.state = self.stateKeywordSeen
self.msg = dict(lineno=lineno, func=tstring)

def addMessage(self, msg):
if not msg.get('label'):
return
default = msg.get('default')
if default:
messages = (u''.join(msg['label']), u''.join(default))
else:
messages = u''.join(msg['label'])
self.messages.append(
(msg['lineno'], msg['func'], messages, []))

extract_python = PythonExtractor()

---------------------------------------------------------------------


As regards Mako, the default extractor handles pluralization out of the box.


As far as I'm concerned, all my localization problems are solved :-)


Laurent.

[Laurent DAVERIO]

> A quick and dirty way for now could be to create a symlink:
>
> foo/foo/bar -> foo.bar/foo/bar
>
> Whether you use a symlink or physically duplicate the "bar/" directory,
> a "pcreate -l" now returns:
>
>> Available scaffolds:
>> alchemy: Pyramid SQLAlchemy project using url dispatch
>> foobar: foo.bar template
>> starter: Pyramid starter project
>> zodb: Pyramid ZODB project using traversal

Unfortunately, the symlink also breaks "pserve" on "foo", because it
also tries to register the views defined inside foo.bar:


> File "/usr/home/daverio/pydev/foo/foo/__init__.py", line 181, in main
> return config.make_wsgi_app()
> File "/usr/home/daverio/pydev/lib/python2.7/site-packages/pyramid/config/__init__.py", line 956, in make_wsgi_app
> self.commit()
> File "/usr/home/daverio/pydev/lib/python2.7/site-packages/pyramid/config/__init__.py", line 630, in commit
> self.action_state.execute_actions(introspector=self.introspector)
> File "/usr/home/daverio/pydev/lib/python2.7/site-packages/pyramid/config/__init__.py", line 1084, in execute_actions
> tb)
> File "/usr/home/daverio/pydev/lib/python2.7/site-packages/pyramid/config/__init__.py", line 1076, in execute_actions
> callable(*args, **kw)
> File "/usr/home/daverio/pydev/lib/python2.7/site-packages/pyramid/config/views.py", line 1177, in register
> route_name)
> pyramid.exceptions.ConfigurationExecutionError: <class 'pyramid.exceptions.ConfigurationError'>: No route named bar.page2 found for view registration
> in:
> Line 47 of file /usr/home/daverio/pydev/foo/foo/bar/views/__init__.py:
> @view_config(route_name='bar.page2', renderer='foo.bar:templates/page2.mako', permission='view')


I've tried using :

config.scan(ignore='foo.bar')

instead of just "config.scan()", but apparently it doesn't help...

Any idea? :-)