UTF-8 locale encoding: recommended approach?

[Thomas De Schampheleire]

Hello,

In Kallithea we sometimes have users reporting problems that are
caused by them not starting the application from an UTF-8 environment.
This can normally be solved by setting 'LANG' and 'LC_ALL' to a UTF-8
value but this can be done in several places.

Please see https://bitbucket.org/conservancy/kallithea/issues/310/accentuated-characters-improperly-rendered
as an example.

Here it is reported that the issue can be solved by changing Apache
config, and that Gearbox did not have the problem to begin with.
A suggestion from a Kallithea developer to change app_cfg.py to set
environment variables did not seem to have effect on the Apache
situation.

My question is now: what is the recommended way to handle this?
Does Turbogears2/Gearbox already does something automatically?
Any other feedback on this topic?

Thanks a lot,
Thomas

[Alessandro Molina]

TurboGears itself doesn't usually get in the way with encoding/decoding, I guess some encoding/decoding is happening when reading outputs for HG repositories.

The system encoding ( locale.getpreferredencoding ) frequently dictates what should happen when crossing the boundary between bytes/strings, as by default apache (when started from init scripts) sets locale to "C" ( as documented at http://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html ) my guess is that strings read from hg might be encoded/decoded with that locale resulting in some characters being impossible to decode.

I guess that the only "special" thing that gearbox is doing is just not messing with the locale.

There was an article about this issue from Graham Dumpleton himself which I guess can be helpful -> http://blog.dscpl.com.au/2014/09/setting-lang-and-lcall-when-using.html

If my guess that the improper decoding/encoding is happening at "hg layer" I'm not sure it's something where TG can intervene.

--
You received this message because you are subscribed to the Google Groups "TurboGears" group.
To unsubscribe from this group and stop receiving emails from it, send an email to turbogears+unsubscribe@googlegroups.com.
To post to this group, send email to turbo...@googlegroups.com.
Visit this group at https://groups.google.com/group/turbogears.
For more options, visit https://groups.google.com/d/optout.