Supervisord and Ascii encoding problem ?

[Laurent DAVERIO]

Hello list,

I have a very vague question about Supervisord.

Today, I managed to solve (or work around) an annoying Ascii encoding
problem that kept coming up several times in a Pylons app of mine.

The problem was occurring in a method trying to open files with
"special" characters in their filenames, the said filenames also being
stored in PostgreSQL. Although I thought I had handled everything
properly (UTF-8 strings cast to Unicode, etc.), the problem was still there.

Turns out it only occurs when Pylons is launched via Supervisord. If I
use Daemontools instead, or just run "paster serve" from the command
line, the problem goes away.

Now, I vaguely remember that my Zope websites wouldn't run properly when
I tried to switch from Daemontools to Supervisord, that's why I had
stayed with the former. It was years ago, and I never investigated the
problem.

So, maybe, I should ask the question in a Supervisord-related mailing
list instead of here. But as it may be of interest to fellow Pyloners
(and Pyloneresses, as well as Pyramiders, Pyramideresses, etc.) and I
haven't found anything via Google, here it is :

Has anyone noticed similar encoding problems with Supervisord? I suspect
it has something to do with the way it handles the (encoding of the)
stdin/stdout/stderr of its child processes.

And it seems to have a definite incidence on all Python apps when run by
supervisord.

Thanks in advance,


Laurent DAVERIO.

[Jesaja Everling]

Hi Laurent,

I definitely remember having the same problem with Supervisor. I can't
remember exactly what I changed at the moment, but It might have been
this:

[supervisord]
environment=LC_ALL='en_US.UTF-8',LANG='en_US.UTF-8'

This is in my /etc/supervisor/supervisord.conf. If this is in there by
default, I changed something else... ;)

Best Regards,

Jesaja Everling

> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Laurent DAVERIO]

Thank you very much Jesaja,

I'll keep that in mind :-)

Laurent.


Le 08/02/13 17:15, Jesaja Everling a �crit :

[Grey Li]

Thank you Jesaja! You saved my day!

[Mike Orr]

This sounds like one of the problems that PEP 538 in Python 3.7 is
designed to solve:

https://docs.python.org/3.7/whatsnew/3.7.html

"""
An ongoing challenge within the Python 3 series has been determining a
sensible default strategy for handling the “7-bit ASCII” text encoding
assumption currently implied by the use of the default C locale on
non-Windows platforms.

PEP 538 updates the default interpreter command line interface to
automatically coerce that locale to an available UTF-8 based locale as
described in the documentation of the new PYTHONCOERCECLOCALE
environment variable. Automatically setting LC_CTYPE this way means
that both the core interpreter and locale-aware C extensions (such as
readline) will assume the use of UTF-8 as the default text encoding,
rather than ASCII.
"""

I have never had this problem with Supervisord, but maybe it's because
I live in the US so the default is my locale.

I do get occasional problems with requests that have non-ASCII (or
non-properly encoded) characters in the URL that raise Internal Server
Error when the URL is parsed, but those are all illegitimate requests
so I ignore them. It happens with one of my Pylons applications; it
hasn't happened in Pyramid. But the Pylons application is the most
widely used and has the most external links to it so it's more likely
that bots would find it and throw trash at it or send URLs in foreign
languages to it.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/8e1a6f1b-b17d-4502-bca0-65f01bbb68ae%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Mike Orr <slugg...@gmail.com>