how to use SetConsoleOutputCP in Unicode build console mode / conflict with setlocale,_wsetlocale
Creighton MacDonnell
I have some console mode programs that are started from .bat files for which I
would like to be able to non-Latin characters such as Cyrillic on the Windows
console.
If I manually open a console window and type "chcp 65001" to set the console
into the UTF-8 code page, then I can run the .bat file and see the Cyrillic. But
if I add the "chcp 65001" to the first line of the .bat file, it flushes the
rest of the .bat file because the code page has changed. If I try to have .bat
file with a single line "chcp 65001 && real_bat_file.bat", the rest of the line
after "chcp 65001" gets flushed too.
Apparently the way that one is supposed do this is make the call
"SetConsoleOutputCP(CP_UTF8);" - see "Console Functions" at
http://msdn.microsoft.com/en-us/library/ms682073.
But the call to "SetConsoleOutputCP(CP_UTF8);" will be overridden by a call to
"setlocale" or "_wsetlocale". And any subsequent calls to
"SetConsoleOutputCP(CP_UTF8);" get ignored. This can be seen in this small test
program.
=====================================================
#include <windows.h>
#include <locale.h>
#include <stdio.h>
int main() {
UINT old_cp = GetConsoleOutputCP();
SetConsoleOutputCP(CP_UTF8);
fputs("hello 1: ΓΔΕΘΛΞΠΣΦΨЪЩШЫЮЯ\n", stdout);
setlocale(LC_ALL, "Russian");
//_wsetlocale(LC_ALL, L"Russian");
fputs("hello 2: ΓΔΕΘΛΞΠΣΦΨЪЩШЫЮЯ\n", stdout);
SetConsoleOutputCP(CP_UTF8);
fputs("hello 3: ΓΔΕΘΛΞΠΣΦΨЪЩШЫЮЯ\n", stdout);
SetConsoleOutputCP(old_cp);
}
=====================================================
The effect is the same using MinGW and Visual C++ (2008 Express Edition).
The problem is that a call is made to "_wsetlocale" by wxWidgets in the file
"src\common\intl.cpp". It is done under the auspices of a call to wxSetLocale.
It appears to me that in 2.9.1 the call will ultimately be to "setlocale"
instead, which will be no better for my purposes.
So doing "SetConsoleOutputCP(CP_UTF8);" gets overridden when I set the wxWidgets
locale.
I have struggled to find a way to suppress the call to "_wsetlocale" but still
have the "_()" macro work. Having "_()" work would be good enough for my purposes.
Does anyone have any ideas?
Thanks.
Vadim Zeitlin
CM> Apparently the way that one is supposed do this is make the call
CM> "SetConsoleOutputCP(CP_UTF8);" - see "Console Functions" at
CM> http://msdn.microsoft.com/en-us/library/ms682073.
CM>
CM> But the call to "SetConsoleOutputCP(CP_UTF8);" will be overridden by a call to
CM> "setlocale" or "_wsetlocale". And any subsequent calls to
CM> "SetConsoleOutputCP(CP_UTF8);" get ignored.
This is really strange, I don't understand why should the subsequent calls
be ignored, do you?
CM> So doing "SetConsoleOutputCP(CP_UTF8);" gets overridden when I set the
CM> wxWidgets locale.
In 2.9 setting the locale and translating the strings was separated. The
former is still done by wxLocale but the latter is done by wxTranslations
and you should be able to use wxTranslations without wxLocale.
Notice however that in 2.9 there is another wxSetlocale() call which is
done at startup in wxAppTraitsBase::SetLocale() so you will need to define
a custom traits class overriding this method and doing nothing in it.
Regards,
VZ
--
TT-Solutions: wxWidgets consultancy and technical support
http://www.tt-solutions.com/
Creighton MacDonnell
On 03/01/2011 3:34 PM, Vadim Zeitlin wrote:
> On Mon, 03 Jan 2011 11:09:21 -0700 Creighton MacDonnell<cmacd...@shaw.ca> wrote:
>
> CM> Apparently the way that one is supposed do this is make the call
> CM> "SetConsoleOutputCP(CP_UTF8);" - see "Console Functions" at
> CM> http://msdn.microsoft.com/en-us/library/ms682073.
> CM>
> CM> But the call to "SetConsoleOutputCP(CP_UTF8);" will be overridden by a call to
> CM> "setlocale" or "_wsetlocale". And any subsequent calls to
> CM> "SetConsoleOutputCP(CP_UTF8);" get ignored.
>
> This is really strange, I don't understand why should the subsequent calls
> be ignored, do you?
I can only suppose that the Windows CRT library handling of locales is so tied
to the the consoles code page being the same as the one the locale string
specified that they decided to prevent the code page to be changed while a
locale other than "C" is place.
> CM> So doing "SetConsoleOutputCP(CP_UTF8);" gets overridden when I set the
> CM> wxWidgets locale.
>
> In 2.9 setting the locale and translating the strings was separated. The
> former is still done by wxLocale but the latter is done by wxTranslations
> and you should be able to use wxTranslations without wxLocale.
>
> Notice however that in 2.9 there is another wxSetlocale() call which is
> done at startup in wxAppTraitsBase::SetLocale() so you will need to define
> a custom traits class overriding this method and doing nothing in it.
>
> Regards,
> VZ
>
By the way, I realized later that the translations them selves were working. But
the wxFputs routine that was being used to write the translated strings to the
console had a line "wxCharBuffer buf(wxConvLibc.cWC2MB(ws));" which was not
converting the wide string to UTF-8 as I had expected.
I dealt with my problem by using code like this:
#include <wx/wx.h>
// logger to stream
class etLogStd : public wxLog {
wxLog *old;
FILE *stream;
public:
etLogStd(FILE *s) {
stream = s;
SetTimestamp(NULL);
old = wxLog::SetActiveTarget(this);
}
~etLogStd() {
wxLog::SetActiveTarget(old);
}
protected:
void DoLogString(const wxChar *, time_t);
};
// logger to stream
// To show Unicode on a Windows console the call
// "SetConsoleOutputCP(CP_UTF8)" must be made. But this
// conflicts with the call to "_wsetlocale" made from
// wxWidgets "intl.cpp". So we must restore the "C" locale
// temporarily. And so all calls must be done temporarily,
// for each line of output.
void etLogStd::DoLogString(const wxChar *msg, time_t t) {
wxCharBuffer buf(wxString(msg).ToUTF8());
char *save_locale = strdup(setlocale(LC_ALL, NULL));
setlocale(LC_ALL, "C");
UINT save_cp = GetConsoleOutputCP();
SetConsoleOutputCP(CP_UTF8);
fputs(buf, stream);
fputs("\n", stream);
fflush(stream);
SetConsoleOutputCP(save_cp);
setlocale(LC_ALL, save_locale);
free(save_locale);
};