RFD: New UI locale API

149 views
Skip to first unread message

Vadim Zeitlin

unread,
Apr 17, 2021, 11:04:34 AM4/17/21
to wx-dev
Hello,

We need to provide replacement for wxLocale because changing C locale is
fatally broken under macOS 11 and we simply can't use wxLocale there in its
current state. In theory, we could try to tweak wxLocale itself and add a
flag telling it not to change C locale by default, but I think this would
be very confusing and it makes more sense to deprecate this class entirely,
due to all the other problems in its (25+ year old) design and the
accumulated workarounds for them, and add some new class for dealing with
the UI locale only, and exclusively, instead.

What should this new class allow us to do? AFAICS the minimum requirements
are just:

1. Do whatever is necessary to use the current user locale in the UI, i.e.
that dates and numbers appear in the format the user expects *without*
changing the C locale. This should be straightforward for MSW and Mac,
as we'd use the same code as now and just skip changing the C locale,
but much less so for GTK where the C locale is used to determine the UI
appearance too. In fact, I only hope that we can make GTK work without
changing the C locale, but I'm not entirely sure about this and I have
really no idea what are we going to do if GTK requires changing C locale
when macOS requires not changing it.

2. Allow retrieving information about the current user locale, i.e. do
what wxLocale::GetInfo() already does. This is definitely simple to do
too and would require just some refactoring of the existing code.

3. Allow getting the information necessary to load the translations for the
current language using wxTranslations: this needs to be discussed in
more details, as "information" here should almost certainly be something
different/more than just a wxLanguage value, both because we don't know
about all languages, and because ideally we'd want to support a list of
languages, order by user preferences, rather than a single language.

4. Update existing code using wxString::{To,From}Double() with user-visible
strings to use wxNumberFormatter::{From,To}String() instead, and change
wxNumberFormatter to use the UI locale, rather than the C one. This
shouldn't be very difficult, but we need to find all the places that
need to be updated, which includes at least (generic) wxSpinCtrlDouble,
wxGrid, wxPropertyGrid and probably others.

There are more things we could provide, of course, but IMO the above
should be enough for a typical application and seems relatively simple to
do. In particular, I don't think we need to provide to change the UI locale
to anything else right now -- while this could, in principle, be useful, it
raises many other questions and I'm not even sure why would anybody want to
do it in the first place.


So I'd like to propose adding a new wxUILocale class (as often, I dislike
the name but I just can't come up with anything better, please let me know
if you have other suggestions) doing just the above. Moreover, as using it
will not affect the global C locale any longer, I see no reason not to
initialize it inside wxApp itself, which would ensure that we use correct
UI locale by default, which used to be briefly the case between 2.8 and
2.9.4, before this was reverted in 2.9.5 to avoid surprises due to the
unexpected C locale change.

Any thoughts about this would be very welcome, thanks in advance!
VZ

Stefan Csomor

unread,
Apr 17, 2021, 11:38:27 AM4/17/21
to wx-...@googlegroups.com
Hi

4. Update existing code using wxString::{To,From}Double() with user-visible
strings to use wxNumberFormatter::{From,To}String() instead, and change
wxNumberFormatter to use the UI locale, rather than the C one. This
shouldn't be very difficult, but we need to find all the places that
need to be updated, which includes at least (generic) wxSpinCtrlDouble,
wxGrid, wxPropertyGrid and probably others.

Just a small thing, but I think - unless we manage to do on all platforms not to change c-locale at all - to keep calls for converting numbers etc. in a stable way like when using "C_NUMERIC" to support externalizing data, regardless which UI Locale we are in.

Another topic: While I don't think we have to provide means to change the visible UI Locale, I think it should be possible to create UI Locales different from the current UI eg by the usual string notation. So that we are able to format a number or a date eg in French or German even when running in an English system. This means that a UI Locale would deliver both eg format patterns for a date so that long dates can have different patterns according to the language, as well as contribute the translation of weekdays. Per default of course this would be the current UI locale.

Best,

Stefan

Vadim Zeitlin

unread,
Apr 17, 2021, 12:18:39 PM4/17/21
to wx-...@googlegroups.com
On Sat, 17 Apr 2021 15:38:24 +0000 Stefan Csomor wrote:

SC> Just a small thing, but I think - unless we manage to do on all
SC> platforms not to change c-locale at all

This is the goal of these changes. If we can't do it, we're probably not
going to change anything at all just because I see no solution that
would be not even good but better than doing nothing. I.e. if we can't make
it work consistently between Linux/GTK and Mac/Cocoa, we'll just have to
document that Apple code is broken and tell people to avoid using wxLocale
in their code under Mac. This is obviously bad but, again, breaking
behaviour under Linux in order to "fix" it under Mac is not better.

But I do hope we can achieve this. E.g. at least for wxSpinCtrlDouble we
can use custom implementation of "input" and "output" signals to convert
the numbers to/from strings ourselves, which will allow us to make this
work as expected without changing the C locale. For wxCalendarCtrl things
are already worse, as it uses strftime(), and so the only way I could find
to make it work was by changing the locale before creating it and resetting
it back afterwards, which is clearly far from ideal, but not *too* bad. I
don't know which other native GTK widgets are affected by the locale (if
anybody else can think of some, please let me know!), but at least we can
deal with those two.

So, again, I do assume we're not going to change C locale at all. And if
we can't do this, I just give up.


SC> to keep calls for converting numbers etc. in a stable way like when
SC> using "C_NUMERIC" to support externalizing data, regardless which UI
SC> Locale we are in.

Sorry, I'm not sure how would this be different from the existing (and
since a very long time) wxString::{From,To}CDouble()? I.e. what exactly
would you like to have for the numbers (we do need some new APIs for the
dates, see below)?


SC> Another topic: While I don't think we have to provide means to change
SC> the visible UI Locale, I think it should be possible to create UI
SC> Locales different from the current UI eg by the usual string notation.
SC> So that we are able to format a number or a date eg in French or German
SC> even when running in an English system.

Is this really a common need? In principle, why not, of course, but I
don't think we need to have it and I don't think I'm going to work on this
unless someone convinces me that it's more important than I think it is.


SC> This means that a UI Locale would deliver both eg format patterns for a
SC> date so that long dates can have different patterns according to the
SC> language, as well as contribute the translation of weekdays.

But you're right that we do need something like wxNumberFormatter for the
dates too. I had somehow decided that wxDateTime::Format() would be enough,
if used with the correct format retrieved from wxUILocale, but it isn't,
because it doesn't use the correct week day and month names if we don't
set C locale. So we're either going to add a new wxDateTimeFormatter class
or add wxDateTime::{Format,Parse}UI() methods -- any arguments for/against
either of these approaches or any other suggestions?


Unfortunately we're going to have the same problem as with wxCalendarCtrl
under Unix with them, i.e. changing locale seems to be the only way to do
it (unless we can rely on xlocale being available everywhere by now?). In
the worst case, the solution will be the same too, i.e. we'll change the
locale once, get all their values, and then change it back -- bad, but
still doable.

Under MSW we can get the values for %a/%A and %b/%B from GetLocaleInfo()
using LOCALE_S[ABBREV]DAYNAME<1-7> and LOCALE_S[ABBREV]MONTHNAME<1-12>,
respectively, and we can handle %p/%P using LOCALE_S[AP]M. But there is
also, as I've just discovered, LOCALE_SSHORTESTDAYNAME<1-7> which is
shorter than the abbreviated name ("Lu" rather than "Lun" for "Lundi" in
French, I think?), so we might need to have a flag for selecting between
them, unless we decide to always use one or the other one.

And under Mac NSDateFormatter should be able to do what we need, although
this will clearly require some work too... But at least, again, so far this
seems doable.


Thanks again for your input (in whichever locale format)!
VZ

Stefan Csomor

unread,
Apr 17, 2021, 1:43:40 PM4/17/21
to wx-...@googlegroups.com
Hi

Am 17.04.21, 18:18 schrieb "Vadim Zeitlin" <wx-...@googlegroups.com im Auftrag von va...@wxwidgets.org>:

On Sat, 17 Apr 2021 15:38:24 +0000 Stefan Csomor wrote:

SC> Just a small thing, but I think - unless we manage to do on all
SC> platforms not to change c-locale at all

This is the goal of these changes. If we can't do it, we're probably not
going to change anything at all just because I see no solution that
would be not even good but better than doing nothing. I.e. if we can't make
it work consistently between Linux/GTK and Mac/Cocoa, we'll just have to
document that Apple code is broken and tell people to avoid using wxLocale
in their code under Mac. This is obviously bad but, again, breaking
behaviour under Linux in order to "fix" it under Mac is not better.

But I do hope we can achieve this. E.g. at least for wxSpinCtrlDouble we
can use custom implementation of "input" and "output" signals to convert
the numbers to/from strings ourselves, which will allow us to make this
work as expected without changing the C locale. For wxCalendarCtrl things
are already worse, as it uses strftime(), and so the only way I could find
to make it work was by changing the locale before creating it and resetting
it back afterwards, which is clearly far from ideal, but not *too* bad. I
don't know which other native GTK widgets are affected by the locale (if
anybody else can think of some, please let me know!), but at least we can
deal with those two.

So, again, I do assume we're not going to change C locale at all. And if
we can't do this, I just give up.

ok

SC> to keep calls for converting numbers etc. in a stable way like when
SC> using "C_NUMERIC" to support externalizing data, regardless which UI
SC> Locale we are in.

Sorry, I'm not sure how would this be different from the existing (and
since a very long time) wxString::{From,To}CDouble()? I.e. what exactly
would you like to have for the numbers (we do need some new APIs for the
dates, see below)?

no, I just wanted this functionality to stay in (and work in ge DE_ch as well __ - which it doesn't right now)

SC> Another topic: While I don't think we have to provide means to change
SC> the visible UI Locale, I think it should be possible to create UI
SC> Locales different from the current UI eg by the usual string notation.
SC> So that we are able to format a number or a date eg in French or German
SC> even when running in an English system.

Is this really a common need? In principle, why not, of course, but I
don't think we need to have it and I don't think I'm going to work on this
unless someone convinces me that it's more important than I think it is.

it's not really expensive to do compared to having just one UILocale, and opens a lot of options, and I'm ready to invest my time into making this work.
A Locale should IMHO be bound to a formatter or to a format - wherever we store the locale pointer - Cocoa does it in the Formatter, ICU in the Format.

SC> This means that a UI Locale would deliver both eg format patterns for a
SC> date so that long dates can have different patterns according to the
SC> language, as well as contribute the translation of weekdays.

But you're right that we do need something like wxNumberFormatter for the
dates too. I had somehow decided that wxDateTime::Format() would be enough,
if used with the correct format retrieved from wxUILocale, but it isn't,
because it doesn't use the correct week day and month names if we don't
set C locale. So we're either going to add a new wxDateTimeFormatter class
or add wxDateTime::{Format,Parse}UI() methods -- any arguments for/against
either of these approaches or any other suggestions?

I'd suggest having a wxDateFormatter class, it can if necessary preprocess the pattern, opens more and cleaner options for implementation and extension.

Thanks,

Stefan


Vadim Zeitlin

unread,
Apr 17, 2021, 2:21:50 PM4/17/21
to wx-...@googlegroups.com
On Sat, 17 Apr 2021 17:43:37 +0000 Stefan Csomor wrote:

SC> SC> to keep calls for converting numbers etc. in a stable way like when
SC> SC> using "C_NUMERIC" to support externalizing data, regardless which UI
SC> SC> Locale we are in.
SC>
SC> Sorry, I'm not sure how would this be different from the existing (and
SC> since a very long time) wxString::{From,To}CDouble()? I.e. what exactly
SC> would you like to have for the numbers (we do need some new APIs for the
SC> dates, see below)?
SC>
SC> no, I just wanted this functionality to stay in (and work in ge DE_ch
SC> as well __ - which it doesn't right now)

I thought using {To,From}Double() didn't work correctly in de_CH locale
under macOS, but using {To,From}CDouble() should still work -- precisely
because it's locale-independent. Or mm I missing some other problem here?

SC> SC> Another topic: While I don't think we have to provide means to change
SC> SC> the visible UI Locale, I think it should be possible to create UI
SC> SC> Locales different from the current UI eg by the usual string notation.
SC> SC> So that we are able to format a number or a date eg in French or German
SC> SC> even when running in an English system.
SC>
SC> Is this really a common need? In principle, why not, of course, but I
SC> don't think we need to have it and I don't think I'm going to work on this
SC> unless someone convinces me that it's more important than I think it is.
SC>
SC> it's not really expensive to do compared to having just one UILocale,
SC> and opens a lot of options, and I'm ready to invest my time into making
SC> this work.

Thanks, and I agree that it probably shouldn't be too difficult, but there
are still going to be some questions, notably about how do we identify
locales in the first place. Using just strings is the simplest, of course,
but also provides minimal (well, no) type-safety... So I'd like to make the
required minimum work first, and then add this.

SC> A Locale should IMHO be bound to a formatter or to a format - wherever
SC> we store the locale pointer - Cocoa does it in the Formatter, ICU in
SC> the Format.

I think formatter should take the locale instead, but there is probably
not much difference between the two approaches.

SC> I'd suggest having a wxDateFormatter class, it can if necessary
SC> preprocess the pattern, opens more and cleaner options for
SC> implementation and extension.

Yes, I also think wxDateTimeFormatter would be better. We also need to
decide which format string format to use, there is Unicode Technical
Standard #35:

https://www.unicode.org/reports/tr35/tr35-31/tr35-dates.html#Date_Format_Patterns

which is used by Apple (hooray for not doing their own thing, for once),
but, of course, to spoil things, MSW uses something similar, yet clearly
different:

https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-date-and-time-format-strings

so I wonder if using this kind of format is not going to end up more
confusing than just sticking to the traditional strftime one...

VZ

Václav Slavík

unread,
Apr 19, 2021, 12:43:25 PM4/19/21
to wx-...@googlegroups.com
Hi,

> What should this new class allow us to do?

I’d say only query, not change, user’s configuration, i.e. language and number/date/calendar/etc. formatting.

> This should be straightforward for MSW and Mac,
> as we'd use the same code as now and just skip changing the C locale,
> but much less so for GTK where the C locale is used to determine the UI
> appearance too. In fact, I only hope that we can make GTK work without
> changing the C locale, but I'm not entirely sure about this and I have
> really no idea what are we going to do if GTK requires changing C locale
> when macOS requires not changing it.

I think C locale is required by GTK+. Unix is the one platform that uses locale by design instead of having inherited it from POSIX.

Supporting it doesn’t require any special treatment in wx: GTK+ applications call setlocale(LC_ALL, “”) during startup (unless you disable it, as wx does). The only risk I can see in leaving GTK+ to its default behavior is that locales using decimal comma would be even less tested by wx users now that they would only risk being used in C functions on Unix. But that’s still better than just giving up on macOS, isn’t it?

> 3. Allow getting the information necessary to load the translations for the
> current language using wxTranslations: this needs to be discussed in
> more details, as "information" here should almost certainly be something
> different/more than just a wxLanguage value, both because we don't know
> about all languages, and because ideally we'd want to support a list of
> languages, order by user preferences, rather than a single language.

Yes. See GetPreferredUILanguage() in src/common/translation.cpp — its implementation calls platform-native API to retrieve priority list of languages. That should be moved into wxUILocale, then, with only selection from available translations left to wxTranslations.

> There are more things we could provide, of course, but IMO the above
> should be enough for a typical application and seems relatively simple to
> do. In particular, I don't think we need to provide to change the UI locale
> to anything else right now -- while this could, in principle, be useful, it
> raises many other questions and I'm not even ]=[sure why would anybody want to
> do it in the first place.

I agree.

Should changing or handling non-default locales be added, I think handling the default UI locale definitely must remain a separate code path, i.e. “default language/locale” must be a distinct value from explicitly set language. It should never have the form of "HandleExplicitLanguage(DetectUserLanguage())”. A lot of issues in wxLocale come from doing just that, and lacking awareness of e.g. new OS locales.

> Any thoughts about this would be very welcome, thanks in advance!

I think, rather strongly, that the new class should not use wxLanguage in its interface, as you alluded to above. The list of wxLanguage values is already outdated, and not in sync with any, let alone _all_ platforms, and this too is a cause of problems in wxLocale. We’ll need language identifiers that can cover everything supported by the underlying OS, including various, and particularly future, OS versions.

These days that pretty much means BCP 47 language tags (fr, fr-FR, de-CH etc.). Modern macOS and Windows[*] switched to using them, web uses them, they are well defined and widely understood.

Regards,
Václav

[*] Some languages/locales supported in Windows 10 don’t have LCID anymore and can only be properly handled through the new tag-based functions.

Vadim Zeitlin

unread,
Apr 19, 2021, 2:46:42 PM4/19/21
to wx-...@googlegroups.com
On Mon, 19 Apr 2021 18:43:18 +0200 Václav Slavík wrote:

VS> > What should this new class allow us to do?
VS>
VS> I’d say only query, not change, user’s configuration, i.e. language and
VS> number/date/calendar/etc. formatting.

I think we should at least keep the possibility of implementing support
for changing the UI locale in the future, even if we (well, I) don't do it
right now. E.g. I definitely can see applications that want to always use
US locale and with some effort I can also image some use cases for using
any other ones (e.g. for a language-learning program, perhaps?).


VS> I think C locale is required by GTK+.

The fact that they have provided gtk_disable_setlocale() is IMO a good
indicator that even GTK developers came to realize it was a mistake.

VS> Supporting it doesn’t require any special treatment in wx: GTK+
VS> applications call setlocale(LC_ALL, “”) during startup (unless you
VS> disable it, as wx does). The only risk I can see in leaving GTK+ to its
VS> default behavior is that locales using decimal comma would be even less
VS> tested by wx users now that they would only risk being used in C
VS> functions on Unix. But that’s still better than just giving up on
VS> macOS, isn’t it?

Just to be clear: removing the call to gtk_disable_setlocale() would
definitely be wrong. We've already gone through this in 2.9.[1-4] days,
when we had decided to change the C locale by default and this broke so
many applications (because they couldn't parse the numbers or input or,
potentially even worse, produced numbers in wrong format on output) that we
had to revert this. I.e. I am absolutely against trying to do it again.

VS> > 3. Allow getting the information necessary to load the translations for the
VS> > current language using wxTranslations: this needs to be discussed in
VS> > more details, as "information" here should almost certainly be something
VS> > different/more than just a wxLanguage value, both because we don't know
VS> > about all languages, and because ideally we'd want to support a list of
VS> > languages, order by user preferences, rather than a single language.
VS>
VS> Yes. See GetPreferredUILanguage() in src/common/translation.cpp — its
VS> implementation calls platform-native API to retrieve priority list of
VS> languages. That should be moved into wxUILocale, then, with only
VS> selection from available translations left to wxTranslations.

OK, thanks.

VS> Should changing or handling non-default locales be added, I think
VS> handling the default UI locale definitely must remain a separate code
VS> path, i.e. “default language/locale” must be a distinct value from
VS> explicitly set language. It should never have the form of
VS> "HandleExplicitLanguage(DetectUserLanguage())”. A lot of issues in
VS> wxLocale come from doing just that, and lacking awareness of e.g. new
VS> OS locales.

I agree, but by now, i.e. after my recent changes and your even more
recent one, this really ought to be fixed in wxLocale itself too. Better
very late than never, I guess...


VS> > Any thoughts about this would be very welcome, thanks in advance!
VS>
VS> I think, rather strongly, that the new class should not use wxLanguage
VS> in its interface, as you alluded to above. The list of wxLanguage
VS> values is already outdated, and not in sync with any, let alone all
VS> platforms, and this too is a cause of problems in wxLocale. We’ll need
VS> language identifiers that can cover everything supported by the
VS> underlying OS, including various, and particularly future, OS versions.
VS>
VS> These days that pretty much means BCP 47 language tags (fr, fr-FR,
VS> de-CH etc.). Modern macOS and Windows[*] switched to using them, web
VS> uses them, they are well defined and widely understood.

Yes, I agree that we should use them (i.e. ISO 639 codes) too, but OTOH if
we don't implement changing the locale for now, the point is somewhat moot.
If do implement it, I think the only thing I'd like to do would be to have
a very thin wxLanguageCode (suggestions for better names are welcome, as
usual...) class instead of using raw strings in the API.

Thanks!
VZ

Václav Slavík

unread,
Apr 20, 2021, 1:04:40 PM4/20/21
to wx-...@googlegroups.com
Hi,

> The fact that they have provided gtk_disable_setlocale() is IMO a good
> indicator that even GTK developers came to realize it was a mistake.

It’s opt-out for special needs, including accommodating users like wx.

> Just to be clear: removing the call to gtk_disable_setlocale() would
> definitely be wrong. We've already gone through this in 2.9.[1-4] days,
> when we had decided to change the C locale by default and this broke so
> many applications (because they couldn't parse the numbers or input or,
> potentially even worse, produced numbers in wrong format on output) that we
> had to revert this. I.e. I am absolutely against trying to do it again.

Have wxUILocale be an explicit opt-in, then, and use C formatting otherwise. But I don’t see how can we ignore GTK+’s need for a locale. Just don’t support wxUILocale there at all? That’s not feasible.

IMHO making wxUILocale opt-in and documenting that opting in switches locale in wxGTK is the only realistic way to handle it.

> VS> These days that pretty much means BCP 47 language tags (fr, fr-FR,
> VS> de-CH etc.). Modern macOS and Windows[*] switched to using them, web
> VS> uses them, they are well defined and widely understood.
>
> Yes, I agree that we should use them (i.e. ISO 639 codes)

It’s a bit more than just that, there’s optional ISO-3166-1 country or country region (e.g. es-419) after the ISO-639-{1,2,3} code, also optional script (sr-Latn, zh-{Hans,Hant} etc). Generally, it’s not treatable as an opaque string e.g. for translations loading.

> too, but OTOH if we don't implement changing the locale for now, the point is somewhat moot.

GetPreferredUILanguages() needs to return a list of languages...

> If do implement it, I think the only thing I'd like to do would be to have
> a very thin wxLanguageCode (suggestions for better names are welcome, as
> usual...) class instead of using raw strings in the API.


Right.

Regards,
Václav

Vadim Zeitlin

unread,
Apr 20, 2021, 1:19:08 PM4/20/21
to wx-...@googlegroups.com
On Tue, 20 Apr 2021 19:04:35 +0200 Václav Slavík wrote:

VS> > The fact that they have provided gtk_disable_setlocale() is IMO a good
VS> > indicator that even GTK developers came to realize it was a mistake.
VS>
VS> It’s opt-out for special needs, including accommodating users like wx.

It would perhaps be flattering to think that we're important enough to
make GTK developers add functions for accommodating wx, but I don't believe
it at all. I.e. if they've added this function, it must have been because
other, much more important GTK users, have requested it.

VS> > Just to be clear: removing the call to gtk_disable_setlocale() would
VS> > definitely be wrong. We've already gone through this in 2.9.[1-4] days,
VS> > when we had decided to change the C locale by default and this broke so
VS> > many applications (because they couldn't parse the numbers or input or,
VS> > potentially even worse, produced numbers in wrong format on output) that we
VS> > had to revert this. I.e. I am absolutely against trying to do it again.
VS>
VS> Have wxUILocale be an explicit opt-in,

I really hope to use it by default. The fact that you need to do something
special in order to have wx{Calendar,DatePicker,Spin}Ctrl work correctly
doesn't seem right at all to me.

VS> then, and use C formatting otherwise. But I don’t see how can we ignore
VS> GTK+’s need for a locale.

Before I waste time on trying to make it work, could you please explain
why do you think it's hopeless? AFAICS it's going to be tricky and maybe
dirty, but doable. For wxSpinCtrl I've already done preparatory work in
https://github.com/wxWidgets/wxWidgets/pull/2334 in fact, and we can use
"input"/"output" signals to do our own formatting cleanly enough. For the
others it looks like we'll need to change the locale during their creation
but could reset it back after doing it. This is clearly bad, but still not
as bad as changing it globally, I think.

Are there any other places where GTK uses the current locale?

VS> IMHO making wxUILocale opt-in and documenting that opting in switches
VS> locale in wxGTK is the only realistic way to handle it.

This would definitely be much simpler, but it would make somewhat of a
mockery of the promise of having a portable API, if it has a globally
significant side effect on some platforms but not the others.

VS> > VS> These days that pretty much means BCP 47 language tags (fr, fr-FR,
VS> > VS> de-CH etc.). Modern macOS and Windows[*] switched to using them, web
VS> > VS> uses them, they are well defined and widely understood.
VS> >
VS> > Yes, I agree that we should use them (i.e. ISO 639 codes)
VS>
VS> It’s a bit more than just that, there’s optional ISO-3166-1 country or
VS> country region (e.g. es-419) after the ISO-639-{1,2,3} code, also
VS> optional script (sr-Latn, zh-{Hans,Hant} etc). Generally, it’s not
VS> treatable as an opaque string e.g. for translations loading.

I'll need to read this in more details then, thanks.

VS> > too, but OTOH if we don't implement changing the locale for now, the point is somewhat moot.
VS>
VS> GetPreferredUILanguages() needs to return a list of languages...

Ah, of course, you're right.

Thanks,
VZ

Václav Slavík

unread,
Apr 21, 2021, 2:10:16 PM4/21/21
to wx-...@googlegroups.com
Hi,

> It would perhaps be flattering to think that we're important enough to
> make GTK developers add functions for accommodating wx,

I meant that this _category_ of users, i.e. bindings/wrappers/port, might have played part in it...

> VS> then, and use C formatting otherwise. But I don’t see how can we ignore
> VS> GTK+’s need for a locale.
>
> Before I waste time on trying to make it work, could you please explain
> why do you think it's hopeless? AFAICS it's going to be tricky and maybe
> dirty, but doable. For wxSpinCtrl I've already done preparatory work in
> https://github.com/wxWidgets/wxWidgets/pull/2334 in fact, and we can use
> "input"/"output" signals to do our own formatting cleanly enough. For the
> others it looks like we'll need to change the locale during their creation
> but could reset it back after doing it. This is clearly bad, but still not
> as bad as changing it globally, I think.

OK, if you’re willing to invest time into “patching up” GTK+ then yes, it’s possible:

1. Change the locale at startup, query for all information wxUILocale needs, revert to “C”
2. If not set yet, set LANGUAGE env. variable from LC_* (may not be needed on modern Linux distributions, I’m not sure how widespread LANGUAGE is these days) to ensure appropriate translations
3. Do whatever is necessary to make certain controls behave at runtime

It’s the 3 that would worry me. _Currently_ it may be enough to change the locale temporarily during creation of controls, but that is an implementation detail of GTK+ that may change any time. It feels to me like you’d just be trading dealing with locale-related issues on Win/Mac for dealing with issues with _not_ having locale set on GTK+.

Depending on the state of GTK+, which I frankly don’t really follow these days, that may be a good trade, or not. If you’re confident this is a reasonable approach, I’m the first to admit I may just be overly pessimistic above.


> Are there any other places where GTK uses the current locale?

I don’t know, which is the point — it may change in the future. Translations of stock windows and dialogs is an obvious one, but that can be taken care by ensuring $LANGUAGE is set. Number and date+time formatting in file open/save dialogs (incl. the ones triggered by file picker control) comes to mind too.

Regards,
Václav

Vadim Zeitlin

unread,
Aug 5, 2021, 2:18:08 PM8/5/21
to wx-...@googlegroups.com
On Wed, 21 Apr 2021 20:10:11 +0200 Václav Slavík wrote:

VS> OK, if you’re willing to invest time into “patching up” GTK+ then yes,
VS> it’s possible:
VS>
VS> 1. Change the locale at startup, query for all information wxUILocale needs, revert to “C”
VS> 2. If not set yet, set LANGUAGE env. variable from LC_* (may not be needed on modern Linux distributions, I’m not sure how widespread LANGUAGE is these days) to ensure appropriate translations
VS> 3. Do whatever is necessary to make certain controls behave at runtime
VS>
VS> It’s the 3 that would worry me. Currently it may be enough to change
VS> the locale temporarily during creation of controls, but that is an
VS> implementation detail of GTK+ that may change any time. It feels to me
VS> like you’d just be trading dealing with locale-related issues on
VS> Win/Mac for dealing with issues with not having locale set on GTK+.
VS>
VS> Depending on the state of GTK+, which I frankly don’t really follow
VS> these days, that may be a good trade, or not. If you’re confident this
VS> is a reasonable approach, I’m the first to admit I may just be overly
VS> pessimistic above.

Unfortunately you were rightfully pessimistic and I was atypically but,
more importantly, wrongly, optimistic. After spending some time on this, it
turns out that there is really no way to make GTK behave correctly without
setting the C locale.

Which means we're back to square one: we need to call setlocale() with
wxGTK (or, alternatively, stop calling gtk_disable_setlocale() to let GTK
do it on its own), but we can't call setlocale() with wxOSX as it breaks
menu rendering there (even with the latest 11.5.1 macOS version).

Using wxUILocale would help for MSW/Mac users as it would at least allow
to avoid calling setlocale() there, while still providing a way to retrieve
locale-specific information, but it's really not that much better than
using wxLocale::GetOSInfo() and so it is probably not even worth adding it,
is it?

So the only thing I can see to do for 3.2.0 is to allow using wxLocale
without calling setlocale() under MSW and Mac. The question is whether we
should still continue doing it by default and just add
wxLOCALE_DONT_SET_CLOCALE flag (which means that menus will still be broken
by default under Mac) or stop doing it by default and add some
wxLOCALE_SET_C_LOCALE flag to force doing it even when on the systems where
it's not necessary.

What do you think would be better?
VZ

Mart Raudsepp

unread,
Aug 6, 2021, 7:04:53 AM8/6/21
to wx-...@googlegroups.com
Ühel kenal päeval, N, 05.08.2021 kell 20:18, kirjutas Vadim Zeitlin:
> Which means we're back to square one: we need to call setlocale()
> with wxGTK (or, alternatively, stop calling gtk_disable_setlocale()
> to let GTK do it on its own), but we can't call setlocale() with
> wxOSX as it breaks menu rendering there (even with the latest 11.5.1
> macOS version).

What does GTK do on Mac?
I don't see anything special there, what exactly is the menu problem on
Mac with wxOSX? I'm probably just missing a link to the appropriate ml
thread.

As a random related thing while poking around there, it looks like GTK
also gets text direction from language, which in turns means which way
some UI elements are layed out as well in case of horizontal layout,
not just text.


Mart

Vadim Zeitlin

unread,
Aug 6, 2021, 9:03:28 AM8/6/21
to wx-...@googlegroups.com
On Fri, 06 Aug 2021 14:03:30 +0300 Mart Raudsepp wrote:

MR> Ühel kenal päeval, N, 05.08.2021 kell 20:18, kirjutas Vadim Zeitlin:
MR> > Which means we're back to square one: we need to call setlocale()
MR> > with wxGTK (or, alternatively, stop calling gtk_disable_setlocale()
MR> > to let GTK do it on its own), but we can't call setlocale() with
MR> > wxOSX as it breaks menu rendering there (even with the latest 11.5.1
MR> > macOS version).
MR>
MR> What does GTK do on Mac?

I don't know, but GTK probably doesn't use Cocoa menus (or does it? To be
honest, I have no idea if you can emulate the Mac menu on your own).

MR> I don't see anything special there, what exactly is the menu problem on
MR> Mac with wxOSX? I'm probably just missing a link to the appropriate ml
MR> thread.

It's https://trac.wxwidgets.org/ticket/19023 and the root of the problem
is a bug in CoreSVG Mac framework, which is used to draw all the small UI
elements such as menu check and radio marks and toolbar overflow buttons in
macOS 11, and just doesn't work in any locale using comma as decimal
separator.

Regards,
VZ

Mart Raudsepp

unread,
Aug 6, 2021, 10:33:06 AM8/6/21
to wx-...@googlegroups.com
Ühel kenal päeval, R, 06.08.2021 kell 15:03, kirjutas Vadim Zeitlin:
> On Fri, 06 Aug 2021 14:03:30 +0300 Mart Raudsepp wrote:
>
> MR> Ühel kenal päeval, N, 05.08.2021 kell 20:18, kirjutas Vadim
> Zeitlin:
> MR> > Which means we're back to square one: we need to call setlocale()
> MR> > with wxGTK (or, alternatively, stop calling
> gtk_disable_setlocale()
> MR> > to let GTK do it on its own), but we can't call setlocale() with
> MR> > wxOSX as it breaks menu rendering there (even with the latest
> 11.5.1
> MR> > macOS version).
> MR>
> MR> What does GTK do on Mac?
>
>  I don't know, but GTK probably doesn't use Cocoa menus (or does it? To
> be honest, I have no idea if you can emulate the Mac menu on your own).

If you mean the global application menu, I think it does:
https://gitlab.gnome.org/GNOME/gtk/-/blob/master/gtk/gtkapplication-quartz-menu.c

Note that this is a link to currently GTK4 state of things. I would be
surprised if it doesn't work, as it just recently got a brand new GDK
backend as well.
If anyone has tested in the same kind of settings, is a different
question. Or if CoreSVG gets involved indirectly or not.


Mart

Vadim Zeitlin

unread,
Aug 6, 2021, 10:42:41 AM8/6/21
to wx-...@googlegroups.com
On Fri, 06 Aug 2021 17:31:43 +0300 Mart Raudsepp wrote:

MR> Ühel kenal päeval, R, 06.08.2021 kell 15:03, kirjutas Vadim Zeitlin:
MR> > On Fri, 06 Aug 2021 14:03:30 +0300 Mart Raudsepp wrote:
MR> >
MR> > MR> Ühel kenal päeval, N, 05.08.2021 kell 20:18, kirjutas Vadim
MR> > Zeitlin:
MR> > MR> > Which means we're back to square one: we need to call setlocale()
MR> > MR> > with wxGTK (or, alternatively, stop calling
MR> > gtk_disable_setlocale()
MR> > MR> > to let GTK do it on its own), but we can't call setlocale() with
MR> > MR> > wxOSX as it breaks menu rendering there (even with the latest
MR> > 11.5.1
MR> > MR> > macOS version).
MR> > MR>
MR> > MR> What does GTK do on Mac?
MR> >
MR> >  I don't know, but GTK probably doesn't use Cocoa menus (or does it? To
MR> > be honest, I have no idea if you can emulate the Mac menu on your own).
MR>
MR> If you mean the global application menu, I think it does:
MR> https://gitlab.gnome.org/GNOME/gtk/-/blob/master/gtk/gtkapplication-quartz-menu.c

This code definitely uses NSMenu, but I'm not sure if it also uses
NSMenuItems, perhaps they use something custom for the menus contents? Of
course, we do use, and want to use, NSMenuItem, so even if GTK doesn't
suffer from this problem, it doesn't help us to solve it.

MR> Note that this is a link to currently GTK4 state of things. I would be
MR> surprised if it doesn't work, as it just recently got a brand new GDK
MR> backend as well.

If they do use NSMenuItem, I don't see how could it work on macOS 11+. As
the stack trace in our bug shows, the problem happens deep inside system
code without any wx code being involved at all.

Regards,
VZ

John Ralls

unread,
Aug 6, 2021, 12:06:40 PM8/6/21
to wx-...@googlegroups.com
It does. Gtk has been able to use the macOS menu bar for well over 10 years. There are two ways to do so, https://gitlab.gnome.org/GNOME/gtk-mac-integration and https://gitlab.gnome.org/GNOME/gtk/-/blob/gtk-3-24/gtk/gtkapplication-quartz-menu.c. Many popular Gtk applications including Ardour, Geany, Gimp, GnuCash, and Inkscape are available as Mac applications using one or the other method.

It works perfectly well on macOS 11 and 12, though we've gotten reports of a window sizing issue on macOS 12 with scaled HiDPI screens that's not related to using the macOS menu bar.

Gtk4 has a complete rewrite of the Gdk-Quartz backend but the quartz-specific code in gtk is inherited from Gtk3.

To use any of that you must compile Gtk with (for Gtk3) -Dquartz-backend=true -Dx11-backend=false. It doesn't make sense to me that anyone would use a Quartz Gtk build with wxWidgets on macOS: Either use Gtk's quartz backend with a Gtk application or use wxWidget's Quartz build with a wxWidgets application. Mixing the two is asking for trouble.

Regards,
John Ralls

Vadim Zeitlin

unread,
Aug 6, 2021, 12:13:22 PM8/6/21
to wx-...@googlegroups.com
On Fri, 6 Aug 2021 09:06:34 -0700 John Ralls wrote:

JR> It does. Gtk has been able to use the macOS menu bar for well over 10
JR> years.

Just to be clear, I have no doubts about this. The question is whether
check and radio menu items appear correctly in it in a locale using comma
as a decimal separator, such as German or French or many others. If you
have a possibility to test GTK under macOS 11 system, please try doing it
in a simplest possible program using a check menu item and calling
setlocale(LC_ALL, "de_DE"), for example.

As I said before, I'm pretty sure you will see the same problem, i.e. the
check mark won't appear and you will see the following message logged to
the controlling terminal:

CoreSVG has logged an error. Set environment variabe "CORESVG_VERBOSE" to learn more.

But if I'm wrong and GTK has somehow found a way around it, it would
definitely be great!

TIA,
VZ

John Ralls

unread,
Aug 6, 2021, 10:39:05 PM8/6/21
to wx-...@googlegroups.com
Yes, it does indeed have that error with gtk-mac-integration. I don't think that gtkapplicationmenu-quartz implements checked menu items. This is the first I've heard of the problem and I just checked gitlab issues to make sure I hadn't missed a report.

The error comes up when the menu attempts to draw so I suspect that the images are in SVG and the private CoreSVG.framework uses a locale-sensitive parser to parse the SVG, and that fails when it encounters a C-locale number like 1.234 when the locale has a different character for the decimal point. I'll file a bug with Apple about that once I make sure that it's still a problem in macOS 12. Apple seems not to care much about POSIX locales though so it may get ignored like my 8 year old bug about de_CH having the wrong thousands separator.

There are properties of NSMenuItem for onStateImage, offStateImage, and mixedStateImage so it should be possible to work around the problem by setting those to a not-SVG based NSImage. I started to experiment with that this afternoon but ran out of time.

The CoreSVG framework seems to have appeared first in MacOSX.10.15.sdk. I don't have a 10.15 (Catalina) VM handy to test with, has anyone tested to be sure that it works there? I did test on 10.14: No CoreSVG.framework and no problems with NSMenuItem onStateImage.

Regards,
John Ralls

Vadim Zeitlin

unread,
Aug 7, 2021, 11:46:16 AM8/7/21
to wx-...@googlegroups.com
On Fri, 6 Aug 2021 19:39:00 -0700 John Ralls wrote:

JR> Yes, it does indeed have that error with gtk-mac-integration.

Thanks for confirming this!

JR> The error comes up when the menu attempts to draw so I suspect that the
JR> images are in SVG and the private CoreSVG.framework uses a
JR> locale-sensitive parser to parse the SVG, and that fails when it
JR> encounters a C-locale number like 1.234 when the locale has a different
JR> character for the decimal point.

This is exactly what I think happens too. Unfortunately I couldn't find
any way to work around the problem, e.g. I hoped to find a way to call
setlocale(LC_NUMERIC, "C") just before drawing and restore the previous
locale after doing it, but there doesn't seem to be any good place to do
this (and, of course, changing locale like this would be far from ideal
anyhow because it affects the other threads and could result in some really
difficult to find bugs when something would fail in a worker thread just
because the user happened to open a menu in the main thread at just the
wrong moment).

JR> I'll file a bug with Apple about that once I make sure that it's still
JR> a problem in macOS 12. Apple seems not to care much about POSIX locales
JR> though so it may get ignored like my 8 year old bug about de_CH having
JR> the wrong thousands separator.

Unfortunately I'm afraid you're right about this too, but please do still
file this bug, just in case we're both being over-pessimistic.

JR> There are properties of NSMenuItem for onStateImage, offStateImage, and
JR> mixedStateImage so it should be possible to work around the problem by
JR> setting those to a not-SVG based NSImage. I started to experiment with
JR> that this afternoon but ran out of time.

This could provide a work around for GTK, but wx really wants to use
native appearance and this affects more than menu items (e.g. the toolbar
overflow button is also drawn by CoreSVG), so unfortunately I don't think
this can help us.

JR> The CoreSVG framework seems to have appeared first in MacOSX.10.15.sdk.
JR> I don't have a 10.15 (Catalina) VM handy to test with, has anyone
JR> tested to be sure that it works there?

Yes, it does, see https://trac.wxwidgets.org/ticket/19023#comment:14

Regards,
VZ

John Ralls

unread,
Aug 7, 2021, 4:25:35 PM8/7/21
to wx-...@googlegroups.com


> On Aug 7, 2021, at 8:46 AM, Vadim Zeitlin <va...@wxwidgets.org> wrote:
>
> JR> I'll file a bug with Apple about that once I make sure that it's still
> JR> a problem in macOS 12. Apple seems not to care much about POSIX locales
> JR> though so it may get ignored like my 8 year old bug about de_CH having
> JR> the wrong thousands separator.
>
> Unfortunately I'm afraid you're right about this too, but please do still
> file this bug, just in case we're both being over-pessimistic.

Done. I was able to make a demo program by adding only `setlocale(LC_ALL, "");` to main() in the objective-c app template in Xcode.

Fingers crossed.

Regards,
John Ralls

John Ralls

unread,
Mar 15, 2022, 2:56:51 PM3/15/22
to wx-...@googlegroups.com
And... it worked!

I got a notification from Apple yesterday that they think my issue was resolved in the 12.3 release. I tested today and the test app that I'd included in my bug report correctly displays its check-mark next to the currently open window in the Window menu.

Regards,
John Ralls

Vadim Zeitlin

unread,
Mar 16, 2022, 6:46:02 PM3/16/22
to wx-...@googlegroups.com
On Tue, 15 Mar 2022 11:56:39 -0700 John Ralls wrote:

JR> > On Aug 7, 2021, at 1:25 PM, John Ralls <jra...@ceridwen.us> wrote:
JR> >
JR> >> On Aug 7, 2021, at 8:46 AM, Vadim Zeitlin <va...@wxwidgets.org> wrote:
JR> >>
JR> >> JR> I'll file a bug with Apple about that once I make sure that it's still
JR> >> JR> a problem in macOS 12. Apple seems not to care much about POSIX locales
JR> >> JR> though so it may get ignored like my 8 year old bug about de_CH having
JR> >> JR> the wrong thousands separator.
JR> >>
JR> >> Unfortunately I'm afraid you're right about this too, but please do still
JR> >> file this bug, just in case we're both being over-pessimistic.
JR> >
JR> > Done. I was able to make a demo program by adding only `setlocale(LC_ALL, "");` to main() in the objective-c app template in Xcode.
JR> >
JR> > Fingers crossed.
JR>
JR> And... it worked!

Wow, that was an unexpected twist.

JR> I got a notification from Apple yesterday that they think my issue was
JR> resolved in the 12.3 release. I tested today and the test app that I'd
JR> included in my bug report correctly displays its check-mark next to the
JR> currently open window in the Window menu.

Great, thanks! I'll update the documentation to say that wxLocale can be
used again with 12.3+. We'll still recommend using wxUILocale instead in
the new code however.

Thanks again for reporting this bug to Apple!
VZ
Reply all
Reply to author
Forward
0 new messages