Improvements for handling default locale (PR #23226)

VZ

unread,

Feb 5, 2023, 11:23:38 AM2/5/23

to wx-...@googlegroups.com, Subscribed

The changes here fix Mac-specific problems discussed in #23114, i.e.

Querying current locale before setting it now returns "C" there, just as everywhere else.
wxLocale::Init(wxLANGUAGE_DEFAULT) doesn't fail any longer with mixed locales.
While GetSystemLocale() still returns wxLANGUAGE_UNKNOWN for such locales, there is a new GetSystemLocaleId() function which allows to get e.g. "en-FR" string for them.

NB: this PR also incorporates #23119 and #23147.

cc @utelle @csomor (Stefan, I've removed Mac-specific code you added back in 9325432 (fixing strange bug under OS X (null strings from within CoreFoundation), 2007-12-19), but I'm pretty sure it's the right thing to do)

You can view, comment on, or merge this pull request online at:

https://github.com/wxWidgets/wxWidgets/pull/23226

Commit Summary

8b69631 Make comment in wxLocale::GetLanguageInfo() even more detailed
4c23721 Add a pseudo test to show system locale and language
ce04dbc Recommend using wxUILocale::GetSystemLanguage() in wxLocale docs
f0b54be Handle wxLANGUAGE_DEFAULT specially in wxLocale::IsAvailable()
d6c041e Don't assume that wxLocale::Init(wxLANGUAGE_DEFAULT) succeeds
98a9cd0 Don't ignore LC_XXX in wxUILocaleImpl::GetPreferredUILanguages()
c13b364 Fix wxUILocale::FindLanguageInfo() to work for mixed locales
9f2a416 Return wxLANGUAGE_UNKNOWN from GetSystemLocale() in mixed locales
1471485 Add wxUILocale::GetSystemLocaleId()
e3146f9 Create valid (if empty) "C" locale under macOS
856c037 Remove outdated special case for Mac from wxSetlocale()

File Changes

(9 files)

Patch Links:

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

Ulrich Telle

unread,

Feb 5, 2023, 3:47:18 PM2/5/23

to wx-...@googlegroups.com, Subscribed

Although I tested only under Windows and Linux up to now, here comes a first feedback. Mac test will follow.

I tested with the internat sample.

Windows (system language German)
- With language and locale de-DE all results as expected.
- With language de-DE and regional formatting fr-FR German translations are used, except for the calendar control where French month and day names are used (but since the system was set to French regional formatting this was to be expected).
- Testing for availability with en_DE results in "locale en-DE / English (Germany) available" - and that's correct.
- Testing for availability with fr_DE results in "locale fr-DE is not available" - and that's correct.
Linux (system language German)
- LC_ALL not set: locale de_DE.utf-8 is used for UI and C locale, all results as expected (except for a blank field under the heading Date)
- Testing for availability with en_DE results in "locale en_DE is available / Identifier en_GB.utf-8 / English (United Kingdom)" - and that is somewhat surprising. Why is the locale available, although it can't be set actually? Why is the identifier en_GB, although the identifier en_DE is contained in the wx language database?
- Testing for availability with fr_DE results in "locale fr_DE is available / Identifier fr_FR.utf-8 / French (France)"" - again that is somewhat surprising. Why is the locale available, although it can't be set actually? Why is the identifier fr_FR?
- LC_ALL=de_DE: locale de_DE is used for UI and C locale, all results as expected (except for a blank field under the heading Date)
- LC_ALL=en_DE:
  - Setting LC_ALL like that results in a shell warning _setlocale: LC_ALL: can't change the locale settings (en_DE): File or directory not found
  - The program asks "Would you like to use the program in English (Germany)?" That is expected, because the locale en_DE is defined in the wx language database.
  - locale C is used for UI and C locale, all results in English as expected (except for a blank field under the heading Date)
- LC_ALL=fr_DE:
  - Setting LC_ALL like that results in a shell warning _setlocale: LC_ALL: can't change the locale settings (fr_DE): File or directory not found
  - The program asks "Would you like to use the program in English (United States)?" That is expected, because the locale fr_DE is not defined in the wx language database.
  - locale C is used for UI and C locale, all results in English as expected (except for a blank field under the heading Date)
IMHO the locales like en_DE and fr_DE should be reported as not available under Linux, because they can't be set via setlocale. If they are accepted, then most likely the expected behaviour would be to use the language part for the UI language and the region part for the regional formatting. However, this doesn't happen, because the actual locale will be C.

I will have to look at the code changes for Linux to understand what is going on.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 5, 2023, 5:28:36 PM2/5/23

to wx-...@googlegroups.com, Subscribed

Thanks for testing! I think the discussion of support for "mixed" locales under Linux should be restricted to #23217. And there is also the more specific #23218 for the "locale is unexpectedly available" part of the problem.

Other than that it looks like you haven't found any problems, right?

Thanks again!

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Ulrich Telle

unread,

Feb 5, 2023, 5:33:26 PM2/5/23

to wx-...@googlegroups.com, Subscribed

Now I tested also under Mac OS Ventura. Again, I tested with the internat sample.

system settings German (Germany)
- locale de_DE is used for UI, locale C/UTF-8/C/C/C/C is used for C locale.
- Window menu and all subentries not translated (due to commit f7d0742 - message catalog wxstd.pot needs to be regenerated and all language catalogs need to be adjusted),
- the calendar control uses English month and day names, numbers correctly use comma as decimal separator,
- the date field under the heading Date is blank instead of showing the current date,
- the locale name in English shows Deutsch (Deutschland) instead of German (Germany) - the required changes to src/osx/core/uilocale.mm were not yet applied.
- Testing for availability with en_DE results in "locale en_DE is available / Identifier en_DE / English (Germany)"
- Testing for availability with fr_DE results in "locale fr_DE is not available_"
system settings German (France)
- locale de_FR is used for UI, locale C/UTF-8/C/C/C/C is used for C locale.
- Testing for availability with de_FR results in "locale de_FR is not available_" - although it corresponds to the system settings,
- for locale name and calendar control same behaviour as above
system settings French (Germany)
- locale fr_DE is used for UI, locale C/UTF-8/C/C/C/C is used for C locale.
- Testing for availability with fr_DE results in "locale fr_DE is not available_" - although it corresponds to the system settings,
- for locale name and calendar control same behaviour as above
system settings English (Germany)
- locale en_DE is used for UI, locale C/UTF-8/C/C/C/C is used for C locale.
- Testing for availability with en_DE results in "locale en_DE is available / Identifier en_DE / English (Germany)"
- for locale name and calendar control same behaviour as above
- the date field under the heading Date is correctly shown formatted as a German date,

I find it interesting that the test for availability fails for de_FR and fr_DE, although the default locale was reported with exactly these values for the corresponding test cases. One would expect that a locale that is reported as the default locale would also be reported as available.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Ulrich Telle

unread,

Feb 5, 2023, 5:59:01 PM2/5/23

to wx-...@googlegroups.com, Subscribed

I think the discussion of support for "mixed" locales under Linux should be restricted to #23217. And there is also the more specific #23218 for the "locale is unexpectedly available" part of the problem.

IMHO under Linux the problem is that it is not detected when setlocale fails - the locale is reported as available nevertheless.

Other than that it looks like you haven't found any problems, right?

There are a few minor issues:

Linux: locales are reported as available, even if setlocale fails for them
Linux and MacOS: the date field under the heading Date is blank quite often. If the language is English the current date is usually shown correctly.
MacOS: method GetLocalizedNames not yet adjusted,
MacOS: Window menu entries not translated
MacOS: calendar control always uses English month and day names
MacOS: some mixed locales reported as not available, even if the default locale is the same
MacOS: all wxstd message catalogs need to be regenerated/adjusted

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 5, 2023, 6:16:43 PM2/5/23

to wx-...@googlegroups.com, Push

@vadz pushed 1 commit.

2ca7644 Fail if environment variables define unknown locale under Unix

—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 6, 2023, 4:07:13 PM2/6/23

to wx-...@googlegroups.com, Subscribed

Concerning this part:

Linux: locales are reported as available, even if setlocale fails for them

This is due to the code in TryCreateMatchingLocale() which explicitly looks for any locale with the same language, even if the region is different. I admit I have no idea why it was done, in spite of doing this myself in 3f8fd90 (Support creating Unix wxUILocale for languages without region, 2021-09-01) -- but the actual code doesn't do what the commit says, as it also creates locales even if we do have a different region.

Thinking more about this, I believe the idea here was to allow choosing a different region using the same language, i.e. if fr_BE is not supported, fall back on fr_FR if it is. This is probably still a good idea (although I'm not even 100% sure about this), but using en_US as fallback for en_FR almost surely is not, right?

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Ulrich Telle

unread,

Feb 6, 2023, 5:09:16 PM2/6/23

to wx-...@googlegroups.com, Subscribed

Concerning this part:

Linux: locales are reported as available, even if setlocale fails for them

This is due to the code in TryCreateMatchingLocale() which explicitly looks for any locale with the same language, even if the region is different. I admit I have no idea why it was done, in spite of doing this myself in 3f8fd90 (Support creating Unix wxUILocale for languages without region, 2021-09-01) -- but the actual code doesn't do what the commit says, as it also creates locales even if we do have a different region.

IMHO a failing call to setlocale is a good indicator that a locale shoud be reported as "not available".

AFAICR the approach in pre-wxUILocale times was similar. However, I find this approach questionable. I didn't change it to not break wxLocale compatibility, but regarding wxUILocale I would rather drop this fallback "strategy".

I'm not sure what the actual expectations of a user are, when he/she uses a "mixed" locale of which it is known that it is not supported by the OS.

The reason for setting a locale like fr_US or de_US is usually that the user is located in the given region, but doesn't speak well the region's main language and therefore prefers to use his native language for the UI, but a regional formatting, so that other people do not misinterpret dates and numbers.

Thinking more about this, I believe the idea here was to allow choosing a different region using the same language, i.e. if fr_BE is not supported, fall back on fr_FR if it is. This is probably still a good idea (although I'm not even 100% sure about this), but using en_US as fallback for en_FR almost surely is not, right?

Well, I'm not really convinced that falling back on a locale with the same language, but a different region is such a good idea. Although it seems to be rather plausible to use fr_FR instead of fr_BE (France and Belgium are neighboring countries after all), it is absolutely not plausible to fall back from en_DE to en_GB - date and number formatting are very different in Germany and the United Kingdom.

Under Windows many language/region combinations are predefined, but not all. Nevertheless, you can set the locale to almost any combination programmatically, even though the settings dialog doesn't offer it. Windows "constructs" internally a locale that tries to match the language and the region as close as possible. For example, for fr_US it would use the French language, but date and number formatting would follow the US rules. This is far better than falling back to a different locale replacing the region in a somewhat obscure manner. However, our code under Windows reports "constructed" locales as "not available". And IMHO that is a good decision.

I think the most logical approach is to allow for date and number formatting (and similar locale dependent things) to set a locale that is actually supported by the OS, and to let the user choose a different language for the UI. This should cover most use cases.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 6, 2023, 6:26:23 PM2/6/23

to wx-...@googlegroups.com, Subscribed

The trouble is that I'd like to apply these fixes to 3.2 too and removing this fallback would definitely be a compatibility breakage, so I don't want to do it there. But I agree that we should stop doing this in master, so I've added a commit doing this.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 6, 2023, 6:26:24 PM2/6/23

to wx-...@googlegroups.com, Push

@vadz pushed 2 commits.

3dfe253 Fix GetSystemLocale() return value for "C" locale
1cf59d3 Remove fallback on locale using different region under Unix

—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 6, 2023, 6:43:05 PM2/6/23

to wx-...@googlegroups.com, Subscribed

Finally, concerning the locales availability under Mac: it turns out en-FR is supported by Cocoa, i.e. it's one of the locales returned by [NSLocale availableLocaleIdentifiers], while neither fr-DE nor de-FR are in this list.

I don't know if it's worth handling the current locale specially, maybe it is, but for now I won't do it.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Ulrich Telle

unread,

Feb 7, 2023, 3:47:01 AM2/7/23

to wx-...@googlegroups.com, Subscribed

Finally, concerning the locales availability under Mac: it turns out en-FR is supported by Cocoa, i.e. it's one of the locales returned by [NSLocale availableLocaleIdentifiers], while neither fr-DE nor de-FR are in this list.

Question: Should we update our language database to include the locales from the above list that are not yet included?

I don't know if it's worth handling the current locale specially, maybe it is, but for now I won't do it.

Since on MacOS there is no way to set (Use()) a locale other than the default locale, the user can always access the current locale to retrieve the information. So, for now it is ok as it is.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 7, 2023, 4:17:28 AM2/7/23

to wx-...@googlegroups.com, Subscribed

Question: Should we update our language database to include the locales from the above list that are not yet included?

I don't know how does macOS determine which locales to support, so probably not -- as I don't see how are we going to maintain it in the future. But if anybody has a source for this data, then perhaps we should...

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 7, 2023, 7:16:22 AM2/7/23

to wx-...@googlegroups.com, Push

@vadz pushed 1 commit.

db14662 Don't use LC_MESSAGES for determining locale on non-glibc systems

—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 8, 2023, 11:31:16 AM2/8/23

to wx-...@googlegroups.com, Subscribed

I'm going to merge this without waiting for more reviews/testing because I want to backport it to 3.2 and for this it should be merged into master first, but any feedback would be still very welcome even later!

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

VZ

unread,

Feb 8, 2023, 11:49:13 AM2/8/23

to wx-...@googlegroups.com, Subscribed

Merged #23226 into master.

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Reply all

Reply to author

Forward