Regression: GetBestTranslation() returns suboptimal and/or inconsistent results (Issue #26044)
arch1t3cht
arch1t3cht created an issue (wxWidgets/wxWidgets#26044)Description
Bug description:
After 57e1c9e, wxTranslations::GetBestTranslation() can return a language that is not the user's display language. What language is returned can change depending on when the function is called.
Specifically, consider the following situation:
- On Windows, the user's display language is set to
English (United Kingdom), but the user also has a (say) Japanese language pack installed, e.g. for a Japanese keyboard. - A Japanese translation is available for the catalog in question
(new wxTranslations())->GetBestTranslation()is called very early on. I'm not quite sure how early it needs to be, but for example it needs to come before any call towxMessageBox().
In such a case, GetBestTranslation() will return ja even though this is not the user's display language. (I'm guessing this is due to the fact that en_GB does not match en exactly, but ja does match ja exactly?)
If wxMessageBox() is called before (new wxTranslations())->GetBestTranslation(), the call returns en instead, so that the results are inconsistent.
Patch or snippet allowing to reproduce the problem:
Apply the following diff to current master:
diff --git a/samples/internat/internat.cpp b/samples/internat/internat.cpp index 5f6072e7f8..bd36dede4f 100644 --- a/samples/internat/internat.cpp +++ b/samples/internat/internat.cpp @@ -203,6 +203,15 @@ bool MyApp::OnInit() if ( !wxApp::OnInit() ) return false; + wxFileTranslationsLoader::AddCatalogLookupPathPrefix("."); + + // wxMessageBox("Hello"); + + wxTranslations* const trans = new wxTranslations(); + wxMessageBox(wxString::Format("Best translation is %s. %d available translations.", trans->GetBestTranslation("internat"), (int)trans->GetAvailableTranslations("internat").size())); + + return false; + // For demonstration purposes only, ask the user if they want to run the // program using the current system language. In real programs, we would do // it unconditionally for localized programs -- or never do it at all for
To Reproduce:
image.png (view on web)- Configure the Windows system language as in the above screenshot.
- Apply the above patch and compile and run
samples/internat. - Observe that the language
jais returned. - Uncomment the
wxMessageBoxcall and compile and run the sample again. - Observe that
enis returned now.
Note / Question
This came up downstream in Aegisub (relevant code) when updating to wxWidgets >= 3.3. Aegisub's current language selection logic on first startup is:
- Select the result of
GetBestTranslation()if it is nonempty - If not, show a language selection prompt to the user, with the result of
wxLocale::GetSystemLanguage()at the top.
Reading through the internat sample program, which only calls GetSystemLanguage() (through wxLANGUAGE_DEFAULT), I'm not quite sure if this is the correct way to go about this. If this issue is really just due to incorrect wxWidgets usage, do let me know.
Platform and version information
- wxWidgets version you use: master. I bisected the behavior change to 57e1c9e.
- wxWidgets port you use: wxMSW
- OS and its version: Windows 11. I didn't yet run into it on other operating systems, but I'm also not yet sure how to produce such a system language setup on Linux or Mac.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
VZ
vadz left a comment (wxWidgets/wxWidgets#26044)I don't know if using Japanese translations is intended if the corresponding language pack is installed, but it's definitely wrong that the result changes depending on when the function is called. @utelle Please let me know if you have any idea why does this happen, otherwise I'll try to debug it when I can (not before the end of the year).
I also believe that your use of GetBestTranslation() is exactly how it's supposed to be used, but this function has changed several times, so I'm not really sure any more and will cc @vslavik just in case I'm wrong about it.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ulrich Telle
utelle left a comment (wxWidgets/wxWidgets#26044)I don't know if using Japanese translations is intended if the corresponding language pack is installed,
If the Japanese language pack is installed, Japanese will be included in the list of user-preferred languages. That is, depending on the available translations it could possibly be selected as the best translation - unless there are "better" options available.
but it's definitely wrong that the result changes depending on when the function is called.
I agree. The result should not depend on when the function is called. However, I don't know exactly what happens within wxMessageBox - it could have side effects indirectly influencing the behaviour of GetBestTranslation.
@utelle Please let me know if you have any idea why does this happen, otherwise I'll try to debug it when I can (not before the end of the year).
At the moment I don't know why this happens. I have to look into the code in more detail. However, most likely this will not happen before December 27.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Václav Slavík
vslavik left a comment (wxWidgets/wxWidgets#26044)I also believe that your use of GetBestTranslation() is exactly how it's supposed to be used
Yes, it is.
After 57e1c9e, wxTranslations::GetBestTranslation() can return a language that is not the user's display language.
Note the explanation in the screenshot: "Microsoft Store apps will appear in the first supported language in this list”. wx goes out of its way to mimic modern apps' (“Microsoft Store apps”; what MS means by that is “WinUI apps” I believe) language preference logic, but that’s IMHO reasonable.
The surprising thing (besides the wxMessageBox one) is that en isn’t considered the best match for {"en-GB", "ja"} preference from {"en", "ja", ...} available set. There’s a test for this in the suite (albeit with en-US, but that shouldn’t matter) here.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ulrich Telle
utelle left a comment (wxWidgets/wxWidgets#26044)I think I found the cause of the problem. Method wxLocaleIdent::GetBestMatch() uses internally indirectly global data structures (containing parameters about matching locales). If GetBestTranslation() is called, before those global data structures are initialized, the internal algorithm will return a more or less random result.
A side effect of calling wxMessageBox() is that the global data structures are initialized. Therefore the correct result should be returned if GetBestTranslation() is called thereafter.
The solution will be to call wxUILocaleImpl::CreateLanguagesDB() most likely within method wxUILocaleImpl::GetMatchDistance() to be on the safe side. I have to check carefully whether this will be enough or whether there are other methods implicitly depending on the global data structures.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
VZ
vadz left a comment (wxWidgets/wxWidgets#26044)Sorry, I didn't have time to look at this yet, but using any globals not initialized on demand by wrapping them in static functions (which ensures initialization on first access) or using wxModule is definitely a problem, I don't know how could I miss this during the reviews :-(
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ulrich Telle
utelle left a comment (wxWidgets/wxWidgets#26044)Sorry, I didn't have time to look at this yet, but using any globals not initialized on demand by wrapping them in static functions (which ensures initialization on first access) or using
wxModuleis definitely a problem, I don't know how could I miss this during the reviews :-(
Well, the database of languages of wxWidgets has always (at least as far back as version 2.8) been a global structure that was initialized on first access from static functions. I added only some additional arrays for the new functionality in wxUILocale. In most places the initialization function is called from static functions which need access to the data. But I have to admit that I overlooked it for method GetBestMatch().
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.