Redesigning the Languages and Fonts & Colors preference panels for BCP 47 support
Gordon P. Hemsley
Our plans for improving BCP 47 (Language Identification) support in
Gecko/Firefox will require an extensive revamping of the Languages and
Fonts & Colors preference panels in the Content tab. Neither of these
interfaces have been significantly changed since the Netscape days and
will not be able to handle the changes we plan to make.
Currently, the Languages pane works by explicitly defining which
languages and language-region combinations the user can set as a
preference. This is a problem because the full list of languages that we
wish to support is about 40 times longer than the languages that are
currently supported, and BCP 47 allows support for an essentially
unlimited combination of subtags, including languages, scripts, regions,
and variants. The current UI for language selection will simply not be
able to handle the substantially increased number of possible
combinations. (As it stands now, users can't even be specific that they
want Portuguese as it is spoken in Portugal [pt-PT], for example, being
forced to choose the generic Portuguese [pt]. Nor can they choose
Spanish as spoken in the United States [es-US], because it is not
presented as a choice. Most people who want to specify their preferred
languages are forced to bypass the Languages UI and edit the
'intl.accept_languages' preference in about:config.)
In addition, support for BCP 47 will eventually lead to font and
encoding selection and fallback using the script subtags for most of
what it needs to do. This led to a discussion about how the various
related preferences will become obsolete and will be able to be
combined. However, this raised the question as to how the Fonts & Colors
preference panel will come into play. There is an initial question about
how often those preferences are changed at all. In addition, the
Advanced panel will somehow need to support the 150+ possible scripts
that BCP 47 will introduce, though there was a question as to whether
that UI was even necessary. There is also the idea that encoding of the
future is UTF-8, so we also may not need the Character Encoding section,
either.
At the moment, these are all questions without answers. They need
further discussion, and it would be helpful to have the assistance of
the UX team in making our decisions.
Much of what we have discussed so far is on the wiki:
https://wiki.mozilla.org/User:GPHemsley/BCP_47
You can also see all the related bugs on the 'bcp47' meta tracking bug
on BMO.
Thanks,
Gordon
--
Gordon P. Hemsley
http://gphemsley.org/ • http://gphemsley.org/blog/
Justin Dolske
> Our plans for improving BCP 47 (Language Identification) support in
> Gecko/Firefox will require an extensive revamping of the Languages and
> Fonts & Colors preference panels in the Content tab.
I feel like there's some context missing here. Why should we do this?
What problem is it solving? What's the benefit to our users?
The wiki page (linked from the bottom of your post) _seems_ to encompass
a rather enormous amount of work, so I think it's rather important to
sort out the benefits and goals before plowing into this.
> Currently, the Languages pane works by explicitly defining which
> languages and language-region combinations the user can set as a
> preference. This is a problem because the full list of languages that we
> wish to support is about 40 times longer than the languages that are
> currently supported, and BCP 47 allows support for an essentially
> unlimited combination of subtags, including languages, scripts, regions,
> and variants.
I have to admit, from an admittedly quick skim of things this sounds
like a spec run out of control. The fact that we already expose a giant
list of language choices already feels like a UI failure (even buried in
a sub-sub-pref panel), and increasing that by 40x to Infinity just makes
that worse. :(
Justin
Gordon P. Hemsley
> On 7/5/11 6:08 PM, Gordon P. Hemsley wrote:
>
>> Our plans for improving BCP 47 (Language Identification) support in
>> Gecko/Firefox will require an extensive revamping of the Languages and
>> Fonts & Colors preference panels in the Content tab.
>
> I feel like there's some context missing here. Why should we do this?
> What problem is it solving? What's the benefit to our users?
Well, for starters, the current UI is insufficient for the modern,
multi-lingual Internet. It is also based on unrealistic expectations
about the way language works—how it is used and how it evolves. The
current design (where given language/region combinations are hard-coded,
so that a new bug has to be filed to request new combinations) has not
been able to keep pace with language use on the Internet, and certainly
will not be sustainable in the future.
In addition, the current UI does not allow the user to specify a
particular script that they might prefer. (For example, Serbian is
written in both Latin and Cyrillic scripts, but a user may only be able
to read one or the other. They should have the ability to tell their
browser which one that is, so that the Internet has the potential to
provide them with the correct page.) There are also language variants
that could come into play; those are less in need of a UI, but I would
discourage explicitly designing against them.
The biggest problem, in fact, is to users of minority languages on the
Web, who cannot even specify their language because it has up until now
been considered too obscure for Mozilla. (Granted, this was by accident,
not intention.) Part of the goals of this project is to return all
languages and their speakers to an equal footing on the Web.
> The wiki page (linked from the bottom of your post) _seems_ to encompass
> a rather enormous amount of work, so I think it's rather important to
> sort out the benefits and goals before plowing into this.
The wiki page does, indeed, encompass a rather enormous amount of work,
though not all of it directly involves the UI.
Nevertheless, I do encourage the discussion. Kevin Scannell (my partner
in this endeavor) and I have discussed the benefits and goal for this
project as a whole, but we welcome any particular thoughts you have from
a usability standpoint.
>> Currently, the Languages pane works by explicitly defining which
>> languages and language-region combinations the user can set as a
>> preference. This is a problem because the full list of languages that we
>> wish to support is about 40 times longer than the languages that are
>> currently supported, and BCP 47 allows support for an essentially
>> unlimited combination of subtags, including languages, scripts, regions,
>> and variants.
>
> I have to admit, from an admittedly quick skim of things this sounds
> like a spec run out of control. The fact that we already expose a giant
> list of language choices already feels like a UI failure (even buried in
> a sub-sub-pref panel), and increasing that by 40x to Infinity just makes
> that worse. :(
>
> Justin
You're correct in your assessment that that's a really long list. But
human language is a global phenomenon, and all languages should be equal
on the Web.
We are aware that 8000 languages is a big number, and we agree that the
current way of displaying them (in simply a long list) is not anywhere
near ideal. That's why we want to get the UX team on board with this, to
help use determine a better way to give users these choices. :)
One possibility that just popped into my head is to display two lists:
We will be keeping a separate list of a couple-hundred localized
language names (similar to how it is now), with a master list of
unlocalized names. Perhaps you can split up the more common list and
have the full list somewhere more hidden, so that the user can go there
if they need to, but needn't have to be bombarded with 8000 languages if
they don't have to.
Again, we don't know the correct course of action. As far as I know,
this level of language handling has never been attempted by a Web
browser, and I fully expect this to be a long and detailed discussion,
with a number of prototype iterations.
That is why I humbly request the input and guidance of the UX team. :)
Gervase Markham
> Well, for starters, the current UI is insufficient for the modern,
> multi-lingual Internet. It is also based on unrealistic expectations
> about the way language works—how it is used and how it evolves. The
> current design (where given language/region combinations are hard-coded,
> so that a new bug has to be filed to request new combinations) has not
> been able to keep pace with language use on the Internet, and certainly
> will not be sustainable in the future.
Could you step back one more level for us?
What does this UI do, what does changing it actually affect as I browse
the web? What would go wrong if we just eliminated it entirely?
What percentage of users do you think actually change the current
settings from whatever their shipped defaults are?
Are the shipped defaults different in localized builds?
Gerv
Gordon P. Hemsley
> On 06/07/11 13:57, Gordon P. Hemsley wrote:
>> Well, for starters, the current UI is insufficient for the modern,
>> multi-lingual Internet. It is also based on unrealistic expectations
>> about the way language works—how it is used and how it evolves. The
>> current design (where given language/region combinations are hard-coded,
>> so that a new bug has to be filed to request new combinations) has not
>> been able to keep pace with language use on the Internet, and certainly
>> will not be sustainable in the future.
>
> Could you step back one more level for us?
Oh, sure. :)
> What does this UI do, what does changing it actually affect as I browse
> the web? What would go wrong if we just eliminated it entirely?
Well, currently, because of its design, it likely isn't used by very
many people. But that's not necessarily because they don't need it—it
could very well be that they don't know about it. (Some survey or
something could provide more concrete information.)
With that being said, I do think that removing it entirely would not be
a good move. The antiquated system in place is better than no system at
all—likely used by the well-intention polyglots who are afraid of (or
unaware of) the ability to manipulate the preference in about:config.
So, that leaves the question of how it affects how you browse the Web.
The idea behind the preference is that it sets what values are sent in
the Accept-Languages header that is sent to the server when you request
a page. The server, along with the well-designed, multilingual site that
it hosts, is then able to use that information to decide what language
to serve its content in. BCP 47 (in particular, RFC 4647) details
exactly how it is to parse that information in order to give you the
best possible choice of language for your personal experience.
Currently, the number of websites using that information is probably
relatively low compared to the potential number of websites that could.
In addition, there are different parsing algorithms (from the most basic
to the most complex), depending on exactly how much information you want
to get out of it. As I understand it, multilingual websites SHOULD
always be using the Accept-Languages header to determine what language
content is provided it.
Unfortunately, and in part due to the limited browser support, there is
often a lack of useful information in this header, requiring websites to
use a supplemental way to determine language choice—or completely
ignoring it altogether (though I'm not sure how common that is).
Basically, the Accept-Languages header is designed to be the primary
(and, likely, sole) source of information that a website should use to
determine what language to serve its content in. But the browser is not
providing the support that such an implementation needs in order to be
successful and (near) universal.
> What percentage of users do you think actually change the current
> settings from whatever their shipped defaults are?
As I've alluded to, the percentage of people who actually change the
default language settings is probably rather small at this point. Of
those that do change it, many likely bypass the UI and edit the
preference directly in about:config. (I know that I am one of those people.)
There are many reasons that a person might not change the default values:
1) They don't speak any language besides English (or whatever their
default value is).
2) They don't know the feature exists to be able to use it.
3) Their language (region, script, etc.) doesn't appear in the list.
etc.
And actually, reason 3 is oversimplified. The different types of
language subtags all create different problems.
For some, it may be (and is—there are bugs on file) that their language
is simply too "obscure" to be provided in the list—nobody thought to add it.
For others, it may be that the regional dialect of their language is not
available as a choice, because no one thought to combine it like that
(like pt-PT or es-US).
Or it may be that their language (plus region, if necessary) is there,
but they cannot specify the script they can read, so they are getting
content in a script they can't read. (For example, consider Serbian,
which is commonly written in either Latin or Cyrillic script. If one
person writes something in Latin script and another person writes
something in Cyrillic script, it may be that the two people would not be
able to read each other's writing, even though if they'd read them aloud
to each other, there would be no problem communicating.)
In addition, much of the Web remains in English because that is the
default language preference in the main browsers, unless it has been
specifically localized otherwise. But a user need not have (or want)
their browser localized into a given language in order to read something
written in that language.
So keeping the UI and improving it to fully support BCP 47, and all the
subtag combinations that entails, would go a long way towards maybe the
Web a truly international and multilingual place.
> Are the shipped defaults different in localized builds?
As far as I know, yes, the defaults are changed in localized builds. I
could be wrong, though—or it may differ by locale. You should probably
ask Axel about those specifics.
> Gerv
Hope that answers your questions.
Justin Dolske
> Well, currently, because of its design, it likely isn't used by very
> many people. But that's not necessarily because they don't need it—it
> could very well be that they don't know about it.
I'd question those assumptions. I'd tend to think that it's because only
a tiny fraction of websites are multilingual, and so most of the time
it's useless.
> The idea behind the preference is that it sets what values are sent in
> the Accept-Languages header that is sent to the server when you request
> a page. The server, along with the well-designed, multilingual site that
> it hosts, is then able to use that information to decide what language
> to serve its content in.
Is there any data on how common this is today (even among sites which
have multiple languages available)?
> Currently, the number of websites using that information is probably
> relatively low compared to the potential number of websites that could.
How will implementing BCP 47 improve this situation? Just 5 or 10
languages likely represent the majority of users... So if adoption if
poor among that huge population I don't understand how adding more
choices helps drive adoption.
Seems like it would be more effective to design a way for the server to
say when language choices are available and have the browser interact
with the user to make a choice and set options for future behavior. But,
then, I'm not sure what your goals are here.
> In addition, much of the Web remains in English because that is the
> default language preference in the main browsers, unless it has been
> specifically localized otherwise. But a user need not have (or want)
> their browser localized into a given language in order to read something
> written in that language.
Trends and demographics of server and user growth are what I'd expect to
have a far stronger influence (in combination with the effort required
to make sites available in more than 1 language).
Justin
Gordon P. Hemsley
> On 7/6/11 11:54 AM, Gordon P. Hemsley wrote:
>
>> Well, currently, because of its design, it likely isn't used by very
>> many people. But that's not necessarily because they don't need it—it
>> could very well be that they don't know about it.
>
> I'd question those assumptions. I'd tend to think that it's because only
> a tiny fraction of websites are multilingual, and so most of the time
> it's useless.
I'm not sure that they're assumptions.
The first sentence was in reference to what I detailed later, provided
by anecdotal evidence and my own person experience: Most people
interested in changing their language settings bypass the UI and edit
the preference directly in about:config. These people don't use the UI
because it is insufficient.
The second sentence refers to a possible reason on the other end of the
spectrum. These people don't use the UI because they don't know it
exists, or because they do not speak multiple language.
However, you also offer up another group of users: those who do not use
the UI because they feel that it would not be beneficial enough to their
web browsing experience.
With that being said, I don't think that we should handicap users who do
visit the sites that are multilingual. Nor should discount out-of-hand
what possible progress might be made in the future regarding the
multilinguality of websites.
In that regard, I would like to immediately take off the table the
option of removing the UI altogether, which I believe would be a grave
mistake. According to the W3C[1], Firefox already allows the most
freedom among the browsers in terms of choice of language. It seems all
the other browsers (except perhaps some versions of Opera) either do not
allow the user to modify these preferences or rely on the OS to tell
them what the user's preferences are. I am of the opinion that Firefox
should increase the freedom in this area even further, not decrease it.
>> The idea behind the preference is that it sets what values are sent in
>> the Accept-Languages header that is sent to the server when you request
>> a page. The server, along with the well-designed, multilingual site that
>> it hosts, is then able to use that information to decide what language
>> to serve its content in.
>
> Is there any data on how common this is today (even among sites which
> have multiple languages available)?
I do not have any data, no, but I do note that such is the preferred way
of doing things, per the W3C[2][3], RFC 4647[4], and the HTTP standard[5].
>> Currently, the number of websites using that information is probably
>> relatively low compared to the potential number of websites that could.
>
> How will implementing BCP 47 improve this situation? Just 5 or 10
> languages likely represent the majority of users... So if adoption if
> poor among that huge population I don't understand how adding more
> choices helps drive adoption.
I understand your point, but I'm not sure that it's valid. The great
majority of users, perhaps, are not likely to benefit at the outset,
because they can make their way around the English or German or French
or whatever language has a significant presence on the Internet without
the need for content language negotiation, simply because they have the
privilege of being in the majority. Rather, it is the speakers of the
minority languages (whether we're talking IRL or just on the Web) that
stand to gain the most, at least at the beginning. It will be those
users who will first be able to advertise that they prefer such a
minority language, and then websites might begin to learn that there is
an audience for such people and begin to provide content for them.
Allowing a user to easily specify that they prefer a minority
language—even if their browser is not yet localized into that
language—will provide the foundation needed for organic and grassroots
communities to grow and thrive.
Currently, these users are being ignored, and are forced to browse the
Web as second-class citizens.
> Seems like it would be more effective to design a way for the server to
> say when language choices are available and have the browser interact
> with the user to make a choice and set options for future behavior. But,
> then, I'm not sure what your goals are here.
Well, my goals here are not to design server software—otherwise I'd be
working with Apache, not Mozilla. ;)
Speaking of Apache, though: they do have support for language
negotiation via MultiViews[6][7], so it need only be turned on to make
it work.
In addition, servers may send a Vary header indicating that they do
indeed have such content language negotiation enabled.[8][9] If we feel
that it would be useful or appropriate, we should by all means include
support for that in our plans. (I'm not sure that's necessarily within
the scope of UX, though.)
>> In addition, much of the Web remains in English because that is the
>> default language preference in the main browsers, unless it has been
>> specifically localized otherwise. But a user need not have (or want)
>> their browser localized into a given language in order to read something
>> written in that language.
>
> Trends and demographics of server and user growth are what I'd expect to
> have a far stronger influence (in combination with the effort required
> to make sites available in more than 1 language).
Well, there's no reason why we can't help nudge that along. Implementing
more freedom in language choice will certainly provide the groundwork
for such trends and demographics to increase.
> Justin
Gordon
[1] http://www.w3.org/International/questions/qa-lang-priorities
[2] http://www.w3.org/International/questions/qa-when-lang-neg
[3] http://www.w3.org/International/questions/qa-accept-lang-locales
[4] http://tools.ietf.org/html/rfc4647
[5] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.4
[6] http://www.w3.org/International/questions/qa-apache-lang-neg
[7] http://httpd.apache.org/docs/current/content-negotiation.html
[8] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.44
[9] http://tools.ietf.org/html/draft-ietf-httpbis-p3-payload-14#section-8.1
Asa Dotzler
> Well, there's no reason why we can't help nudge that along. Implementing
> more freedom in language choice will certainly provide the groundwork
> for such trends and demographics to increase.
More choices does not necessarily mean more freedom. You may actually
limit the number of people who can participate by making it overly
complex. That could be a net loss in freedom.
I see a lot of value in what you're trying to accomplish here, but it
looks like a serious increase in the complexity of the interface really
could not be avoided if the goal is to give users the choices you'd like
them to have.
Has there been any thought on how the browser might just "learn" what
languages users were spending time reading and over time adjust the
settings to let websites know their habit rather than their preference?
I realize that wouldn't give the user a tool to advertise their desires
to Web sites, but it might go farther to actually helping more users get
versions of sites in the languages and scripts they prefer.
This might be a question of how deep do you want to go for a few users
who will fiddle with advanced and complex settings vs how wide you could
go making minor improvements for massive audiences.
- A
Hasse
Dolske wrote...
> > The idea behind the preference is that it sets what values are sent in
> > the Accept-Languages header that is sent to the server when you request
> > a page. The server, along with the well-designed, multilingual site that
> > it hosts, is then able to use that information to decide what language
> > to serve its content in.
>
> Is there any data on how common this is today (even among sites which
> have multiple languages available)?
No data, but for example Google, Facebook, Youtube and Hotmail use the
Accept-Languages header to select language. So do mozilla.com, AMO and
SUMO.
--
Hasse
Hasse
> Are the shipped defaults different in localized builds?
Yes.
http://mxr.mozilla.org/l10n-mozilla-beta/search?string=intl.accept_languages
--
Hasse
Gordon P. Hemsley
> Gordon P. Hemsley wrote:
>
>> Well, there's no reason why we can't help nudge that along. Implementing
>> more freedom in language choice will certainly provide the groundwork
>> for such trends and demographics to increase.
>
> More choices does not necessarily mean more freedom. You may actually
> limit the number of people who can participate by making it overly
> complex. That could be a net loss in freedom.
>
> I see a lot of value in what you're trying to accomplish here, but it
> looks like a serious increase in the complexity of the interface really
> could not be avoided if the goal is to give users the choices you'd like
> them to have.
Well, that's why we want to engage the UX team. It may be that
presenting our full list of everything is not the best course of action.
(And, given the discussion so far, I'm inclined to be believe that.)
Our master list is meant to be there for other portions of the code to
have names associated with every possible subtag. (Spellcheckers, for
example, currently default to displaying the raw tag if a name is not
found—not a very good user experience.)
Our primary goal is get away from the current system where the browser
decides what subtag combinations are possible. Our only requirement is
that the user be able to decide what languages they prefer. We are open
to whatever plan will accomplish that in the best way possible for users.
For example, it may be that the user is presented a search box, where
they type in the language/region/script they're looking for. They need
not be bombarded with the entire 8000+ entry list all at once! :)
> Has there been any thought on how the browser might just "learn" what
> languages users were spending time reading and over time adjust the
> settings to let websites know their habit rather than their preference?
Could you expand upon what you mean by that?
Perhaps by tracking what language various text is tagged in (via the
'lang' attribute)?
> I realize that wouldn't give the user a tool to advertise their desires
> to Web sites, but it might go farther to actually helping more users get
> versions of sites in the languages and scripts they prefer.
There's no reason we couldn't combine the two.
> This might be a question of how deep do you want to go for a few users
> who will fiddle with advanced and complex settings vs how wide you could
> go making minor improvements for massive audiences.
Perhaps. Our overall plan touches many different aspects of the browsing
experience.
But we'll need to discuss how we want to redesign the Languages UI,
given the requirement I mentioned above.
> - A
Gordon
P.S. To be clear, when I use exclusive 'we', I am referring to myself
and Kevin Scannell. (Obviously, when I use inclusive 'we', I'm referring
to all of Mozilla, as well.)
André Neves
> No data, but for example Google, Facebook, Youtube and Hotmail use the
> Accept-Languages header to select language. So do mozilla.com, AMO and
> SUMO.
navigation, usually somewhere in the middle.)
Even though I haven't set it anywhere -since I can't read it- and have
"en-gb, en" as accept-languages.
It is possible you are confusing some sort of location-awareness with
respecting accept-headers.
- ANeves
Hasse
usab...@lists.mozilla.org>, Andr� Neves wrote...
It looks like Google ignores accept-languages headers that have English
as first choice. But try set Swedish, or any other language, as your
preferred language, delete any Google cookies you might have, and visit
http://www.google.com/
I get the chosen language every time.
--
Hasse
Gordon P. Hemsley
> In article<mailman.2022.1310035715.4544.dev-
Yes, I believe you are both right. I think Google does a combination of
Accept-Language detection and IP geolocation to determine your language,
which may not be the best or proper thing to do.
Andr�, do you live in Germany?
--
Gordon P. Hemsley
http://gphemsley.org/ � http://gphemsley.org/blog/
André Neves
> Accept-Language detection and IP geolocation to determine your language,
> which may not be the best or proper thing to do.
part of a minority corner case.
> André, do you live in Germany?
Yes.
Ron Hunter
The trouble with geolocation is that it can often report a location
hundreds of kilometers from where the user actually is, in the case of
home installations. In Germany, this could mean a whole different dialect.
Gordon P. Hemsley
Oh, there are a number of problems with using geolocation to identify
the appropriate language. :)
That's why BCP 47 implementation is so important!
--
Gordon P. Hemsley
http://gphemsley.org/ • http://gphemsley.org/blog/
Gordon P. Hemsley
Gen Kanai just shared this link in mozilla.dev.l10n:
http://googleresearch.blogspot.com/2011/07/languages-of-world-wide-web.html
I don't know how useful it is, but it might be a good place to start.
Blair McBride
> Oh, there are a number of problems with using geolocation to
> identify the appropriate language. :)
Agreed, but it could be used to provide a less overwhelming UI for
selecting languages. By knowing the location, we can narrow down the
list of languages to those that would typically be selected by any
people living in that region (rather than displaying all 8000). It
certainly wouldn't cover everyone, but it would improve the experience
for a lot of people. For languages not in that reduced list, the UI for
the complete list would be no worse than what would be provided otherwise.
- Blair
Gordon P. Hemsley
Sure, if we're still talking about websites, geolocation could be useful
as a secondary tool, but the Accept-Language should always come first, IMO.
But I don't think we'd use geolocation in Firefox anyway, right?
(Just trying to steer the conversation back to the issue at hand. :) )
Gordon
Gervase Markham
> But I don't think we'd use geolocation in Firefox anyway, right?
Why not? There's already a web API for Geolocation.
I think the outcome of this discussion is that we have a clear goal:
1) Set the user's Accept-Language header to the most appropriate value
for them.
We have a proposed method of achieving that goal:
A) Create a UI with 8000 language options and request that the user find
the UI and choose between them.
However, there is significant doubt here as to whether this method
provides the best UX. Methods involving learning, or involving providing
a set of guesses based on the user's location or country, or some
combination of these, would provide a better user experience.
Gerv
Gordon P. Hemsley
> On 11/07/11 02:04, Gordon P. Hemsley wrote:
>> But I don't think we'd use geolocation in Firefox anyway, right?
>
> Why not? There's already a web API for Geolocation.
Sure, but isn't that for the Web? Has that ever been used within the
Firefox UI? Should it?
> I think the outcome of this discussion is that we have a clear goal:
>
> 1) Set the user's Accept-Language header to the most appropriate value
> for them.
Yes, but it also must always remain customizable. It might be OK for use
to guess preliminary choices, but the user must always have a final say.
Perhaps it would be good to allow them to choose their languages on
first startup? (Although even that has its downsides, because there are
likely many more language subtags than the user would even know how to
use—many of them would only be familiar to linguists.)
> We have a proposed method of achieving that goal:
>
> A) Create a UI with 8000 language options and request that the user find
> the UI and choose between them.
Yes, that is one proposal, which I think we universally agree is not
ideal. It is merely an extension of the current UI, which is over 10
years old.
> However, there is significant doubt here as to whether this method
> provides the best UX. Methods involving learning, or involving providing
> a set of guesses based on the user's location or country, or some
> combination of these, would provide a better user experience.
>
> Gerv
>
Yes, agreed. There has to be a better way.
But I do question the use of geolocation for this. Think, for example,
of the Native American/First Nation languages of North America (or the
Aboriginal languages of Australia, if that suits you). There are many of
them. Are you going to go through the list of 8000 language names and
sort them all manually into groups of languages common to an area? You'd
have to, or else you'd be discriminatory to anyone who would actually
like to use those languages, which is something that this implementation
is intended to avoid.
But even excluding that, you'd still have to know enough about the
regions of the world to include that information, even for the more
common languages, which would be much harder to do—it would involve
additional information that we do not have, and we're struggling enough
with the information we do have.
Not to mention the fact that language choice is more than just language.
It is also region and script, plus a variant, if that ever comes in
handy. The best you could do would be to use geolocation to suggest
regions, but even that would be useless for anyone who didn't grow up in
the area where they are using Firefox.
Though it won't solve the major problem of the UI, it might be worth
increasing the emphasis on the Accept-Languages header to give the
locales greater influence over what languages are common in their target
area. Some already do modify that setting (for example, the 'br' locale
[Breton] includes 'fr' [French] as a fallback, because Breton is spoken
in the northwest area of France), but I've seen that others do not. And
that always leaves the 'en-US' default that would probably be used by
users who do not have a better localized choice (unless they have a more
nearby/similar choice).
So, yeah, we've got a lot of brainstorming to do, but we also have to
work around linguistic realities and linguistic equality.
Regards,
Neil
>On 11/07/11 02:04, Gordon P. Hemsley wrote:
>
>
>>But I don't think we'd use geolocation in Firefox anyway, right?
>>
>>
>Why not? There's already a web API for Geolocation.
>
>
My understanding is that's only available to Firefox itself but not to
other Mozilla-based projects.
--
Warning: May contain traces of nuts.
Axel Hecht
Jumping in on this thread at some random point:
I see a few circles:
Localized default (what's sent without user interaction)
Localized default suggestions (offered prominently to users when opening
the dialog, much like the drop down today)
... this can be tweaked via language maps as in the google blog post,
learning from visited content (@lang), geo-location, others.
The full monty, which I can only see being constructive in some form of
search.
Axel
Asa Dotzler
Gordon P. Hemsley wrote:
> On 7/11/11 11:13 AM, Gervase Markham wrote:
>> On 11/07/11 02:04, Gordon P. Hemsley wrote:
>>> But I don't think we'd use geolocation in Firefox anyway, right?
>>
>> Why not? There's already a web API for Geolocation.
>
> Sure, but isn't that for the Web? Has that ever been used within the
> Firefox UI? Should it?
I'd trust Firefox to use it before I'd trust the Web to use it. I don't
know if the API is suitable for chrome access, but if it is, I can't see
why we wouldn't use it. Also, Firefox ships with Web content, like our
Start page, even if that Web content is local to Firefox. That's just to
point out that I don't really see why we'd have an artificial line there
-- except in the other direction. That is, we do and should have APIs
that the browser itself can access but the Web cannot.
>> I think the outcome of this discussion is that we have a clear goal:
>>
>> 1) Set the user's Accept-Language header to the most appropriate value
>> for them.
>
> Yes, but it also must always remain customizable. It might be OK for use
> to guess preliminary choices, but the user must always have a final say.
I think there's a fundamental mis-understanding of what "final say"
means here. I don't think a list of 8000 choices is any say at all for
most users. A big list could be simply incomprehensible and in being so
could actually offer little or no final say.
Usability is critical when we're talking about empowering users. Giving
them features they cannot use doesn't really put them in control even if
it makes us feel like we've done the right thing by giving them choices.
Choices are hard for most users. We shouldn't pretend otherwise.
> Perhaps it would be good to allow them to choose their languages on
> first startup? (Although even that has its downsides, because there are
> likely many more language subtags than the user would even know how to
> use—many of them would only be familiar to linguists.)
>
>> We have a proposed method of achieving that goal:
>>
>> A) Create a UI with 8000 language options and request that the user find
>> the UI and choose between them.
>
> Yes, that is one proposal, which I think we universally agree is not
> ideal. It is merely an extension of the current UI, which is over 10
> years old.
I think the current UI is mostly broken because of its complexity and
making it more complex would put users in less control than they are
today by making it incomprehensible to even more people.
- A
Jesper Kristensen
>> A) Create a UI with 8000 language options and request that the user find
>> the UI and choose between them.
>
> Yes, that is one proposal, which I think we universally agree is not
> ideal. It is merely an extension of the current UI, which is over 10
> years old.
Maybe take a look at other UIs where the user has to choose one from a
huge list. Web sites for booking airplane tickets comes to my mind.
There are a lot of airports in the world, yet you have to select one of
them.
-
Jesper Kristensen
Robert Kaiser
> On 7/11/11 11:13 AM, Gervase Markham wrote:
>> On 11/07/11 02:04, Gordon P. Hemsley wrote:
>>> But I don't think we'd use geolocation in Firefox anyway, right?
>>
>> Why not? There's already a web API for Geolocation.
>
> Sure, but isn't that for the Web? Has that ever been used within the
> Firefox UI? Should it?
Currently, it can't, as far as I know. But that might be something that
can be changed. Still, trying to determine language by geolocation is
inherently broken and I heard lots of people curse Google and others for
e.g. thinking they can speak Spanish just because they were on vacation
in Spain, or assuming they speak German because the majority of people
in Switzerland has that language, even though French and Italian are
large there as well. And I don't even start on actual minority languages
that we should allow them to get as a preferred option when they're
available. People have fought wars on being allowed to speak their
preferred language, let's not forget that.
We need some possibility for users to specify what language they prefer
*personally*, and what languages they understand. The really hard thing
is how to make that easy but allow the necessary complexity. Both the
current UI and an auto-selection (e.g. by geolocation) are inadequate,
we know that. Now let's find some adequate solution. ;-)
Robert Kaiser
--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community should think about. And most of the
time, I even appreciate irony and fun! :)
Gordon P. Hemsley
Yes, I think search/autocomplete is probably a good place to start.
Perhaps allow the localizer/locale to provide a list of default choices
which are presented to the user immediately to confirm or deny, plus a
search box for "if you don't see your desired language here".
Airports are probably a good example, as they have both codes and
human-readable names. However, airline booking UIs are notorious for
being poorly designed, so we'll have to first find one that has gotten
high marks for usability.
One thing that we should keep in mind (and I'm not sure if the airports
always do) is that a language may go by multiple names. The current way
of storing language names does not allow for that, so we'll have to come
up with something new. (Airlines tend to list airports by city, but that
becomes a problem when you have, e.g., New York-JFK and New York-LGA.)
In addition, we mustn't forget that our UI has to also be able to expose
a way to select a region and/or script and/or variant, and must be able
to do so for each language. (Thus, we cannot simply choose one to
blanketly apply to all selected languages.)
Good ideas. Keep 'em coming!
Gordon P. Hemsley
> Currently, it can't, as far as I know. But that might be something that
> can be changed. Still, trying to determine language by geolocation is
> inherently broken and I heard lots of people curse Google and others for
> e.g. thinking they can speak Spanish just because they were on vacation
> in Spain, or assuming they speak German because the majority of people
> in Switzerland has that language, even though French and Italian are
> large there as well. And I don't even start on actual minority languages
> that we should allow them to get as a preferred option when they're
> available. People have fought wars on being allowed to speak their
> preferred language, let's not forget that.
Yeah, I'm experiencing that right now, just in playing around with
various values. Despite being in the United States and not having any
values related to the Latin language (though I do have Latin script), I
am getting Google in Latin!
Still debugging that, but I just thought I'd share.
Gordon P. Hemsley
For those wondering, it appears that Google doesn't support script
subtags. But, rather than ignore them, it processes them incorrectly.
It seems that when it gets a language tag, it just strips characters off
the front until it gets two letters it recognizes: If I send it '*-Latn'
or '*-Latg', which are supposed to mean "anything in Latin script" (the
latter meaning the Gaelic variant), I get the Latin language ('la'); if
I send it '*-Cyrl' ("anything in Cyrillic script"), I get Welsh ('cy').
Even worse, if I send it a valid tag (rather than just a range)—like
'sr-Latn' (which is supposed to be "Serbian in Latin script")—it just
strips off the end to get to a subtag it recognizes, meaning I get
Serbian in Cyrillic script, which I am not guaranteed to be able to
read. (In fact, if I'm going so far as to say that I want the Latin
script, I'm probably even more likely to not be able to read Cyrllic.)
Just thought I'd let you know what I found.
Gervase Markham
> At the moment, these are all questions without answers. They need
> further discussion, and it would be helpful to have the assistance of
> the UX team in making our decisions.
Here is a straw-man proposal:
Preferred Languages For Web Pages
Which country are you in? ( United Kingdom |V)
I speak (in order I don't speak:
of preference):
English <- Welsh
-> Gaelic
[Add]
The country picker is pre-populated by geolocation (IP address-based
geolocation on a per-country level is normally fine) and/or by the
locale, if the locale is strongly country-specific. However, you can
pick another country if you choose. The "I speak" and "I don't speak"
lists are populated initially (when you download) by locale, just as the
Accept-Language header is now.
If you choose a new country, a list of languages associated with that
country appears in the "I don't speak" list. You can drag and drop, or
use the arrows between the lists, to move languages from one box to
another. If you change the country again without having moved any of the
items, those languages disappear again. If you have changed the UI, the
languages remain (in "I don't speak") even when you change the country.
The "Add" button provides a typedown addressing box which allows you to
type in a language name and have it added directly to the "I speak" list.
We would need to make a data file mapping countries to the languages
commonly spoken therein. In order to avoid upset, we would establish
criteria. Something like "any language which is the native tongue of
more than 2% of the population".
There will be an l10n problem here - it is harsh to ask localizers to
translate 8,000 language names into their language. How can we solve
that? Just use English? As long as the typedown addressing also matches
on language codes, that might be OK.
Gerv
Axel Hecht
The language names will be available for l10n, but not required. How we
exactly implement that is a different question.
Regarding this proposal, I'd rather not add countries as intermediate
layer. I expect that it's going to be confrontational wherever it's
useful, and just an unnecessary UI element anywhere else.
Also, there are way more countries than useful entries in a drop down ;-)
Axel
Hasse
Markham wrote...
> We would need to make a data file mapping countries to the languages
> commonly spoken therein. In order to avoid upset, we would establish
> criteria. Something like "any language which is the native tongue of
> more than 2% of the population".
I am not sure that a purely number-based criteria is a good way to
select languages, or avoid upset. In Sweden we have five official
minority languages, but only one of them would qualify if we used 2%
(or even 1%) as the only criteria.
--
Hasse
Gervase Markham
> I am not sure that a purely number-based criteria is a good way to
> select languages, or avoid upset. In Sweden we have five official
> minority languages, but only one of them would qualify if we used 2%
> (or even 1%) as the only criteria.
We could have an OR - official language, or 2% of the population.
Gerv
Gervase Markham
> Regarding this proposal, I'd rather not add countries as intermediate
> layer. I expect that it's going to be confrontational wherever it's
> useful, and just an unnecessary UI element anywhere else.
You would prefer us just to present the user with a list of 8000 languages?
> Also, there are way more countries than useful entries in a drop down ;-)
Ha ha. Lots of websites do it :-) And I think users have become used to
finding their country in such a list. I certainly know very well that
some of them have England, some of them have Great Britain and some of
them have United Kingdom...
Gerv
Axel Hecht
> On 12/07/11 12:26, Axel Hecht wrote:
>> Regarding this proposal, I'd rather not add countries as intermediate
>> layer. I expect that it's going to be confrontational wherever it's
>> useful, and just an unnecessary UI element anywhere else.
>
> You would prefer us just to present the user with a list of 8000 languages?
oh noes, I want them to be able to search instead.
>> Also, there are way more countries than useful entries in a drop down ;-)
>
> Ha ha. Lots of websites do it :-) And I think users have become used to
> finding their country in such a list. I certainly know very well that
> some of them have England, some of them have Great Britain and some of
> them have United Kingdom...
https://github.com/GPHemsley/BCP47/blob/master/regionNames.properties is
the generated list, and with whitespace and stuff, probably still lists
some 200 real countries. That's less than 8k, but still too many, IMHO.
Axel
Gordon P. Hemsley
Who is going to cultivate that data of what languages are spoken in what
countries? That's a whole lot of extra work for little payoff. The lists
we have so far (languages, regions, scripts, etc.) have been provided to
us by the IANA Language Subtag Registry. There is no registry I know of
that connects languages with geolocations.
And what good is the "I don't speak" column? We won't have any use for
such a list. Or did you just mean that as the generic "pick from here" list?
> We would need to make a data file mapping countries to the languages
> commonly spoken therein. In order to avoid upset, we would establish
> criteria. Something like "any language which is the native tongue of
> more than 2% of the population".
As previously mentioned by others, such criteria would be highly biased.
The new audience we are trying to reach have likely been historically
repressed by their national governments (and as such would not have
their language marked as "official") or are in small numbers because
their communities are inherently small.
I think it would be wrong to tie language choice to regions, as these
are large and political boundaries that often do not nearly coincide
with the boundaries of speech communities. Endangered and minority
languages are called that because their number of speakers is small
and/or because there is language use that far outweighs (both socially
and numerically) their use.
> There will be an l10n problem here - it is harsh to ask localizers to
> translate 8,000 language names into their language. How can we solve
> that? Just use English? As long as the typedown addressing also matches
> on language codes, that might be OK.
>
> Gerv
Our current plan is to keep a separate subset of names, culled from a
variety of sources, that would be localized. The rest would default to
their English name. Any additional name could be added to the l10n list
in the future, if deemed necessary.
Thanks for the input!
Gervase Markham
> oh noes, I want them to be able to search instead.
But if they have to search, then they have to define the entire list
manually. (Or do you have a design which doesn't require that?)
Having 200 possible sets of defaults, one per country, means that the
languages the user is most likely to want will be put into the interface
for them. In the majority of cases, you will just pick your country and
it'll get it right. That's a lot less hassle.
Gerv
Gervase Markham
> Who is going to cultivate that data of what languages are spoken in what
> countries? That's a whole lot of extra work for little payoff. The lists
> we have so far (languages, regions, scripts, etc.) have been provided to
> us by the IANA Language Subtag Registry. There is no registry I know of
> that connects languages with geolocations.
We started an effort to map the "shape" of the entire public DNS and did
that without too much effort:
http://www.publicsuffix.org/
If this data is useful, we can crowdsource it. It probably doesn't
change very fast. There are only 200 countries - that's not that many.
> And what good is the "I don't speak" column? We won't have any use for
> such a list. Or did you just mean that as the generic "pick from here"
> list?
The "I don't speak" column is languages that are spoken in the country
you have chosen (or a previous country you chose) but you have decided
not to drag into the "I speak" column. It's a pick list of the most
likely languages.
> As previously mentioned by others, such criteria would be highly biased.
> The new audience we are trying to reach have likely been historically
> repressed by their national governments (and as such would not have
> their language marked as "official") or are in small numbers because
> their communities are inherently small.
I thought this feature was about allowing people to correctly set
Accept-Language. But now you seem to be saying that it's about
correcting perceived slights to small language groups?
> I think it would be wrong to tie language choice to regions,
My plan does not _tie_ them, it makes suggestions. Those suggestions
will be the right suggestions in many or most cases.
> with the boundaries of speech communities. Endangered and minority
> languages are called that because their number of speakers is small
> and/or because there is language use that far outweighs (both socially
> and numerically) their use.
The UI for Firefox should not be driven by the needs of the few, but the
needs of the many.
If it turns out that 99.99% of people don't set this at all, we should
remove the UI and get people to create special-purpose addons for each
set of languages they care about. We've pushed features used by much
more than 0.01% of our user base into addons before.
> Our current plan is to keep a separate subset of names, culled from a
> variety of sources, that would be localized.
But surely this language favouritism will be repressive of the speakers
of minority languages?
;-P
Gerv
Axel Hecht
> On 12/07/11 16:40, Axel Hecht wrote:
>> oh noes, I want them to be able to search instead.
>
> But if they have to search, then they have to define the entire list
> manually. (Or do you have a design which doesn't require that?)
I'd expect to have them search among the localized language names and
the English defaults as grabbed from the registry, something like
https://github.com/GPHemsley/BCP47/blob/master/language.txt.
Also, you shouldn't construct the list completely, ever, I guess, but
adjust the one that comes from the localized value of your browser.
> Having 200 possible sets of defaults, one per country, means that the
> languages the user is most likely to want will be put into the interface
> for them. In the majority of cases, you will just pick your country and
> it'll get it right. That's a lot less hassle.
I think for that majority, the localized defaults are already good.
I guess it's more about minorities, like the Danish in Northern Germany,
Turkish in Berlin (neither language might be a good suggestion for
Oberammergau), or Indian folks in London.
Those people might actually have a special interest in multilingual
sites, too.
Do we have anyone that'd like to experiment with an add-on or two?
Axel
Gordon P. Hemsley
> On 12/07/11 17:51, Gordon P. Hemsley wrote:
>> Who is going to cultivate that data of what languages are spoken in what
>> countries? That's a whole lot of extra work for little payoff. The lists
>> we have so far (languages, regions, scripts, etc.) have been provided to
>> us by the IANA Language Subtag Registry. There is no registry I know of
>> that connects languages with geolocations.
>
> We started an effort to map the "shape" of the entire public DNS and did
> that without too much effort:
> http://www.publicsuffix.org/
>
> If this data is useful, we can crowdsource it. It probably doesn't
> change very fast. There are only 200 countries - that's not that many.
How are you suggesting we use that list? By seeing where a person's IP
address resolves to? Or what?
I'm not sure that crowdsourcing the language-location relationship would
be any better than simply allowing the localizers to include the
information.
Because you have to remember the people who are using Firefox outside of
their native home (either because they're on vacation, or because
they've moved). If they are far removed from where they grew up, there
is a good chance that the language(s) they are looking for would not be
on any region-specific list.
Also, the reason for not doing region-based language lists is surely the
same reason why localized builds are not based on region first, no?
>> And what good is the "I don't speak" column? We won't have any use for
>> such a list. Or did you just mean that as the generic "pick from here"
>> list?
>
> The "I don't speak" column is languages that are spoken in the country
> you have chosen (or a previous country you chose) but you have decided
> not to drag into the "I speak" column. It's a pick list of the most
> likely languages.
So, any information left in that column would be discarded once the
dialog was closed, correct?
>> As previously mentioned by others, such criteria would be highly biased.
>> The new audience we are trying to reach have likely been historically
>> repressed by their national governments (and as such would not have
>> their language marked as "official") or are in small numbers because
>> their communities are inherently small.
>
> I thought this feature was about allowing people to correctly set
> Accept-Language. But now you seem to be saying that it's about
> correcting perceived slights to small language groups?
The two go hand in hand. If you're a speaker of a majority language,
there is less chance that you will need to change the Accept-Language
header, because your localized version of Firefox will have already set
it properly. It is the speaker of the endangered/minority language that
will want to (or have to) add their language to the Accept-Language
header. Or the unusually-multi-lingual person who may speak many
languages not related to or close to each other.
>> I think it would be wrong to tie language choice to regions,
>
> My plan does not _tie_ them, it makes suggestions. Those suggestions
> will be the right suggestions in many or most cases.
Perhaps.
I suppose it would have the benefit of already selecting the region
subtag... but again, it's like putting the cart before the horse, IMO.
>> with the boundaries of speech communities. Endangered and minority
>> languages are called that because their number of speakers is small
>> and/or because there is language use that far outweighs (both socially
>> and numerically) their use.
>
> The UI for Firefox should not be driven by the needs of the few, but the
> needs of the many.
This is about empowering users and supporting the Open Web for speakers
of all languages, not just the most popular.
> If it turns out that 99.99% of people don't set this at all, we should
> remove the UI and get people to create special-purpose addons for each
> set of languages they care about. We've pushed features used by much
> more than 0.01% of our user base into addons before.
Well, don't throw the baby out with the bathwater just yet. A
cause/effect relationship has not yet been established between the
obsolescence of the UI and the number of people who use it.
>> Our current plan is to keep a separate subset of names, culled from a
>> variety of sources, that would be localized.
>
> But surely this language favouritism will be repressive of the speakers
> of minority languages?
>
> ;-P
Yeah, I see what you did there. ;)
True, this is somewhat of a way to kick the can down the road a bit, in
terms of what languages we support to be localized. It's a great
improvement on what we have now, but it's not everybody. This was a
decision that was made to ease the burden on the localizers, who won't
want to translate 8000 language names, plus the names of regions,
scripts, variants, etc. (Although much of that data may already be
available in Unicode's CLDR.)
Also, rather than just ignore everything that isn't intentionally
localized, this new plan allows for at least an English fallback if a
localized version isn't available. (The current system displays the raw
language tag if it doesn't recognize it—including if you simply combine
two subtags it doesn't think should go together. For example, if you
enter 'es-US', it will display that raw, despite knowing that, separate,
'es' is Spanish and 'US' is the United States.)
> Gerv
Gordon
P.S. I apologize for the inordinately large number of clichés used in
this response. I don't know what got into me.
Gervase Markham
> I'd expect to have them search among the localized language names and
> the English defaults as grabbed from the registry, something like
> https://github.com/GPHemsley/BCP47/blob/master/language.txt.
A model which moves to "typedown search for your language name" is more
user-friendly that "pick from an 8000-language list", but I think we can
do better.
>> Having 200 possible sets of defaults, one per country, means that the
>> languages the user is most likely to want will be put into the interface
>> for them. In the majority of cases, you will just pick your country and
>> it'll get it right. That's a lot less hassle.
>
> I think for that majority, the localized defaults are already good.
Right. And what I am suggesting would use those as a starting point. It
would just suggest some other common languages for people from each country.
Gerv
Gervase Markham
> How are you suggesting we use that list? By seeing where a person's IP
> address resolves to? Or what?
By getting users to pick a country. We can default that country
selection either based on the locale they downloaded, or country-level
GeoIP.
> I'm not sure that crowdsourcing the language-location relationship would
> be any better than simply allowing the localizers to include the
> information.
Localizers would certainly be a good source of info for this, but we
don't have localizers everywhere. Also, it would be per country, not per
locale.
> Because you have to remember the people who are using Firefox outside of
> their native home (either because they're on vacation, or because
> they've moved). If they are far removed from where they grew up, there
> is a good chance that the language(s) they are looking for would not be
> on any region-specific list.
That is true. However, the logic behind your statement seems to be that
if a particular mechanism doesn't work for everyone, we can't use it -
i.e. that using this UI must be equally difficult (or easy) for a
Belgian citizen living in Argentina whose first language is Indonesian,
and for an American living in America whose first language is English. I
don't think that's true, because there are far more of the latter than
the former.
> Also, the reason for not doing region-based language lists is surely the
> same reason why localized builds are not based on region first, no?
Localized builds have both language-specific and region-specific
customizations.
> So, any information left in that column would be discarded once the
> dialog was closed, correct?
Well, it wouldn't be part of Accept-Language :-) We could either discard
all of it, or some of it, or keep the list the same so if they reopened
the dialog it was still there as a pick-list. This seems like a small
detail.
>> I thought this feature was about allowing people to correctly set
>> Accept-Language. But now you seem to be saying that it's about
>> correcting perceived slights to small language groups?
>
> The two go hand in hand.
I don't agree. Allowing people to correctly set Accept Language is a
perfectly reasonable function for a browser to perform. Correcting
perceived slights to small language groups is not.
That is not to say that we should not cater for them, but we should not
design our UI around the 1% use case rather than the 99%.
I should say, at this point, before you spend more effort on interacting
with my idea, that I don't have any special say over Firefox's UI. I'm
just a barrack-room UI designer[0] ;-)
Gerv
[0] See "barrack-room lawyer".