SuMo KB in bn-BD & bn-IN locales :: A new perspective

[Ashickur Rahman]

Dear All,

I am writing this mail to speak on behalf of SuMo warriors of Bangladesh.
SuMo stands for Support Mozilla (https://support.mozilla.org/); a group of
Mozillians and Mozilla staff together responds to any support need from
various Mozilla Product / Services users. The support is provided in many
languages (https://support.mozilla.org/en-US/locales) for non-English
speaking users.

We a group of volunteer from Bangladesh render the support for our citizens
who uses Mozilla products; regardless the UI language. The way it works can
be described somewhat like this, there are categorized Acknowledge article
exists in English; we translated those article in our language and put in a
bn-BD (language ISO Code) specific URL within Mozilla.org domain. The
articles are called as Knowledge base; hence SuMo KB is a form we use to
term the articles.

We have translated and localized many SuMo KB and also successfully
participated in localization challenges pitched from Mozilla to boost the
number of SuMo KB in regional languages.

As we all know, Citizens of Paschimbanga, India also speaks Bangla the same
language. They do have SuMo warriors too; they also translate SuMo KB in
Bengali (bn-IN) for their users. They are also a great team and consist of
awesome dynamic Mozillians.

For a period of time, Mozilla assessed the articles and found very little
differences in the final articles after translated into bn-BD and bn-IN; on
the contrary significant amount of energy and resources are being used as
input for an almost similar output. This is something we have been doing
unknowingly. Mozilla promotes Open Source model; and in open source we do
not like to reinvent the wheel. So, how to eliminate the waste of
resources. Mozilla planned to merge bn-BD and bn-IN locale for SuMo.

Please, don't confuse with Mozilla's perception; Mozilla is not an
authority to judge similarities and differences in two different locales.
It just found the final produced articles have too little differences. This
happened because of the nature of SuMo KB articles written in English,
considering a generic form for around the world.

So, Mozilla proposed us to merge bn-BD and bn-IN SuMo l10n initiatives as a
pilot. After reviewing the results the efforts may expand to other projects
and pathways of Mozilla; where this a localization involved.

Now, this is in a public mailing list for all of your comments and
feedback. Let us discuss.



Regards
Ashickur Rahman
Mozilla Bangladesh - SuMo Community lead

[Swarnava Sengupta]

cc'ing Sayak, Biraj & Sankarshan for more input on this.

> _______________________________________________
> Sumo mailing list
> Su...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/sumo
>

[Sankarshan Mukhopadhyay]

In order to understand this situation would be relevant to have the
following available:

[+] the full and complete findings from Mozilla around the assessment
[+] detail of the methodology adopted for the assessment including the
individuals undertaking the assessment, the sample set of content, the
duration of the assessment
[+] detail of the formal 'plan' from Mozilla around this topic
including timelines and community formulation
[+] detail of whether such assessment has been previously or,
hereafter conducted for any other pair of locales
[+] summary of such other assessments if conducted, for other pairs of locales
[+] a complete and clear definition of the term 'merge' including
scope and retrospective effect if any

>> Please, don't confuse with Mozilla's perception; Mozilla is not an
>> authority to judge similarities and differences in two different locales.
>> It just found the final produced articles have too little differences.
>> This
>> happened because of the nature of SuMo KB articles written in English,
>> considering a generic form for around the world.
>>
>> So, Mozilla proposed us to merge bn-BD and bn-IN SuMo l10n initiatives as
>> a
>> pilot. After reviewing the results the efforts may expand to other
>> projects
>> and pathways of Mozilla; where this a localization involved.
>>
>> Now, this is in a public mailing list for all of your comments and
>> feedback. Let us discuss.
>>
>>
>>
>> Regards
>> Ashickur Rahman
>> Mozilla Bangladesh - SuMo Community lead
>> _______________________________________________
>> Sumo mailing list
>> Su...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/sumo

--
sankarshan mukhopadhyay
<https://about.me/sankarshan.mukhopadhyay>

[Michal Dziewonski]

Hello Everyone,

I am not a speaker of either dialect of Bengali, therefore I trust the
judgement of the native speakers we have the privilege to work with - you.

I have learned about the idea to merge the two locales at the end of 2014.
I know that both communities have been discussing this on many levels, with
many arguments for and against.

I do not have access or am not aware of any of the data points mentioned by
Sankarshan, since I was not involved in the earlier discussions - I started
working on SUMO's KB only in September 2014. What I do have is native
speakers in both locales agreeing that the differences between the locales
are small enough to enable the users to benefit from existing localization
for either of the locales.

I also have this:

- https://support.mozilla.org/bn-IN/localization
- https://support.mozilla.org/bn-BD/localization

As you can see, from my perspective it makes sense to perform the merge as
soon as possible.

Now, what do I mean by "merge"?

Given the recent work <https://bugzilla.mozilla.org/show_bug.cgi?id=800880>
of one of our star contributors (Safwan), SUMO should soon have the option
to be set up with "fallback" locales on article level.

What this would mean in practical terms for the users is as follows:

A bn-IN speaker visits SUMO, looking for a helpful article. If the bn-IN
page exists, the user is shown that version. If it does not, and as a
result the user can't find a localized help page in their language, the
fallback system redirects that user to the corresponding bn-BD article. If
there is no bn-BD version of the article, the user is shown the page in
English.

Would that make sense to the communities involved in this discussion?
However strongly you feel for or against such a solution, please remember
that we are focusing on the user here, and my mission in SUMO is to make
sure that our users find useful help in a language they can understand. For
the evaluation of that, I rely on your native knowledge of the situation
and the context.

I'm eager to hear your take on this.

Cheers,

Michał
SUMO Community
sumo <https://support.mozilla.org/en-US/user/vesper>|twitter
<https://twitter.com/vesper_mozilla>|facebook
<https://facebook.com/michal.mozilla>|IRC: *vesper *(#sumo
<https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo>
#sumo-l10ns
<https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo-l10ns>
)|email policy
<http://qz.com/359726/email-lets-other-people-schedule-your-life-heres-how-to-stop-it/>
*Please take a few minutes to provide feedback for me*
<https://docs.google.com/a/mozilla.com/forms/d/1dDM1V1GjrTsMvCeWDvFzNmLSSpNK8_3DsF9xiB_vyRE/viewform>
- I really appreciate that!

2015-06-17 3:05 GMT+02:00 Sankarshan Mukhopadhyay <
sankarshan....@gmail.com>:

[Sankarshan Mukhopadhyay]

On Thu, Jun 18, 2015 at 3:11 PM, Michal Dziewonski
<mdzie...@mozilla.com> wrote:
> What I do have is native speakers in both locales agreeing that the
> differences between the locales are small enough to enable the users to
> benefit from existing localization for either of the locales.

Michal, thank you for taking time to respond. I am hoping that
individuals/staff members/Mozilla associates before you would be able
to respond to some, if not all the points I have sought detail about.

Meanwhile, would it be possible for you to point me to specific
instances where native speakers have publicly and on record identified
these small differences as being not an impediment to allow users to
benefit from either locale?

[Michal Dziewonski]

Sankarshan,

Here's a quote from Jeff's email summarising the discussion in Portland. I
am also CCing Jeff to make sure he's away of the current status of the
pilot.












*"Hi all, I wanted to follow up from previous conversations with an update
on our Bengali l10n community. We had a good discussion in Portland between
Biraj, Soumya Chakerboty, and Mak. The discussion was framed around
identifying the needs of the Bengali-speaking user in the region. We were
able to maintain that focus to an extent, but inter-community conflict
issues also became a focal point in the discussion. It was clear that the
bn-BD members were in favor of merging efforts and saw the advantages for
the user. The locale-specific conventions (currency, time and date format,
etc.) could be automated in order to provide the best user experience for
the user. The bn-IN members were not in favor of merging efforts. They saw
a merger as throwing away a lot of their good work and as an unjustified
action, as they have performed excellently as their own community. They
also cited come conflicts they've had with members of the bn-BD community.
Eventually, the conversation moved to directing attention at the bn-IN
members to address why merging didn't make sense to them. Had the
discussion continued, it would have become more accusatory, defensive, and
unproductive, so we put a pin in it and decided to revisit the discussion
at a later date. After the discussion (and Michel & Arky can add more
details here), Mak and Biraj were able to come to an agreement to use SUMO
as a pilot for merging efforts. It seems both communities struggle to
adequately cover SUMO article translation. I'm unaware of the
specifications of this pilot. Axel, Arky, and I are traveling to India in
February and plan to get Biraj and Mak together again to continue the
discussion there. If anyone has more details to offer, please feel free to
do so here. We're also happy to answer any questions about the discussion.
Thanks, Jeff"*
To the best of my knowledge, there are no other notes from that meeting
available.

Cheers,

Michał
SUMO Community
sumo <https://support.mozilla.org/en-US/user/vesper>|twitter
<https://twitter.com/vesper_mozilla>|facebook
<https://facebook.com/michal.mozilla>|IRC: *vesper *(#sumo
<https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo>
#sumo-l10ns
<https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo-l10ns>
)|email policy
<http://qz.com/359726/email-lets-other-people-schedule-your-life-heres-how-to-stop-it/>
*Please take a few minutes to provide feedback for me*
<https://docs.google.com/a/mozilla.com/forms/d/1dDM1V1GjrTsMvCeWDvFzNmLSSpNK8_3DsF9xiB_vyRE/viewform>
- I really appreciate that!

2015-06-18 12:03 GMT+02:00 Sankarshan Mukhopadhyay <
sankarshan....@gmail.com>:

[Sankarshan Mukhopadhyay]

On Thu, Jun 18, 2015 at 3:48 PM, Michal Dziewonski
<mdzie...@mozilla.com> wrote:
> After the discussion (and Michel & Arky can add more details here), Mak and
> Biraj were able to come to an agreement to use SUMO as a pilot for merging
> efforts. It seems both communities struggle to adequately cover SUMO article
> translation. I'm unaware of the specifications of this pilot. Axel, Arky,
> and I are traveling to India in February and plan to get Biraj and Mak
> together again to continue the discussion there.

Was there a public announcement of this plan-of-record?

[Jeff Beatty]

To add to this, I, personally, haven't seen any discussion about this
after the Portland meeting. We were unable to bring Mak to India for the
meetup I mentioned in that earlier email, so we were unable to continue
that discussion.

From the discussions I've had, I know of no recorded statements which
would satisfy your request, as these discussions (at least the one's
that I have been a part of) have all taken place in person. If this is
still a pilot project that the two Bengali efforts are interested in
pursuing, it is up to them to come together with Michal to determine how
to proceed. Considering the volume of work, I still believe a combined
effort for Bengali SUMO is better for users than a separate Bengali SUMO.

Jeff

El 6/18/15 a las 4:18 AM, Michal Dziewonski escibió:

> Sankarshan,
>
> Here's a quote from Jeff's email summarising the discussion in
> Portland. I am also CCing Jeff to make sure he's away of the current
> status of the pilot.
>

> /"Hi all,

>
> I wanted to follow up from previous conversations with an update on
> our Bengali l10n community. We had a good discussion in Portland
> between Biraj, Soumya Chakerboty, and Mak. The discussion was framed
> around identifying the needs of the Bengali-speaking user in the
> region. We were able to maintain that focus to an extent, but
> inter-community conflict issues also became a focal point in the
> discussion. It was clear that the bn-BD members were in favor of
> merging efforts and saw the advantages for the user. The
> locale-specific conventions (currency, time and date format, etc.)
> could be automated in order to provide the best user experience for
> the user. The bn-IN members were not in favor of merging efforts. They
> saw a merger as throwing away a lot of their good work and as an
> unjustified action, as they have performed excellently as their own
> community. They also cited come conflicts they've had with members of
> the bn-BD community. Eventually, the conversation moved to directing
> attention at the bn-IN members to address why merging didn't make
> sense to them. Had the discussion continued, it would have become more
> accusatory, defensive, and unproductive, so we put a pin in it and
> decided to revisit the discussion at a later date.
>

> After the discussion (and Michel & Arky can add more details here),
> Mak and Biraj were able to come to an agreement to use SUMO as a pilot
> for merging efforts. It seems both communities struggle to adequately
> cover SUMO article translation. I'm unaware of the specifications of
> this pilot. Axel, Arky, and I are traveling to India in February and
> plan to get Biraj and Mak together again to continue the discussion there.
>

> If anyone has more details to offer, please feel free to do so here.
> We're also happy to answer any questions about the discussion.
>
> Thanks,
> Jeff"
>

> /

> To the best of my knowledge, there are no other notes from that
> meeting available.

> /
> /

> Cheers,
>
> Michał
> SUMO Community
> sumo <https://support.mozilla.org/en-US/user/vesper>|twitter
> <https://twitter.com/vesper_mozilla>|facebook
> <https://facebook.com/michal.mozilla>|IRC: *vesper *(#sumo
> <https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo>
> #sumo-l10ns
> <https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo-l10ns>)|email
> policy
> <http://qz.com/359726/email-lets-other-people-schedule-your-life-heres-how-to-stop-it/>
> *Please take a few minutes to provide feedback for me*
> <https://docs.google.com/a/mozilla.com/forms/d/1dDM1V1GjrTsMvCeWDvFzNmLSSpNK8_3DsF9xiB_vyRE/viewform>
> - I really appreciate that!
>
> 2015-06-18 12:03 GMT+02:00 Sankarshan Mukhopadhyay
> <sankarshan....@gmail.com

> <mailto:sankarshan....@gmail.com>>:

>
> On Thu, Jun 18, 2015 at 3:11 PM, Michal Dziewonski
> <mdzie...@mozilla.com <mailto:mdzie...@mozilla.com>> wrote:
> > What I do have is native speakers in both locales agreeing that the
> > differences between the locales are small enough to enable the
> users to
> > benefit from existing localization for either of the locales.
>
> Michal, thank you for taking time to respond. I am hoping that
> individuals/staff members/Mozilla associates before you would be able
> to respond to some, if not all the points I have sought detail about.
>
> Meanwhile, would it be possible for you to point me to specific
> instances where native speakers have publicly and on record identified
> these small differences as being not an impediment to allow users to
> benefit from either locale?
>
>

> --
> sankarshan mukhopadhyay
> <https://about.me/sankarshan.mukhopadhyay>

> _______________________________________________
> Sumo mailing list
> Su...@lists.mozilla.org <mailto:Su...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/sumo
>
>

--
*Jeff Beatty*
Localization Engineer
@mozilla_l10n <http://twitter.com/mozilla_l10n>
801.367.3763

[Sankarshan Mukhopadhyay]

On Thu, Jun 18, 2015 at 7:39 PM, Jeff Beatty <jbe...@mozilla.com> wrote:
> Considering the volume of work, I still believe a combined effort for
> Bengali SUMO is better for users than a separate Bengali SUMO.

I see that you use 'Bengali'. Is that a conflation of the locales on
your part or, is that the motivation for this 'pilot'?

[Sankarshan Mukhopadhyay]

On Thu, Jun 18, 2015 at 3:11 PM, Michal Dziewonski

<mdzie...@mozilla.com> wrote:
> I also have this:
>
> https://support.mozilla.org/bn-IN/localization

> https://support.mozilla.org/bn-BD/localization
>
> As you can see, from my perspective it makes sense to perform the merge as
> soon as possible.

Michal, what you provide in those two URLs constitute a dashboard. The
short version of what I see is that the group of individuals who
contribute to SuMo from India don't seem to have enough contributions
coming in for the bn-IN locale.

In other words, the bn-IN community has debilitated. The proposal I
see being discussed indicates that there is a general agreement (based
on data/study yet to be made available to a larger group) that this
lack of content in bn-IN can be easily replaced by
replacement/redirecting to content available in bn-BD.

This is an interesting scenario. It implies that the Mozilla community
and the staff members are putting in a precedent that content in two
locales are equivalent and replaceable based on unknown metrics. What
long term effect could this bring about?

Here's a small set of conjectures.

[+] bn-IN localization community fails to meet the string translation
release requirements for a specific release. Based on the precedent
set above, it is possible to consider a drop-down replacement and
shipping of bn-BD content for the application strings

[+] any other 'pair of locales which look similar' have this issue,
let's say ta and ta-LK, the precedent allows a similar set of
adjustments to be brought about

[+] with Mozilla setting a precedent as above, it would be a well
established guidance for other Free and Open Source Software projects
to consider likewise. For example, GNOME can now decide that it would
be appropriate to subsume bn-IN locale content with the bn one (if I
recall this correctly, Mozilla is the only upstream project with a
bn-BD and bn-IN. The pair is bn-IN and bn for everything else. One can
trivially look up the Mozilla bug related to the creation of these
locales for Mozilla and figure out why)

I am happy that you see the concern. Unfortunately, it would appear
that you are unable to differentiate between investing in a community
when it is sick and shipping a shiny product. The current set of
discussions way back in January didn't have 'experts' in locales or,
language technologies. If we really do want to seek them, then there
is a different and better way of doing this than sudden announcements
on lists.

In short, should this 'pilot' be considered and implemented, the
digital existence of bn-IN locale would be decimated. And this is not
hyperbole or, exaggeration of facts. It is how the long game will
eventually play out across the project communities.

I would be happy to seed content from en (en_US?) to bn-IN by means of
machine translation (<https://wiki.mozilla.org/Intellego> ?) rather
than seeing short term gains make bn-IN a lost community.

[Biraj Karmakar]

Hi all,

Sorry for late reply.


On Thu, Jun 18, 2015 at 3:48 PM, Michal Dziewonski <mdzie...@mozilla.com>
wrote:

>
>

> The bn-IN members were not in favor of merging efforts.

Yes, we are not favor of this merging efforts.

> After the discussion (and Michel & Arky can add more
> details here), Mak and Biraj were able to come to an agreement to use SUMO
> as a pilot for merging efforts.

As far as I remember , at portland meeting, I told infront of everyone that
I could not take any decision on SUMO pilot for merging efforts, because I
am not SUMO lead from bn-IN. Also, I told please start a open discussion on
that so every community member can put their thoughts. SUMO leader is
@swarnava .

Axel, Arky, and I are traveling to India in
> February and plan to get Biraj and Mak together again to continue the
> discussion there. If anyone has more details to offer, please feel free to
> do so here. We're also happy to answer any questions about the discussion.

> Thanks, Jeff"*

> To the best of my knowledge, there are no other notes from that meeting
> available.
>
>

Yes in bangalore, we had not any meeting on this topic. There is no updates
after Portland meeting.

Thanks,

Biraj

> Cheers,
>
> Michał
> SUMO Community
> sumo <https://support.mozilla.org/en-US/user/vesper>|twitter
> <https://twitter.com/vesper_mozilla>|facebook
> <https://facebook.com/michal.mozilla>|IRC: *vesper *(#sumo
> <https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo>
> #sumo-l10ns
> <
> https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23sumo-l10ns
> >
> )|email policy
> <
> http://qz.com/359726/email-lets-other-people-schedule-your-life-heres-how-to-stop-it/
> >
> *Please take a few minutes to provide feedback for me*
> <
> https://docs.google.com/a/mozilla.com/forms/d/1dDM1V1GjrTsMvCeWDvFzNmLSSpNK8_3DsF9xiB_vyRE/viewform
> >
> - I really appreciate that!
>
> 2015-06-18 12:03 GMT+02:00 Sankarshan Mukhopadhyay <

> sankarshan....@gmail.com>:

>
> > On Thu, Jun 18, 2015 at 3:11 PM, Michal Dziewonski
> > <mdzie...@mozilla.com> wrote:

> > > What I do have is native speakers in both locales agreeing that the
> > > differences between the locales are small enough to enable the users to
> > > benefit from existing localization for either of the locales.
> >
> > Michal, thank you for taking time to respond. I am hoping that
> > individuals/staff members/Mozilla associates before you would be able
> > to respond to some, if not all the points I have sought detail about.
> >
> > Meanwhile, would it be possible for you to point me to specific
> > instances where native speakers have publicly and on record identified
> > these small differences as being not an impediment to allow users to
> > benefit from either locale?
> >
> >

> > --
> > sankarshan mukhopadhyay
> > <https://about.me/sankarshan.mukhopadhyay>

> > _______________________________________________
> > Sumo mailing list
> > Su...@lists.mozilla.org

> > https://lists.mozilla.org/listinfo/sumo

> >
> _______________________________________________
> Sumo mailing list
> Su...@lists.mozilla.org

> https://lists.mozilla.org/listinfo/sumo
>

[rab...@gmail.com]

Hi Michal,

I just went through the bug you specified (80088) and actually something more worrying caught my eye.
I can see from this comment (https://bugzilla.mozilla.org/show_bug.cgi?id=800880#c21) about the mapping that how the fallback will work is based on a mapping(?!?).

And about that mapping (and I quote here) I see comments such as "But As I can say, hi-IN is the first official Language of India. As its used in all the official Work in India, the best fallback local of gu-IN would must be hi-IN. As they belong from Same country and hi-IN is the official Language which must be used all over the country in India for Official work. So it must be better to show hi-IN version rather than en-US version if we dont have any content available in gu-IN"

I have the same concern as Lodolo. Was the fallback mapping discussed with any localizers before? Do they even agree with the mapping?
Were there any studies to see if they even like that mapping being forced into some other locale language and not English?

Just from the statement above I guess in that case all indian regional languages (apart from bn_IN) will fall back to hi-IN? Is that acceptable for those locale stakeholders/speakers/users?

I hope I am not digressing, but this also aligns to one of the points Sankarshan has raised.

Rabimba