l10n testing for v1.3 starts Feb 3rd

[delphine]

Pike's already mentioned this recently: we finally have dates for Firefox OS l10n testing for v1.3:

* First round of testing: Feb. 3 - Feb 7
* Second round of testing: Feb. 20 - Feb 26

As you can see, first round starts in 10 days, so please catch up with any l10n work that needs to be ready by then

thanks :)

[Iacopo Benesperi]

delphine ha scritto:

Is there some specific routine that I have to go through to help testing
the localisation? Or can I just continue to use the phone regularly?

thanks,
Iacopo

[delphine]

Hi Iacopo

Thanks for asking :) Yes there is. We're working on it now as a matter of fact.
I will make the test cases public once they are updated for v1.3
That way everyone will be able to help out!

thanks
delphine

[delphine]

Here is the spreadsheet with the test cases for v1.3:
https://docs.google.com/spreadsheet/ccc?key=0AvRct4nrPb_ldHpIZnVneHlHVGhpWUdSN3RLa2dubEE&usp=sharing#gid=1
(please note that there are multiple tabs at the bottom of the spreadsheet)

Testing starts Monday. The spreadsheet is in read-only mode, so if you want to help out with it, just shoot me an email and I'll grant you edit access (Iacopo: I'm adding you now)

Just remember to enter your name in the column of what you're testing, so we know who did what. Also, remember to test using the latest v1.3 build available.

Thanks in advance to all those who will contribute to this ;)
delphine

[Iacopo Benesperi]

delphine ha scritto:

> Testing starts Monday. The spreadsheet is in read-only mode, so if you want to help out with it, just shoot me an email and I'll grant you edit access (Iacopo: I'm adding you now)

Thank you!
Can you just please explain a bit how that spreadsheet works and what am
I supposed to write on it?

thanks,
Iacopo

[delphine]

Of course! I'm currently writing up all you need to know and putting it up on a wiki page this weekend (ie in time for Monday)

[delphine]

Here is the link to the etherpad explaining the l10n testing procedure for Firefox OS v1.3: https://etherpad.mozilla.org/l10n-fxos-testing-procedure

It's still a work in progress, but most of what you need to know is already there. I'll put it up somewhere on a wiki once it's definite.

In the meantime, if anything's unclear or if you have any questions, please reach out to me. I'll improve the pad as we go along.

Thanks
delphine

[Iacopo Benesperi]

delphine ha scritto:

> Here is the spreadsheet with the test cases for v1.3:
> https://docs.google.com/spreadsheet/ccc?key=0AvRct4nrPb_ldHpIZnVneHlHVGhpWUdSN3RLa2dubEE&usp=sharing#gid=1
> (please note that there are multiple tabs at the bottom of the spreadsheet)
>
> Testing starts Monday. The spreadsheet is in read-only mode, so if you want to help out with it, just shoot me an email and I'll grant you edit access (Iacopo: I'm adding you now)

Since this first round of testing is finished - some afterthoughts: how
are these tests conducted? For Italian, I've noticed that many tests
have been carried out by Mozilla employees and definitely not members of
the Italian community (or Italian speakers).
Provided that I'm just not able to perform all the tests myself (maybe
the Italian community as a whole can run all the tests though), I'm
asking: how can a non-Italian speaking person assess the test and mark
it as pass? How can he/she find l10n errors, typos and so on if he/she
can't speak Italian? Therefore: how much we can rely on these tests?

Iacopo

[Francesco Lodolo [:flod]]

Il 09/02/14 08:36, Iacopo Benesperi ha scritto:

> Provided that I'm just not able to perform all the tests myself (maybe
> the Italian community as a whole can run all the tests though), I'm
> asking: how can a non-Italian speaking person assess the test and mark
> it as pass? How can he/she find l10n errors, typos and so on if he/she
> can't speak Italian? Therefore: how much we can rely on these tests?

These tests cover two important aspects: string truncation and missing
(untranslated) strings, the latter sometimes due to localizability issues.

A person who doesn't speak the language can't obviously assess the
localization quality and correctness. For this same reason sometimes you
have false positives: for example I received a bug for a truncated
string because I localize "Connetting" as "Connessione…".

Francesco

[Iacopo Benesperi]

Francesco Lodolo [:flod] ha scritto:

> These tests cover two important aspects: string truncation and missing
> (untranslated) strings, the latter sometimes due to localizability issues.

Yes, I can see how having someone (even non-native language speaker)
check all the screens is important, but then why don't we use three
grades for the test, like "fail, looks fine, pass" with three colours
red, orange and green so that non-native speaker can still give good
information by saying looks fine, but native speaker can do some deeper
test by also looking at the actual language?
If I see a green cell I tend to go past it, while if I see an orange one
then I may want to check if the language is actually ok, giving the test
a better quality. On the other hand, test-wise, orange can be considered
good enough for test passing if noone can go a step further.
What do you think about this proposal?

Iacopo

[Francesco Lodolo [:flod]]

Il 09/02/14 09:07, Iacopo Benesperi ha scritto:

> If I see a green cell I tend to go past it, while if I see an orange one
> then I may want to check if the language is actually ok, giving the test
> a better quality. On the other hand, test-wise, orange can be considered
> good enough for test passing if noone can go a step further.
> What do you think about this proposal?

It definitely makes sense to me (another pro is that you see how much
coverage is needed from l10n people), but let's see what Delphine thinks
about it, since she's managing most of QA :-)

Francesco

[delphine]

Thanks for that idea Iacopo, we could definitely work on integrating something of the sort for future l10n test runs.
However I think I might be missing something in your proposal: when would a non-native tester determine when to put orange rather than green in a cell? I'm not sure this works, because this would mean that the non-native would always have to put orange since he doesn't know the language... So you'd still have to go through all the tests to check that the language is ok

delphine

[Francesco Lodolo [:flod]]

Il 10/02/14 12:56, delphine ha scritto:

> However I think I might be missing something in your proposal: when would a non-native tester determine when to put orange rather than green in a cell? I'm not sure this works, because this would mean that the non-native would always have to put orange since he doesn't know the language... So you'd still have to go through all the tests to check that the language is ok

As I understand this proposal (or maybe as I'd like it):

* A non native speaker could put an orange cell, to say that the
feature has passed a basic visual test (localized, not truncated).
* A native speaker could check it and replace the orange cell with a
green one. Passing a "full test" implies that the string has passed
also the basic test.

Orange: feature is localized, and not truncated.
Green: feature is localized, not truncated, and with a good localization.
Green >> Orange, no point in having a basic visual test on a green cell,
while it's worth having a native speaker testing an orange one for
language quality.

Francesco

[delphine]

Le lundi 10 février 2014 13:05:55 UTC+1, Francesco Lodolo [:flod] a écrit :
> Il 10/02/14 12:56, delphine ha scritto:
>

> As I understand this proposal (or maybe as I'd like it):
>
>
>
> * A non native speaker could put an orange cell, to say that the
>
> feature has passed a basic visual test (localized, not truncated).
>
> * A native speaker could check it and replace the orange cell with a
>
> green one. Passing a "full test" implies that the string has passed
>
> also the basic test.
>
>
>
> Orange: feature is localized, and not truncated.
>
> Green: feature is localized, not truncated, and with a good localization.
>
> Green >> Orange, no point in having a basic visual test on a green cell,
>
> while it's worth having a native speaker testing an orange one for
>
> language quality.
>
>
>
> Francesco

Ok this makes more sense to me. We've covered Italian during our 1st round of testing last week, so we can definitely do something like this going onwards in our 2nd round for other locales, and in future l10n test runs
thanks
delphine

[Axel Hecht]

I'd prefer a non-orange color, though. Orange in all of our automation
stands for "test failure".

Axel

[Iacopo Benesperi]

Axel Hecht ha scritto:

> I'd prefer a non-orange color, though. Orange in all of our automation
> stands for "test failure".

The colour is not really important, it was just to give an idea of the
thing. It can be light blue for example, or yellow.
Francesco summarised well the proposal. Again: we can consider orange
(blue, yellow, whatever) good enough to proceed, as we're doing now
after all, but this way we give a visual indication to a native speaker
that, if he/she has time, can further improve the QA by checking if the
text displayed is actually ok for that language.

Iacopo

[Axel Hecht]

I like blue, actually, as it's used for restarted builds on TBPL. Thus
its semantics for "let's do this once more with love" sounds about right.

Axel