Unresolved entities and attributes

[Axel Hecht]

While hacking on a compare-locales impl for l20n, we came across
unresolved entities and attributes.

That is hashes without index and/or default values, used for either the
value or an attribute.

Is there a usecase for that?

My personal take is that that should be a compiler error.

Most prominently, I think that hash keys are language-internal, and as
such shouldn't be exposed to the outer world.

If we had lists, I'd be inclined to let those cross the border back to
the program. Mostly because I can see them being used for configuration.
Though at that point one could possibly also just return a string with
tokens. Not sure.

Basically, I wonder if we have an actual use case for this.

And once you construct one, I might answer with a macro to do that.
Adding that 'cause of how I did my last experiment with referencing
internal manifolds.

Axel

[Staś Małolepszy]

On Thu, Sep 10, 2015 at 5:35 PM, Axel Hecht <l1...@mozilla.com> wrote:

> While hacking on a compare-locales impl for l20n, we came across
> unresolved entities and attributes.
>
> That is hashes without index and/or default values, used for either the
> value or an attribute.
>
> Is there a usecase for that?
>

One use-case is the one being currently discussed in the byteUnit thread.
In essence: how to store arbitrary data which can be re-used in
translations?

Regardless of the outcome of that discussion, perhaps we should consider
banning hashes without default values. This would definitely make for
safer translations and improve the quality of l20n code.

What do you think: should we make that a ParserError?

>
> My personal take is that that should be a compiler error.
>
> Most prominently, I think that hash keys are language-internal, and as
> such shouldn't be exposed to the outer world.
>

I agree on hash keys. There are other parts of the public interface of an
entity, however: value and public attributes. Is this also part of the
'resolvable' discussion? I think it would make sense to report missing
values and public attributes as compare-locales errors because they violate
the social contract and can break the expectations of the consumer code.
In compare-locales.js we wanted to have a special state for this:
malformed. What are your thoughts about this?

-stas

[Staś Małolepszy]

On Thu, Sep 10, 2015 at 5:35 PM, Axel Hecht <l1...@mozilla.com> wrote:

>

> And once you construct one, I might answer with a macro to do that. Adding
> that 'cause of how I did my last experiment with referencing internal
> manifolds.
>

I like the idea of using macros to store some language-specific arbitrary
data. As Axel pointed out to me in Vidyo, if you need a language-specific
data, you probably also need at least some logic to it.

I'm now thinking that all private information should be stored in macros,
and that all macros should be private.

-stas

[Axel Hecht]

On 9/11/15 5:55 PM, Staś Małolepszy wrote:
> On Thu, Sep 10, 2015 at 5:35 PM, Axel Hecht <l1...@mozilla.com> wrote:
>
>> While hacking on a compare-locales impl for l20n, we came across
>> unresolved entities and attributes.
>>
>> That is hashes without index and/or default values, used for either the
>> value or an attribute.
>>
>> Is there a usecase for that?
>>
>
> One use-case is the one being currently discussed in the byteUnit thread.
> In essence: how to store arbitrary data which can be re-used in
> translations?
>
> Regardless of the outcome of that discussion, perhaps we should consider
> banning hashes without default values. This would definitely make for
> safer translations and improve the quality of l20n code.
>
> What do you think: should we make that a ParserError?

Might make sense.

>>
>> My personal take is that that should be a compiler error.
>>
>> Most prominently, I think that hash keys are language-internal, and as
>> such shouldn't be exposed to the outer world.
>>
>
> I agree on hash keys. There are other parts of the public interface of an
> entity, however: value and public attributes. Is this also part of the
> 'resolvable' discussion? I think it would make sense to report missing
> values and public attributes as compare-locales errors because they violate
> the social contract and can break the expectations of the consumer code.
> In compare-locales.js we wanted to have a special state for this:
> malformed. What are your thoughts about this?

We're calling malformed an error/warning right now. Missing is an error,
obsolete is a warning.

I'm not sure that's ultimately the right answer, in particular in the
light of locales which want to re-use en-US accesskeys.

I'd like to make that easy.

Doesn't make l10n-merge easier, though :-)

I'm wondering, can we restrict ourselves in the model here to DOM nodes.

In which case an obsolete attribute might actually be an error in the
sense that the l10n-merge code should remove it, while a missing
attribute (or value) might be just a warning?

Axel

[zbran...@mozilla.com]

On Monday, September 14, 2015 at 6:37:06 AM UTC-7, Axel Hecht wrote:
> We're calling malformed an error/warning right now. Missing is an error,
> obsolete is a warning.
>
> I'm not sure that's ultimately the right answer, in particular in the
> light of locales which want to re-use en-US accesskeys.
>
> I'd like to make that easy.
>
> Doesn't make l10n-merge easier, though :-)
>
> I'm wondering, can we restrict ourselves in the model here to DOM nodes.
>
> In which case an obsolete attribute might actually be an error in the
> sense that the l10n-merge code should remove it,

Sounds good to me.

> while a missing attribute (or value) might be just a warning?

I have more trouble with that.

Missing attribute is not very different from missing a value and I'm concerned if we try to make it work.

Example:

A `placeholder` attribute of an entity is de-facto its value. Missing it should be an error and we should use fallback language entity then.

Accesskeys are interesting, in that I can see how you might want to skip them in your partial locale and reuse the 'masters' one, but in that case, I'd expect some sort of metainformation stating "missing accesskeys are ok, reuse the parent locale ones" whitelist.

zb.

[Axel Hecht]

The question boils down to:

Is there a DOM element where we wouldn't want to take the text content
and some attributes because one attribute is missing?
Is there a DOM element where we wouldn't want to take the ..... because
the text content is missing?

Do we need to sanitize obsolete attributes? Or is the code in
https://github.com/l20n/l20n.js/blob/master/src/bindings/html/overlay.js
strong enough?

Axel

[Zibi Braniecki]

On Tuesday, September 29, 2015 at 4:01:52 AM UTC-7, Axel Hecht wrote:
> The question boils down to:
>
> Is there a DOM element where we wouldn't want to take the text content
> and some attributes because one attribute is missing?

Technically no, morally yes. I believe that we should aim for element consistency, so if 'placeholder' or 'title' or 'ariaLabel' is missing, we should take the whole element from fallback.

> Is there a DOM element where we wouldn't want to take the ..... because
> the text content is missing?

Same as above. If textContent is missing from an element and it should be there, we want to take it from fallback. And I opt for consistency of an element.

> Do we need to sanitize obsolete attributes? Or is the code in
> https://github.com/l20n/l20n.js/blob/master/src/bindings/html/overlay.js
> strong enough?

Stas?
zb.

[Axel Hecht]

On 10/2/15 10:02 PM, Zibi Braniecki wrote:
> On Tuesday, September 29, 2015 at 4:01:52 AM UTC-7, Axel Hecht wrote:
>> The question boils down to:
>>
>> Is there a DOM element where we wouldn't want to take the text content
>> and some attributes because one attribute is missing?
> Technically no, morally yes. I believe that we should aim for element consistency, so if 'placeholder' or 'title' or 'ariaLabel' is missing, we should take the whole element from fallback.

My current thinking about localizers is that we'll lose moral debates.

I'd also place ariaLabel in the optional category, like accesskeys. Most
of our languages have only few users, and it's unlikely that we'll see
usage of a11y strings in them. I'd rather use telemetry to inform us on
which languages should invest in a11y. Yeah, chicken-and-egg, but still.

Axel

[Staś Małolepszy]

It looks like the main concern here is the idea of sparse localizations.
My overall vote goes towards maintaining consistency of entities, but I'd
like to suggest to split this topic into three categories:

1. 'standalone' attributes, like title, which provide a separate piece of
content that's somehow related to the value of the element,
2. 'support' attributes, like accesskey, which are strictly related to the
value of the element, and
3. 'dominant' attributes, like ariaLabel or placeholder, where the
attribute is really _the_ value of the element.

I'll start in reversed order:

3. a missing dominant attribute means that the whole entity is no good;
it's unlikely that the entity even has a value, so the consistency rule
gives good results here.

2. a missing support attribute may result in weird UI like we see it in
Firefox when the accesskey doesn't match any of the letters of the string;
for sparse localizations, erring on the side of consistency/responsibility
seems to me like the right choice: if you change the string, you also need
to copy and change the attribute. A missing support attribute is thus an
error, too.

(In particular for accesskeys we could do semi-smart things on buildtime if
we knew the next significant language the sparse localization is based on,
like falling back on the next language and errorring out only if the letter
is not present in the translation string.)

1. for standalone attributes, I think it would make the most sense to
consider per-attribute fallback. However, for conistsency with the
behavior in #2 and #3, I'd suggest treating them as errors, too. I think
this follows the principle of least surprise.

* * *

Coming back to compare-locales, I think we should do the following:

- error on obsolete value/attributes,
- error on missing value/attributes,
- remove the is-resolvable check for hashes,
- parser-error on hashes without a default value.

Specifically, obsolete values/attributes should be errors not warnings
because they can leak into the DOM where they shouldn't and can break the
UI.

-stas

[Axel Hecht]

On 10/22/15 6:06 PM, Staś Małolepszy wrote:
> It looks like the main concern here is the idea of sparse localizations.
> My overall vote goes towards maintaining consistency of entities, but I'd
> like to suggest to split this topic into three categories:
>
> 1. 'standalone' attributes, like title, which provide a separate piece of
> content that's somehow related to the value of the element,
> 2. 'support' attributes, like accesskey, which are strictly related to the
> value of the element, and
> 3. 'dominant' attributes, like ariaLabel or placeholder, where the
> attribute is really _the_ value of the element.
>
> I'll start in reversed order:
>
> 3. a missing dominant attribute means that the whole entity is no good;
> it's unlikely that the entity even has a value, so the consistency rule
> gives good results here.

I also think that in this context, this scenario doesn't pose any
challenge? We'll be falling back to whatever is in the fallback chain,
independent of what we discuss here?

> 2. a missing support attribute may result in weird UI like we see it in
> Firefox when the accesskey doesn't match any of the letters of the string;
> for sparse localizations, erring on the side of consistency/responsibility
> seems to me like the right choice: if you change the string, you also need
> to copy and change the attribute. A missing support attribute is thus an
> error, too.

I don't think I agree.

As much as I agree on the general attitude that we want complete
localizations, we just don't have them.

Maybe my thinking can be reworded better:

Is there a element/attribute(s) combination, where you can't test one
without the other, even for a local build, or a runtime fallback combo?
Is that case common enough to dominate our thinking?

That to me is hard to say "yes" to.

> (In particular for accesskeys we could do semi-smart things on buildtime if
> we knew the next significant language the sparse localization is based on,
> like falling back on the next language and errorring out only if the letter
> is not present in the translation string.)
>
> 1. for standalone attributes, I think it would make the most sense to
> consider per-attribute fallback. However, for conistsency with the
> behavior in #2 and #3, I'd suggest treating them as errors, too. I think
> this follows the principle of least surprise.

As you're talking about consistency here, I think the point in 2. needs
to be figured out first.

Axel