FTL Proposal: Dissolve traits

[zbran...@mozilla.com]

In the thread about bi-annual syntax review, I listed a number of potential problems with the syntax.

In this proposal, I'd like to tackle mainly problem (3) and (4).

====================================

Summary:

I believe that there are exactly three linguistic features currently represented by traits. Each one of them has a very different role and clustering them with syntax is confusing.

Additionally, their syntax is overlapping with the syntax of variants which adds to the confusion.

====================================

Proposal:

Each of the three features could be served better if we aimed at identifying how we should represent them in the syntax to aid readers with their function:

1) Multi-variant value
====================================


Since we are already talking about variants of the value, we should merge this into selectors.
That would require two changes to our syntax:

a) We would need to allow for selector expressions without selector.

brandName = {
[nominative] Firefox
[possesive] Firefoksa
}

It better represents what the content of this data is, and it reduces the number of concepts in FTL.

b) We would want to add a specifier

But the reason we originally introduced traits was to allow for referencing a single variant of the value. If we merge those kind of traits into variants, currently there would be no way to reference a nominative variant of the value.

For that reason, I suggest that we repurpose trait reference to a "specifier" which informs which variant of the selector we want to use:

key = Welcome to { brandName[nominative] }

This syntax is intuitive and serves the function.

The question is how to solve edge cases, like handling multiple selectors in a single value and specifying for that.
I believe that those are edge cases, and our design principle has been to make common cases easy to read and edge cases possible, so I would prefer not to sacrifice a clear solution because of an extreme edge case.

But to satisfy the need to make sure that we can handle more complex solutions, I believe that in the future we could allow for more than one specifier for both nested selectors and chained selectors. It could look like this:

key1 = {
[one] One
[other] Other
} and {
[one] One
[other] Other
}

key2 = { key1[one][other] }

or:

key1 = {
[one] One
[other] {
[masculine] Man
[feminine] Woman
}
}

key2 = {key1[other, masculine] }

Both [X, Y] and [X][Y] approaches can be applied to both nested and chained selectors, but I believe those to be so rare and esoteric, that I don't think we need to support them now and it's enough to know that we do have a way forward if we find the need.

For now, I believe that we should focus on the common use case, which is solved with this change.

2) Multi-value entity
====================================


This is a completely different use of traits and as such, I'd suggest we reflect that in our syntax. Because in all cases we encountered, there is at most a single "value" and multiple "attributes", I believe it's worth treating it as such.
A syntax that is popular for referencing attributes is a "." dot, so a proposed syntax might look like this:


key = Value
key.label = Label
key.accesskey = C

Additional benefit of this syntax is that if the value is not needed, it can be omitted:

file-open.label = Open File
file-open.accesskey = O

I do not suggest separating attribute into their own entities. I believe that parser should combine value and attributes into a single entity.

Counterarguments:

a) This creates a risk of a single entity sprinkled around a long FTL file or even multiple files.

I recognize this as a non-goal, and suggest that we enforce attributes to be only allowed to be listed directly under value.
If the value is not present, all attributes of the same entity must be next to each other.

Any other combination would result in a parser error.

b) Redundancy in the entity name repeated for each attribute

I find this suboptimal, but an acceptable cost of the change.

Alternative proposals:

We could try to omit the redundancy by writing something like:

file-open
.label = File Open
.accesskey = O

3) Meta-information
====================================

So far, we have not encountered a single other use case of the meta-information beyond the gender of the brandName.

Although it's an important use case for many languages to be able to construct messages with the brandname, the lack of in-the-wild cases that we would solve with meta-information makes me believe that we should not optimize our syntax for them.

There are multiple proposals on how to encode it. The dominant is to add tags to the entity. Tags are a common concept in UX outside of software engineering.

It could work like this:

#masculine
brandName = Firefox

key1 = { META(brandName) ->
[masculine] On { brandName }
[feminine] Ona { brandName }
}

In the future we could try to add syntactic sugar to make referencing the meta information easier, but initially, that should be readable enough and solve the one case we have while allowing us to believe that it should work for other cases if we find them.

Thoughts?
zb.

[zbran...@mozilla.com]

For the reference, this is the discussion etherpad that served as a base for this proposal: https://public.etherpad-mozilla.org/p/ftl-syntax-traits

zb.

[Staś Małolepszy]

FWIW, I agree with the analysis of the problem and I like all of the
suggested solutions. (Maybe because I participated in the discussions
about these last week :).

On Fri, Dec 16, 2016 at 11:00 AM, <zbran...@mozilla.com> wrote:

> 1) Multi-variant value
> ====================================

> brandName = {
> [nominative] Firefox
> [possesive] Firefoksa
> }

This would go well with the proposal to enforce a default to be
present in a list of variants. In this example, we'd write
*[nominative].

> key = Welcome to { brandName[nominative] }
>
> This syntax is intuitive and serves the function.
>
> The question is how to solve edge cases, like handling multiple selectors in a single value and specifying for that.

I realized that accessors are very hard to design in the current
syntax. Select expressions can appear in Patterns which are elements
of other Patterns, e.g. " Foo { "{ foo -> ... }" }." This is a
different kind of nesting than the one you present in your examples.
In order to make the semantics complete, we'd need to spec out very
complex accessors, like [foo, [bar]][baz]. I don't want to do that --
not because I'm lazy but because I doubt it's useful at all to
localizations.

What if we limit the foo[bar] accessor syntax to values which are a
Pattern with a single select expression which must not have a
selector? Otherwise [bar] is ignored and normal semantics of resolving
the Pattern apply.

> 2) Multi-value entity
> ====================================

> I find this suboptimal, but an acceptable cost of the change.

It also helps readability, so I'm okay with it. The following reads
clear to me:

file-menu.label = File
file-menu.accesskey = F

While the current equivalent requires more visual parsing:

file-menu =
[xul/label] File
[xul/accesskey] F


> 3) Meta-information
> ====================================

> #masculine
> brandName = Firefox

We could also allow Mozilla-specific tags in the future, like #private
for message which shouldn't be checked by compare-locales.

* * *

We'll need to choose names for these new constructs and I'm lost in
looking for good examples to follow. For #2, HTML uses "attributes"
and React uses "props". For #3, Rust uses "attributes" and Twitter
would call them "hashtags". We also still have "traits" :) Do you
have any naming scheme in mind?

-stas

[Axel Hecht]

Am 16/12/16 um 11:00 schrieb zbran...@mozilla.com:

> In the thread about bi-annual syntax review, I listed a number of potential problems with the syntax.
>
> In this proposal, I'd like to tackle mainly problem (3) and (4).
>
> ====================================
>
> Summary:
>
> I believe that there are exactly three linguistic features currently represented by traits. Each one of them has a very different role and clustering them with syntax is confusing.
>
> Additionally, their syntax is overlapping with the syntax of variants which adds to the confusion.
>

I have comments on the actual proposals below, but I also want to
comment on the problem statement.

One problem statement is the difference between internal and external
traits.

I strongly believe that they should not be used together in practice. As
much as the sytanx allows that, semantically, the gender of a button
isn't the gender of the text in the placeholder attribute.

If there's text that gets re-used with different grammatical forms, or
that has properties, it should be in a standalone message. That message
should then be referenced by widget localizations.

This is something that we can enforce in linters and other tools.

> ====================================
>
> Proposal:
>
> Each of the three features could be served better if we aimed at identifying how we should represent them in the syntax to aid readers with their function:
>
> 1) Multi-variant value
> ====================================
>
>
> Since we are already talking about variants of the value, we should merge this into selectors.
> That would require two changes to our syntax:
>
> a) We would need to allow for selector expressions without selector.
>
> brandName = {
> [nominative] Firefox
> [possesive] Firefoksa
> }
>
> It better represents what the content of this data is, and it reduces the number of concepts in FTL.

I personally think that this makes things worse. Reminds me of the <> we
had in .lols. Everybody cheered when we got rid of those.

> b) We would want to add a specifier
>
> But the reason we originally introduced traits was to allow for referencing a single variant of the value. If we merge those kind of traits into variants, currently there would be no way to reference a nominative variant of the value.
>
> For that reason, I suggest that we repurpose trait reference to a "specifier" which informs which variant of the selector we want to use:
>
> key = Welcome to { brandName[nominative] }
>
> This syntax is intuitive and serves the function.
>
> The question is how to solve edge cases, like handling multiple selectors in a single value and specifying for that.
> I believe that those are edge cases, and our design principle has been to make common cases easy to read and edge cases possible, so I would prefer not to sacrifice a clear solution because of an extreme edge case.
>

You lost me here.

One of the big wins of FTL was that everything about the message was in
the value part of the syntax.

I'd not want to give this up for the multiple messages syntax.

b) is the same thing as traits right now, just that we re-introduce a
different syntax to distinguish private from public ones, right? And
dropping the namespaces.

Not sure if there's a practical difference.

> 3) Meta-information
> ====================================
>
> So far, we have not encountered a single other use case of the meta-information beyond the gender of the brandName.
>
> Although it's an important use case for many languages to be able to construct messages with the brandname, the lack of in-the-wild cases that we would solve with meta-information makes me believe that we should not optimize our syntax for them.
>
> There are multiple proposals on how to encode it. The dominant is to add tags to the entity. Tags are a common concept in UX outside of software engineering.
>
> It could work like this:
>
> #masculine
> brandName = Firefox
>
> key1 = { META(brandName) ->
> [masculine] On { brandName }
> [feminine] Ona { brandName }
> }

We struggled with those functions that don't actually return anything,
but that are just semantic sugar. As tags are multivalued, META had to
be one of those.

I'd rather not add more of them.

Also, as above, this moves content outside of the message value, which
causes grief on the tooling side.

I'm generally not too happy about the tagging piece, as I think it makes
it easier to get them wrong. Like, is "mazculine" a thing that's new or
a typo?

> In the future we could try to add syntactic sugar to make referencing the meta information easier, but initially, that should be readable enough and solve the one case we have while allowing us to believe that it should work for other cases if we find them.
>
> Thoughts?
> zb.
>

Axel

[zbran...@mozilla.com]

Hi Stas, Axel,

Thanks for the feedback!

Stas:

> What if we limit the foo[bar] accessor syntax to values which are a
Pattern with a single select expression which must not have a
selector? Otherwise [bar] is ignored and normal semantics of resolving
the Pattern apply.

Great point. I did not consider that.

I believe that your proposal is good. It's perfectly in line of solving the common case well, and leaving a more esoteric variant of the problem for later, as long as we agree that it seems like we'll be able to find a solution that won't require backward incompatible changes.

I think that all more complex specifiers/accessors can be added later if we find uses for them, and putting this limitation will not impair any use case I can think of right now while allowing us to simplify the syntax and make it more intuitive.

So, +1 :)

Axel:

> One problem statement is the difference between internal and external
traits.

Yes. Out of the three cases I listed, two (multi-variant values and meta-information) are internal, and one (multi-valued translation unit) is external.

> I strongly believe that they should not be used together in practice.

I agree. And the fact that the design of the syntax does not distinguish between them and gets us in a position that, in theory, a gender variant of the value will have the same syntax as a gender meta-information, has been bothering me ever since we came up with traits.

I believe that this proposal solves it by:

a) moving multi-variant values to use our variant selector syntax
b) renaming the external traits to what everyone in the world calls them - attributes

Traits in their original form fit only the meta-information piece, but because of their open, catch-all nature, we put the other two uses in them and then had to come up with namespaces to distinguish between them.

> This is something that we can enforce in linters and other tools.

Or via syntax. Which is what I'm proposing here.


> You lost me here.

Sorry, I was overly defensive. In short, I believe that we should focus on the most common case and make it be the most readable and easy to understand.

The most common use of multi-variant value in FTL is going to be a single variant list for the whole value, like:

brandName =
[nominative] Firefox
[possesive] Firefoksa

And I believe that there's no reason to keep it as separate syntax from our variant selector syntax because those are variants of the message.

My argument is that you will never want to have both, the value, and this case of traits, because those traits *are* the value.

So I suggest merging them back:

brandName = {
[nominative] Firefox
[possesive] Firefoksa
}

When we originally talked about this we discussed more complex cases and how to handle them in FTL, which is what I tried to address here, but those are edge cases and it's worth saying that I believe that we will be able to solve them without any breaking changes if we ever have to.

> I personally think that this makes things worse. Reminds me of the <> we
had in .lols. Everybody cheered when we got rid of those.

I don't think I understand how it reminds you of <>. This and your next comment:

> One of the big wins of FTL was that everything about the message was in
> the value part of the syntax.
>
> I'd not want to give this up for the multiple messages syntax.

makes me think that maybe I failed to communicate this part of the proposal well.

In the current syntax, the multi-varaint value is outside of the value itself.

In other words:

key = { [one] One [other] Other }

is a single value for ID `key`, with the following AST:

{
id: 'key',
value: [Placeable(SelectorExpression)]
}

but:

key =
[nominative] One
[possesive] Other

is not! We have a value here (which is empty) and two traits which are separate from the message. The AST is:

{
id: 'key',
value: null,
traits: [
{ id: 'nominative', value: 'One'},
{ id: 'possesive', value: 'Other'},
]
}


My proposal actually does exactly what you're advocating for - merges those traits back into the message value:

key = { [nominative] One [possesive] Other }

The only missing part is how to refer to a particular case, and I propose that we keep the same syntax: key[nominative] to resolve the first selector of the message using 'nominative' variant.
:stas suggests that for now, we limit the syntax to this, and leave the door open to explore supporting more complex accessors (for values with multiple selectors or nested selectors) in the future

> b) is the same thing as traits right now, just that we re-introduce a
different syntax to distinguish private from public ones, right? And
dropping the namespaces.

Yes! Once we merge (a) back into entity's value, we're left with two uses of traits: multi-value and meta-info. The former is external, the latter internal.
Both don't fit into entity's value variant-selector and shouldn't have a "default" option.
Since they serve very different purposes, I suggest reviewing them separately and tried to come up with how I envision they should look to a reader.
They also differ in severity. (b) is going to be a very common case for most UI toolkits, including Gecko itself and in the future web components. (c) so far seems to serve an important, but a very rare case.

For that reason I'd like to focus on (b) and answer a question:

How would we design a single entity with an optional main value and multiple named additional "values".
I believe that we would name those named values "attributes" and design something like I proposed:

key = Value
key.attribute1 = Value 2
key.attribute2 = Value 3

> We struggled with those functions that don't actually return anything,
but that are just semantic sugar.

I envision that more as a smart-function that iterates over the variants and selects the first that matches.

> Also, as above, this moves content outside of the message value, which
causes grief on the tooling side.

I disagree that it moves content outside of the message value.

The (a) proposal moves the content back into the value, quite in line with your sentiment
The (b) keeps the separate value but makes it more readable - we can't move this content back into the main value because it's whole goal is to be a separate value
And the (c) is, imho, not content, it's meta-information, and as such cannot be part of the value.

It can annotate a value, or an entity. During brainstorm stas asked if it's possible to design it so that it annotates a single value.
We can look for a solution.

> I'm generally not too happy about the tagging piece, as I think it makes
it easier to get them wrong. Like, is "mazculine" a thing that's new or
a typo?

I'm pretty open to other proposals. I just believe that it's distinct enough from the other two uses (a and b) that it should not share the syntax with them.
If we agree on the new syntax for the other two, we're left with meta-infromation use case and we can keep it as-is, but that doesn't address the problem nr. 2 - that the trait syntax looks like variant syntax.

Regarding your concerns about typo. I'm not sure how is it different from making the same typo here:

key =
[gender] mazculine

The tools can aid by raising a warning if you use tags that don't match anything else, or help you design a list of tags applicable to your language and auto-suggest them.

I'm also open to brainstorm different syntax for the meta-information case than tags. Out of the three I'm least opinionated about it and I think there's no "natural" syntax for it like I believe there is for (a) and (b).

It seems to me that your concerns are most prominent with regards to the (c). I'd be happy to keep brainstorming that and separate out the other two parts of the proposal.

zb.