RTL and bidi in Gaia, Gecko support, and planned CSS

[Julien Wajsberg]

Hey,

so, you want to support Right-To-Left (aka RTL) languages in your Gaia application, but you don't really know how to do this. Here is a summary of what I learnt while fixing bug 1053708 [1] for the SMS app.


1. How to test RTL on the device

Easy:

* Enable "Developer menu" in Settings > Device Information > More Information
* Enable "Pseudo-localization" in Settings > Developer
* Configure "Mirrored english" in Settings > Language

2. How to test RTL in Firefox

If your application works in Firefox, it's easy with these steps:

* load the application in Firefox
* open Dev Tools
* execute this code:
  navigator.mozL10n.language.code = 'qps-plocm'
* if you want to move back to english:
  navigator.mozL10n.language.code = 'en-US'

3. Basic RTL introduction

As far as I understand, RTL rules are part of the Unicode specification. RTL is quite easy if a whole document is in LTR or RTL. Things get complicated when some part of a document is RTL while the document is LTR (it's called bi-directionality, aka bidi).
In Unicode, all non punctuation characters embed their writing directions, they're called "strong". In contrary, all punctuation characters are "weak".
Unicode also has special characters (called "pseudo strong") that have no direct output but change RTL behavior. As we'll see, we have both HTML attributes and CSS properties to change the behavior as well, and W3C advises to use them instead of the Unicode control characters [3].

Wikipedia has a nice article: http://en.wikipedia.org/wiki/Bi-directional_text

4. Basic rules

So, the good news is that all Unicode rules work fine since HTML4 [2].

5. CSS usual properties, what does work?

CSS has a lot of "left" and "right" and "X" and "Y". So what about them?
Looks like CSS has some RTL-aware properties and values.

text-align: supports start/end
* "start" is "left" in LTR, and "end" in RTL.
* "end" is the opposite

padding: instead of "padding-left" and "padding-right", we can use "-moz-padding-start" and "-moz-padding-end"
margin: instead of "margin-left" and "margin-right", we can use "-moz-margin-start" and "-moz-margin-end"
border: same
Shorthands don't work. But if you use shorthand with up to 3 values, then you don't need to use RTL-aware properties, because left and right get the same value.
dbaron said the start/end properties do not work exactly like specified in some situations, but they worked good enough for me.

Properties left/right with absolutely positioned elements have an interesting property: if there is a conflict between specified left/right/width and the available width, then the algorithm will ignore "right" in a LTR context, and will ignore "left" in a RTL context. You can see an example at [4].

display: inline or inline-block or flex; works out of the box as you'd expect

5. CSS usual properties, what does not work?

float: does not support start/end, but there has been discussion on the www-style mailing list.
transform: does not support RTL. No discussion yet.
background-position
top/left when you can't use the "conflict rule". There is some discussion to support "offset-start" and "offset-end" instead, and a bug in Gecko (bug 1071099 [7]). After dbaron, there is a "little" bit of work to do it properly :)

The only way to workaround this is to override the LTR CSS using the same selectors preceded by [dir=rtl] with the maintenance burden this gives. It could be a good moment to switch your "float"-based layout to flexible layout :)

6. HTML and CSS, changing how the BiDi algorithm works

Sometimes, the BiDi algorithm will produce an incorrect result for some of your cases. It's possible to change how the BiDi algorithm works. I'll explain here only what we have in HTML and CSS, but not what we have in pure Unicode.

But first I'll need to explain "embed", "override" and "isolate". I hope people that know this well won't say I'm simplifying too much ;)
* embed: let you set the direction of some text. We still use the BiDi algorithm, so if this text contains itself some text that goes to the opposite direction, then this should do what you expect. Also, this affects the text _around_ this text, which may do something you don't expect, especially with punctuation (see [5]).
* override: quite the same, except you don't use the BiDi algorithm anymore. So if this text contains itself some text that should go to the opposite direction, it does not.
* isolate: like embedding, except it does not affect the text around (see [5]).
* isolate-override: like override, but does not affect the text around.

There is also "plaintext" but I admit I don't get it yet.

These values are actually values for the CSS unicode-bidi property. isolate, isolate-override and plaintext are prefixed; and to get "override" you use "bidi-override" in CSS. And then, once you set once of these values, you can set the "direction" property.


As you see in [5], W3C thinks that "isolate" should now be used by default, and I think this is what is implemented in Gecko for some time.


HTML has <bdo> and <bdi> and the "dir" attribute. Basically:
* setting the "dir" attribute is like setting the CSS property "unicode-bidi: embed" with a "direction" property, but has more semantics (works without CSS loaded).
* <bdo dir=""> lets you overriding the BiDi algorithm, so it's like the CSS value "bidi-override", with more semantics.
* <bdi dir=""> lets you create a "isolate" part, so it's like the CSS value "isolate" with more semantics. It accepts the magic attribute "dir='auto'": means that the direction will be calculated using the first "strong" character. I don't know if we can have the same behavior with CSS only, probably yes.


You'll find interesting information in the CSS spec [6], and also some history about isolation in [5].




[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1053708
[2] http://www.w3.org/TR/html401/struct/dirlang.html
[3] http://www.w3.org/International/questions/qa-bidi-controls
[4] https://github.com/mozilla-b2g/gaia/pull/24301/files#diff-5c046a821160971efb3637477688f024R206
[5] http://www.w3.org/International/wiki/Html-bidi-isolation#Background
[6] http://dev.w3.org/csswg/css-writing-modes/#text-direction
[7] https://bugzilla.mozilla.org/show_bug.cgi?id=1071099


Hope it's been interesting enough :) Don't hesitate to correct, ask questions, or explain what "plaintext" does :)

Happy RTL-ing !
--
Julien

[Julien Wajsberg]

I just got another very useful resource from the W3C: http://www.w3.org/International/tutorials/bidi-xhtml/
It looks very informative as well!

_______________________________________________
dev-gaia mailing list
dev-...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-gaia

[Ehsan Akhgari]

Note that dir=auto works on all HTML elements, not just <bdi>. It
always looks at the text content of the element, and chooses the
directionality of the element based on the first strong character in the
text content. As you noted, there is no equivalent CSS property.

dir=auto is specially useful for places where you're displaying user
data where the direction of the text cannot be known beforehand, for
example:

<input id="first_name" dir="auto"> (because the user's first name may be
either RTL or LTR)
<input id="phone_number" dir="ltr"> (because phone numbers, and numbers
in general, are always written LTR, even in RTL languages)

On 2014-10-24 11:02 AM, Julien Wajsberg wrote:
> Hey,
>
> so, you want to support Right-To-Left (aka RTL) languages in your Gaia
> application, but you don't really know how to do this. Here is a summary
> of what I learnt while fixing bug 1053708 [1] for the SMS app.
>
>

> *1. How to test RTL on the device*

>
> Easy:
>
> * Enable "Developer menu" in Settings > Device Information > More
> Information
> * Enable "Pseudo-localization" in Settings > Developer
> * Configure "Mirrored english" in Settings > Language
>

> *2. How to test RTL in Firefox*

>
> If your application works in Firefox, it's easy with these steps:
>
> * load the application in Firefox
> * open Dev Tools
> * execute this code:
> navigator.mozL10n.language.code = 'qps-plocm'
> * if you want to move back to english:
> navigator.mozL10n.language.code = 'en-US'
>

> *3. Basic RTL introduction*

>
> As far as I understand, RTL rules are part of the Unicode specification.
> RTL is quite easy if a whole document is in LTR or RTL. Things get
> complicated when some part of a document is RTL while the document is
> LTR (it's called bi-directionality, aka bidi).
> In Unicode, all non punctuation characters embed their writing
> directions, they're called "strong". In contrary, all punctuation
> characters are "weak".
> Unicode also has special characters (called "pseudo strong") that have
> no direct output but change RTL behavior. As we'll see, we have both
> HTML attributes and CSS properties to change the behavior as well, and
> W3C advises to use them instead of the Unicode control characters [3].
>
> Wikipedia has a nice article:
> http://en.wikipedia.org/wiki/Bi-directional_text
>

> *4. Basic rules*

>
> So, the good news is that all Unicode rules work fine since HTML4 [2].
>

> *5. CSS usual properties**, what does work?*

>
> CSS has a lot of "left" and "right" and "X" and "Y". So what about them?
> Looks like CSS has some RTL-aware properties and values.
>
> text-align: supports start/end
> * "start" is "left" in LTR, and "end" in RTL.
> * "end" is the opposite
>
> padding: instead of "padding-left" and "padding-right", we can use
> "-moz-padding-start" and "-moz-padding-end"
> margin: instead of "margin-left" and "margin-right", we can use
> "-moz-margin-start" and "-moz-margin-end"
> border: same
> Shorthands don't work. But if you use shorthand with up to 3 values,
> then you don't need to use RTL-aware properties, because left and right
> get the same value.
> dbaron said the start/end properties do not work exactly like specified
> in some situations, but they worked good enough for me.
>
> Properties left/right with absolutely positioned elements have an
> interesting property: if there is a conflict between specified
> left/right/width and the available width, then the algorithm will ignore
> "right" in a LTR context, and will ignore "left" in a RTL context. You
> can see an example at [4].
>
> display: inline or inline-block or flex; works out of the box as you'd
> expect
>

> *5. **CSS usual properties**, what does not work?*

>
> float: does not support start/end, but there has been discussion on the
> www-style mailing list.
> transform: does not support RTL. No discussion yet.
> background-position
> top/left when you can't use the "conflict rule". There is some
> discussion to support "offset-start" and "offset-end" instead, and a bug
> in Gecko (bug 1071099 [7]). After dbaron, there is a "little" bit of
> work to do it properly :)
>
> The only way to workaround this is to override the LTR CSS using the
> same selectors preceded by [dir=rtl] with the maintenance burden this
> gives. It could be a good moment to switch your "float"-based layout to
> flexible layout :)
>

> *6. HTML and CSS, changing how the BiDi algorithm works*

[L. David Baron]

On Friday 2014-10-24 11:39 -0400, Ehsan Akhgari wrote:
> Note that dir=auto works on all HTML elements, not just <bdi>. It
> always looks at the text content of the element, and chooses the
> directionality of the element based on the first strong character in
> the text content. As you noted, there is no equivalent CSS
> property.
>
> dir=auto is specially useful for places where you're displaying user
> data where the direction of the text cannot be known beforehand, for
> example:

And note that because dir=auto exists, you don't want to write
selectors using [dir=rtl] or [dir=ltr] attribute selectors.
Instead, you should use the :-moz-dir(ltr) or :-moz-dir(rtl)
pseudo-classes. (This is the :dir() feature in Selectors Level 4.
I should investigate getting it unprefixed, following up on
https://bugzilla.mozilla.org/show_bug.cgi?id=562169#c71 .)

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla https://www.mozilla.org/ 𝄂
Before I built a wall I'd ask to know
What I was walling in or walling out,
And to whom I was like to give offense.
- Robert Frost, Mending Wall (1914)

[Julien Wajsberg]

Le 26/10/2014 01:14, L. David Baron a écrit :
> On Friday 2014-10-24 11:39 -0400, Ehsan Akhgari wrote:
>> Note that dir=auto works on all HTML elements, not just <bdi>. It
>> always looks at the text content of the element, and chooses the
>> directionality of the element based on the first strong character in
>> the text content. As you noted, there is no equivalent CSS
>> property.
>>
>> dir=auto is specially useful for places where you're displaying user
>> data where the direction of the text cannot be known beforehand, for
>> example:
> And note that because dir=auto exists, you don't want to write
> selectors using [dir=rtl] or [dir=ltr] attribute selectors.
> Instead, you should use the :-moz-dir(ltr) or :-moz-dir(rtl)
> pseudo-classes. (This is the :dir() feature in Selectors Level 4.
> I should investigate getting it unprefixed, following up on
> https://bugzilla.mozilla.org/show_bug.cgi?id=562169#c71 .)
>

The selectors we'll use with [dir=rtl] will have something after
[dir=rtl], because we really want to match its descendants. More
accurately we should actually use html[dir=rtl] (tell me if I'm wrong:
the root element is the only element where the configured direction is
inherited).

AFAIU when we generally use "dir=auto" it won't apply to its block
descendants ?

[L. David Baron]

On Sunday 2014-10-26 17:44 +0100, Julien Wajsberg wrote:
> The selectors we'll use with [dir=rtl] will have something after
> [dir=rtl], because we really want to match its descendants. More
> accurately we should actually use html[dir=rtl] (tell me if I'm wrong:
> the root element is the only element where the configured direction is
> inherited).
>
> AFAIU when we generally use "dir=auto" it won't apply to its block
> descendants ?

No, the dir attribute should be inherited to descendants in all
cases. See
https://html.spec.whatwg.org/multipage/dom.html#the-dir-attribute

[Wilson Page]

I made a demo [1] to see if `-moz-dir()` works inside shadow-dom.

• Layout of content inside the shadow-dom responds to changes to the document's `dir` attribute.
  
• `-moz-dir()` inside shadow-dom syle-sheet doesn't represent the parent document's `dir` attribute.
  

I'm not sure if this is the intended behaviour or not.

[1] http://jsbin.com/talite/1/

WILSON PAGE

Front-end Developer
Firefox OS (Gaia)
London Office

Twitter: @wilsonpage
IRC: wilsonpage

---

From: "L. David Baron" <dba...@dbaron.org>
To: "Julien Wajsberg" <jwaj...@mozilla.com>
Cc: "Ehsan Akhgari" <ehsan....@gmail.com>, dev-...@lists.mozilla.org
Sent: Sunday, October 26, 2014 4:58:38 PM
Subject: Re: RTL and bidi in Gaia, Gecko support, and planned CSS

[David Flanagan]

On 10/26/14 9:44 AM, Julien Wajsberg wrote:
>
> The selectors we'll use with [dir=rtl] will have something after
> [dir=rtl], because we really want to match its descendants. More
> accurately we should actually use html[dir=rtl] (tell me if I'm wrong:
> the root element is the only element where the configured direction is
> inherited).
>
>

Thanks, Julien for starting this very helpful thread! In some cases we
need to know the default writing direction of the locale in JS. (For
things like music playback progress indicators where when the user drags
the playback thumb left we need to know if that is toward the start of
the song or toward the end of the song)

Is there a best practice for determining the direction of the current
locale? I assume something like this would work:

var isRTL = false;
navigator.mozL10N.ready(function() {
isRTL = (document.documentElement.dir === 'rtl');
});

Anyone have other JS best practices for dealing with RTL and bidi?

David

[Zibi Braniecki]

On Tuesday, November 11, 2014 3:26:02 PM UTC-8, David Flanagan wrote:
> Is there a best practice for determining the direction of the current
> locale? I assume something like this would work:
>
> var isRTL = false;
> navigator.mozL10N.ready(function() {
> isRTL = (document.documentElement.dir === 'rtl');
> });
>
> Anyone have other JS best practices for dealing with RTL and bidi?

https://developer.mozilla.org/en-US/docs/Web/API/L10n.language.direction

zb.

[Sergi Mansilla]

Sounds like this current move to BiDi-friendly Gaia would be a great opportunity for pushing Web Components that handle RTL automatically. We would save quite a lot of work for things like headers/footers, checkboxes, etc.

[Stephany Wilkes]

I believe Wilson did some work of this nature for header - before we realized it would need to wait for 2.3, when we'd have more time to make edge gestures and task manager match the direction of the back button (or vice versa) in the Header.

But this has always been my great hope for bidi work: that web components could support bidi and that apps that call the common controls/web components would thus benefit from that work.

Stephany

----- Original Message -----
From: "Sergi Mansilla" <sergi.m...@gmail.com>
To: mozilla-...@lists.mozilla.org
Sent: Thursday, November 13, 2014 3:18:40 AM
Subject: Re: RTL and bidi in Gaia, Gecko support, and planned CSS

On Wednesday, November 12, 2014 5:31:03 AM UTC+1, Zibi Braniecki wrote:

Sounds like this current move to BiDi-friendly Gaia would be a great opportunity for pushing Web Components that handle RTL automatically. We would save quite a lot of work for things like headers/footers, checkboxes, etc.

[James Burke]

On Fri, Oct 24, 2014 at 8:02 AM, Julien Wajsberg <jwaj...@mozilla.com> wrote:

6. HTML and CSS, changing how the BiDi algorithm works

For email, I ran across something at the intersection of the direction of the text vs. the alignment of the UI for an RTL language. I ended up doing what is described below, but open to other guidance, still very new to this:

Here are my general approach for some of the rules:

1) use dir=auto on HTML elements that show text that came from user entry. This is both for display elements (like li/div/span) and for text inputs.

2) Use `html:-moz-dir(rtl) <other-selectors-here>` for targeting style overrides for RTL languages. Use flexbox, -moz-(padding|margin)-(start|end) CSS to avoid these. I ended up using some for some background-position cases, and then for some text-align: right overrides for the following case:

An intersection of dir=“auto” that shows some English text, like an email message from an English speaking sender, so has English (LTR) in sender name, subject, and email snippet.

When viewing this in an RTL language (using the Mirrored English psuedo locale for example), this led to some weird display of the message list. For RTL-based messages, I believe the sender, subject and snippet would all be right-aligned, and looks nice with the general right-favored UI. However the English message would have its trailing space on the right, and it broke the symmetry of the list.

Removing the dir=“auto” attribute from the spans used for the text display was not the right answer, because that affects the … ellipses used for `text-overflow: ellipsis` truncation. Without dir=“auto” the ellipsis was added to the “start” of the English text.

So that led to using a `html:-moz-dir(rtl) .class-of-span` override to set the text to be right-aligned, so that for all messages, they had text up against the right side of the list. It also seems like I should explicitly set text-align: left for the LTR case, to get a similar alignment for mixed text cases in LTR locales.

Happy to hear of other solutions though. The general problem is might be stated as:

What to do when slots for text can have user input, that text is generally smaller than that slot, but we might prefer a particular alignment for visual/UI layout concerns. Another example might be a list of items in a select box that can contain both LTR and RTL text entries.

James

[Julien Wajsberg]

Do we have an update on this?
Wilson, I think it makes sense to file a bug about this, right ?

Le 27/10/2014 14:19, Wilson Page a écrit :
> I made a demo [1] to see if `-moz-dir()` works inside shadow-dom.
>
>

> * Layout of content inside the shadow-dom responds to changes to the document's `dir` attribute.
> * `-moz-dir()` inside shadow-dom syle-sheet doesn't represent the parent document's `dir` attribute.

>
> I'm not sure if this is the intended behaviour or not.
>
> [1] http://jsbin.com/talite/1/
>
> WILSON PAGE
>
> Front-end Developer
> Firefox OS (Gaia)
> London Office
>
> Twitter: @wilsonpage
> IRC: wilsonpage
>

> ----- Original Message -----
>

> From: "L. David Baron" <dba...@dbaron.org>
> To: "Julien Wajsberg" <jwaj...@mozilla.com>
> Cc: "Ehsan Akhgari" <ehsan....@gmail.com>, dev-...@lists.mozilla.org
> Sent: Sunday, October 26, 2014 4:58:38 PM

> Subject: Re: RTL and bidi in Gaia, Gecko support, and planned CSS
>

> On Sunday 2014-10-26 17:44 +0100, Julien Wajsberg wrote:
>> The selectors we'll use with [dir=rtl] will have something after
>> [dir=rtl], because we really want to match its descendants. More
>> accurately we should actually use html[dir=rtl] (tell me if I'm wrong:
>> the root element is the only element where the configured direction is
>> inherited).
>>

[Julien Wajsberg]

Le 14/11/2014 23:17, James Burke a écrit :
> When viewing this in an RTL language (using the Mirrored English psuedo
> locale for example), this led to some weird display of the message list.
> For RTL-based messages, I believe the sender, subject and snippet would all
> be right-aligned, and looks nice with the general right-favored UI. However
> the English message would have its trailing space on the right, and it
> broke the symmetry of the list.
>
> Removing the dir=“auto” attribute from the spans used for the text display
> was not the right answer, because that affects the … ellipses used for
> `text-overflow: ellipsis` truncation. Without dir=“auto” the ellipsis was
> added to the “start” of the English text.
>
> So that led to using a `html:-moz-dir(rtl) .class-of-span` override to set
> the text to be right-aligned, so that for all messages, they had text up
> against the right side of the list. It also seems like I should explicitly
> set text-align: left for the LTR case, to get a similar alignment for mixed
> text cases in LTR locales.
>
> Happy to hear of other solutions though. The general problem is might be
> stated as:
>
> What to do when slots for text can have user input, that text is generally
> smaller than that slot, but we might prefer a particular alignment for
> visual/UI layout concerns. Another example might be a list of items in a
> select box that can contain both LTR and RTL text entries.
>

So, we had basically a similar issue in the SMS' recipient panel, and
used a similar solution.

Maybe another possibility is using "width: -moz-fit-content" so that it
doesn't have the trailing white space but I haven't investigated more.

--
Julien

[Naoki Hirata]

It looks like Gandalf had sent a few docs to us before:
https://developer.mozilla.org/en-US/Apps/Build/Localization/Localizing_Firefox_OS_Apps
The doc above talks about the language property and the setter/getter.

https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_content_best_practices
https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/localization_code_best_practices

Regards,
Naoki