Intent to Implement: MathML Core

591 views
Skip to first unread message

Frédéric Wang

unread,
Oct 14, 2019, 4:52:24 AM10/14/19
to blin...@chromium.org


Design doc/Spec:

A design doc will be forthcoming, primarily focusing on blink specific integration with the current LayoutNG engine. The W3C TAG did a review of MathML3, we plan to request one for MathML Core: https://github.com/w3ctag/design-reviews/issues/313#issuecomment-460523527

Summary

We intend on implementing native DOM and OpenType-based layout of mathematical formulas.

Our reference specification is MathML Core which describes with extensive details a fundamental subset of the MathML 3 recommendation. This subset is based on well-established rules from TeX and Microsoft Word as well as practical implementation experience in Gecko and WebKit. It focuses on proper integration into the Web Platform and corresponds to features that are used in practice.

We are still in discussion within the MathML Refresh CG in order to determine which features are really important and whether other aspects have to be refined. However, the code that will be upstreamed and the set of features will more or less correspond to what is currently in our development branch https://github.com/Igalia/chromium-dev/commits/mathml-dev

This intent-to-implement is only about the DOM/layout part of MathML Core but that's already a relatively large piece of code and newly exposed features. For now it does not include any plan for exposing MathML via the accessibility tree or new low-level primitives for math layout, which are pending on this first step. See the Risks section for details.

Motivation

The most popular solutions for inserting mathematical formulas in Web pages are using static images or JS libraries but they cause considerable extra complexity for authors, performance concerns and inconsistencies for users.

One proposal to address these problems is to rely on Houdini standards but they are still developing and significantly in flux. Moreover, that leaves open the questions of what is lacking and of how to combine everything to get a full cross-browser math implementation.

Our proposal is instead to implement the MathML Core specification in order to:

  1. Provide users with efficient, natural, readable and high-quality rendering of mathematical notations, consistently with other text they encounter in the browser.
  1. Provide authors with native, efficient and interoperable rendering of mathematical notations that they are able to reason about consistently with the rest of the Web Platform.
  1. Rigorously define the necessary subset, how it works and properly integrates into the Web Platform and ensure testable and interoperable implementations.
  1. Establish a productive and agreeable starting point for additional work and conversation going forward and make it possible to more easily explore more, consistently with the rest of the platform.

Risks

    Interoperability and Compatibility

First, the features related exclusively to MathML layout are already implemented in Gecko and WebKit so this proposal is actually reducing the interoperability gap. However, Gecko and WebKit actually implement more MathML layout features than what is in MathML Core specification. The proposed solution is to carefully unship existing features from Gecko and WebKit. This is done by introducing preference flags and (in the case of Gecko) counters to measure usage statistics. Mozilla seems positive about this effort since that simplifies maintenance. Apple seems reluctant about unshipping any feature that could theoretically break existing content for their products.

Mozilla and Apple developers have been supportive of the work related to DOM/IDL enhancements for MathML elements. Patches have landed recently in Gecko and WebKit without a preference flag. These are strictly speaking not really new features but just normalizing MathML so that it behaves like HTML and SVG element.

Many MathML Core clarifications on how to interpret CSS properties (e.g. padding) are not implemented in Gecko or WebKit yet, so they have poor scores for the corresponding WPT tests and this is still a potential source of inconsistency. Although MathML users currently rarely use them in practice, it is important to address these for future native and non-native implementations effort and we plan to work on it. Both Mozilla and Apple seem positive about increasing MathML compatibility with CSS.

Another risk is that MathML Core is still a working draft, many changes are currently being discussed and applied and the web platform tests do not cover the whole specification yet. However, it is based on MathML3 features that are actually used in practice and implemented in WebKit and Gecko and the test suite contains about 2000 subtests with most of the non-CSS ones already passing in Gecko, WebKit and our Blink prototype.

Finally, our implementation relies on LayoutNG which is still being developed while legacy layout cause problems. In particular, MathML tables cannot rely on LayoutNG for now and MathML won't render properly in SVG or printed pages. We also have to implement our own low-level APIs for math-specific features since they are not available in Blink (e.g. shaping of stretchy operator, special layout constraints or new CSS properties). This is another source of code instability but is necessary to avoid extra risks with new Web-exposed features that are not shipped in other browsers.

Edge: No signals
Firefox: Shipped
Safari: Shipped

Web / Framework developers: Mixed but generally positive feeling. This is very complex, people from the same community or company do not necessarily agree and one person can belong to several of the community below. We are trying to summarize this:

- Chromium users: One of the features with the highest number of votes.

- Math onto Web CG and developers of MathJax: Very skeptical about native MathML support but aligned with our Web Extensible approach. Some of them have interest in MathML Core, CSS or ARIA efforts and have joined the corresponding W3C groups. https://www.w3.org/community/mathonwebpages https://github.com/w3ctag/design-reviews/issues/313#issuecomment-435356627

- STEM publishers: Very positive as this is used in EPUB standard and simplify E-books & E-readers. They also contributed to the development of the STIX Two math font. Some members joined the MathML Refresh CG. https://github.com/w3ctag/design-reviews/issues/313#issuecomment-434371365 https://twitter.com/ColibrioReader/status/1159785599773138944

- LaTeX community: Very negative about source code verbosity. Positive about using TeX rendering rules or OpenType MATH fonts. Some members wrote LaTeX-to-MathML conversion tools or participate to MathML Core activities.

- MathML community: Negative about unshipping features from browsers. However, some members joined the MathML Core community group and agree that moving less important MathML features to polyfill is a good trade-off if that allows to have MathML finally implemented in Blink.

- Developers of Office Applications and editing tools: Positive. MathML used in office formats, MathML Core compatible with Microsoft Office format. Some members participate to MathML Core activities. https://blogs.msdn.microsoft.com/murrays/2014/04/27/opentype-math-tables

- Developers of Computer Algebra System: Positive. Used as an exchange format. Wolfram actually pushed for the development of MathML http://www.mathmlcentral.com/history.html

- Accessibility community: Positive. Supported by assistive technologies and used in DAISY audio books. Aligned with our idea of removing "Content MathML" and instead relying on ARIA to enhance MathML semantics. Some members joined the MathML Refresh CG. https://github.com/w3ctag/design-reviews/issues/313#issuecomment-455567932 https://twitter.com/chaals/status/1152120903632732160 https://twitter.com/pauljadam/status/1143199920746184704

    Ergonomics

This implementation will rely on LayoutNG so we will get the benefits from the performance improvements there. Preferred width and layout passes only require one recursive call to the children, except for stretchy horizontal operator which may require two layout calls (with unstretched and stretched sizes).

Good mathematical rendering require to read font-specific features like accessing data from the MATH table, using ink text bounds or low-level shaping of stretchy operators. We need to keep an eye on the performance here but we haven't detected any observable issue so far.

Because of lack of pages math fonts shipped by default in operating systems, this feature is likely to require large WOFF fonts. Until the fonts are loaded, the math layout will look bad for users that don't have fonts or web extensions installed. In the future, a solution would be to either ship fonts or improve OpenType MATH to rely on variable fonts ( https://github.com/mathml-refresh/mathml/issues/135 ).

    Activation

Mathematical formulas are deeply nested trees and writing the raw source code of the corresponding MathML trees is impractical. Fortunately, a lot of authoring tools and generators have been developed including JS converters from simple text syntax that can be embedded in custom elements (e.g. https://fred-wang.github.io/TeXZilla/examples/customElement.html ).

We only plan to implement a subset of MathML Core which is itself a subset of MathMl3 features used in WebKit and Gecko. This means that our implementation might not work with all existing content. For these MathML features with low usage, the proposal to preserve backward compatibility is then to write polyfills that are based on real DOM and other current or future Extensible Web features. We will also encourage authors and developer of tools to use these polyfills and/or move to MathML Core.

The lack of math fonts shipped in Android ( https://github.com/googlefonts/noto-fonts/issues/330 ) and macOS & iOS (the former ships an obsolete and non-usable version of STIX) is problematic. MathML Core tries to provide fallback rules but these make implementation and testing more complex and still does not provide good enough math rendering anyway. Since we don't have control over pre-installed fonts, we will do outreach to authors and users about how to use WOFF fonts in pages and to install proper Web extensions.

Most MathML accessibility solutions rely on the DOM except the macOS (NSAccessibility) and Linux (ATK) ones, which are implemented in Gecko and WebKit. We should be sure to warn users that this is not supported yet in Chrome on non-iOS platforms and redirect to alternatives like ChromeVox.

Debuggability

We expect that there is nothing MathML-specific to do here and by design choosing MathML allows to have the same developer tooling for mathematical formulas as for SVG or HTML elements. However, our plan to normalize MathML DOM might help to avoid inconsistencies ( see e.g. https://bugzilla.mozilla.org/show_bug.cgi?id=1231085 in Firefox ).

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? Yes

Is this feature fully tested by web-platform-tests? No

There are about 2000 tests. They don't cover the whole specification yet (cf the per-section spec annotations). We will continue this effort while working on Chromium upstreaming and pursuing the parallel WebKit/Gecko effort mentioned in the Risks section.

MathML Core tests:
    
Tests for proposed CSS features:

Results are available in https://mathml-refresh.github.io/mathml-core/implementation-report.html and our Chromium prototype already has excellent pass rate.

Link to entry on the feature dashboard https://www.chromestatus.com/feature/5240822173794304

This entry was initially opened for the old MathML implementation in Chrome some years ago. For now we have just updated it a bit to avoid confusion regarding the MathML status. A complete update with fields mandatory for intent-to-implement is forthcoming.

Requesting approval to ship? No

-- 
Frédéric Wang

Ian Kilpatrick

unread,
Oct 15, 2019, 1:22:44 PM10/15/19
to Frédéric Wang, blink-dev
Thanks for the due diligence with regard to the specification and tests. This is a vast improvement over the past 9 months and gives us confidence that it's possible to have interoperable implementations in the future.

It is also great to see that this allows for polyfills on top of the "Core" implementation which wasn't possible in some implementations previously.
There will be issues and questions as the implementation proceeds, but looking good so far!

Please make sure that you coordinate with the various accessibility folks/teams to ensure that users have a good experience here. This will likely block an Intent to Ship.

Looking forward to reviewing the first patches!

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/81cfc001-7be4-adb1-0b89-0b0984e7435a%40igalia.com.

Rick Byers

unread,
Oct 15, 2019, 1:44:49 PM10/15/19
to Ian Kilpatrick, Frédéric Wang, blink-dev
Getting a useful subset of MathML to the quality/interoperability bar that chromium requires for major new platform features is going to be a lot of work. But it's clearly work that you (and others at Igalia) are expert at! It's impressive to see the progress that has been made here and I'm particularly excited to now see a viable path for aligning WebKit, Gecko and chromium on fully interoperable behavior!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJL3UpQ9MH_By6ZxEWCYrt-ocyfzaeHH_QpR%3Dyw0989ENKpRaw%40mail.gmail.com.

bkar...@gmail.com

unread,
Oct 18, 2019, 12:10:41 PM10/18/19
to blink-dev
Fwiw, we've been working very successfully with the other vendors and I'm very pleased to say that interop scores on these tests have already begun to improve and align. Very positive.

Frédéric Wang

unread,
Oct 31, 2019, 3:21:33 AM10/31/19
to blin...@chromium.org
Hi everybody,

Please find inline updates on this intent to implement.

We have made a big effort to discuss with different parties and address major concerns inherited from past MathML history. Although we expect more refinements of the spec, design doc and tests, we are now confident that the general roadmap has reached a wide enough consensus.

We were suggested by members of Google's layout team that "API mentor" and "Chrome launch review" are not required here. These two points are not very well documented, so we'd happy to hear more if you disagree.

Per https://www.chromium.org/blink/launching-features it seems we are now ready to proceed to the "Implementation Phase" and we will start to upstream our code.

Thank you everybody for your encouraging feedback!

On 14/10/2019 10:52, Frédéric Wang wrote:
This intent-to-implement is only about the DOM/layout part of MathML Core but that's already a relatively large piece of code and newly exposed features. For now it does not include any plan for exposing MathML via the accessibility tree or new low-level primitives for math layout, which are pending on this first step. See the Risks section for details.

...

Most MathML accessibility solutions rely on the DOM except the macOS (NSAccessibility) and Linux (ATK) ones, which are implemented in Gecko and WebKit. We should be sure to warn users that this is not supported yet in Chrome on non-iOS platforms and redirect to alternatives like ChromeVox.

Igalia is already collaborating with Google on accessibility and shall continue to do so for math. We will update this thread when we have more information but it looks like that, contrary to what was said here, the work on MathML accessibility can actually start without waiting for the layout code to land.

Design doc/Spec:

A design doc will be forthcoming, primarily focusing on blink specific integration with the current LayoutNG engine. The W3C TAG did a review of MathML3, we plan to request one for MathML Core: https://github.com/w3ctag/design-reviews/issues/313#issuecomment-460523527

First version of the design doc: https://docs.google.com/document/d/1biGEaWN8ThNTDtAbT1M5GIf6N5uQLWdxh2QhrG9uN5c

TAG review request for MathML Core: https://github.com/w3ctag/design-reviews/issues/438

We will continue to update the design doc to address current feedback and expect the TAG review to happen in December.

-- 
Frédéric Wang

Frédéric Wang

unread,
Nov 19, 2019, 12:47:09 PM11/19/19
to blin...@chromium.org, Dominic Mazzoni
On 31/10/2019 08:21, Frédéric Wang wrote:
...

Most MathML accessibility solutions rely on the DOM except the macOS (NSAccessibility) and Linux (ATK) ones, which are implemented in Gecko and WebKit. We should be sure to warn users that this is not supported yet in Chrome on non-iOS platforms and redirect to alternatives like ChromeVox.

Igalia is already collaborating with Google on accessibility and shall continue to do so for math. We will update this thread when we have more information but it looks like that, contrary to what was said here, the work on MathML accessibility can actually start without waiting for the layout code to land.

Hi,

During BlinkOn, we discussed a bit about the accessibility aspect with Dominic Mazzoni and a reasonable short-term plan for the next semester seems to try and achieve parity with Gecko and WebKit on Linux and macOS. A preliminary proposal has been added as a new section in the design doc:

https://docs.google.com/document/d/1biGEaWN8ThNTDtAbT1M5GIf6N5uQLWdxh2QhrG9uN5c/edit?ts=5da5ed29#heading=h.4lpwikd6z70m

I'm not exactly sure what are the plan and status for MathML accessibility support in other platforms (Android, Chromium OS and Windows) but I understand the current approaches are DOM-based and so shouldn't be affected by the rest of the changes.

Feedback and comments from more informed Chromium accessibility developers is welcome!

Thank you,

-- 
Frédéric Wang
Reply all
Reply to author
Forward
0 new messages