Extending MathJax to allow editing

[markdemich]

I know there's MathQuill, but I'm wondering if anyone has put some
thought into extending the MathJax code base to allow for editing.
I've written a few equation editors in my days in Java, Delphi and
ActionScript. I need to port my ActionScript (flash) one to
javascript. I was thinking that instead of re-inventing the wheel, I
could possibly extend MathJax to allow editing. Originally, I was
looking at MathQuill, but their redendering abilities are limited
compared to MathJax. I'm assuming MathJax maintains some sort of Math
dom for the equation in question. However, I suspect that since it's
purpose is to take MathML or some other input and just convert it to
HTML that it would require a descent amount of work to allow
manipulating that dom from the keyboard. If anyone familiar with the
code base has any thoughts on this, I would love to hear them. Thanks.

[Thomas Leathrum]

There are a couple of things I would suggest you look at before you re-invent the wheel -- first, the MathJax documentation for "Modifying Math on the Page":

http://www.mathjax.org/docs/2.0/typeset.html

And an associated sample:

http://cdn.mathjax.org/mathjax/latest/test/sample-dynamic.html

A couple of more sophisticated editors using MathJax are JaxEdit and HostMath:

http://code.google.com/p/jaxedit/

http://www.hostmath.com/

There is work going on to provide editing support in other editors, with the rendering driven by MathJax -- MathJax itself is really about the rendering, but the API does provide support for dynamic modifications, so editing on the fly is certainly feasible.

[markdemich]

Thanks for all the info. I looked at both jaxedit and hostmath, and
just to be clear, I'm looking at making a WYSIWYG math editor with
MathJax rendering on the fly "as the editor". Basically, I want to
implement MathQuill using the MathJax object model as the base.

Now, someone is probably asking, why not just use MathQuill. The
problem is that MathQuill is limited in what it can currently render.
Plus, I'll be using MathJax to render other non-editable equations.
So I figured it may be smarter to make a MathJax extension of sorts
that turns MathJax into an editor as opposed to modifying MathQuill to
allow rendering of things it doesn't currently render. Plus, I would
not have to worry about some equations looking different than others
because some were rendered using MathQuill and others MathJax.

[Davide P. Cervone]

> Thanks for all the info. I looked at both jaxedit and hostmath, and
> just to be clear, I'm looking at making a WYSIWYG math editor with
> MathJax rendering on the fly "as the editor". Basically, I want to
> implement MathQuill using the MathJax object model as the base.

If you mean using MathJax's internal representation of the equation as
the basis for your editor, I wouldn't recommend that. (If you mean
using the MathJax object-orientated programming library, then that is
OK.) The internal representation is basically presentation MathML,
which is probably not the best structure for an editor.

My recommendation would be to use your own data structure for encoding
your understanding of the equation being edited. It would probably be
a tree structure with operations like + and / being the nodes with two
branches, those being the operands on either side. You would also
need nodes for things like summations, parentheses and so on. If you
use an object-oriented programming model, you can have a method that
outputs MathML (or TeX or whatever) and pass that to MathJax for
rendering. When the person updates the equation in the editor, you
pass a new MathML or TeX string to MathJax and the equation is
rerendered. That avoids your having to learn about the internals of
MathJax. There is no guarantee that those are not going to change in
future versions, so you are safer to do it this way.

Davide

[jan.po...@gmail.com]

So I really want to ask: Why not just use MathQuill? Would you care to share the limitations you encountered? I am about to work on a WYSIWYG math editor as well, so it would be great to have a good starting point. Have you made any progress, meanwhile?

Thanks,

Jan

[Peter Krautzberger]

I hope you don't mind me responding from a MathJax point of view. 

Two things always struck me as odd about MathQuill. One, I could never find documentation on MathQuill's input syntax and two I don't think it can produce MathML. This may seem unnecessary for WYSIWYG point of view, but I think it's important. If input is non-standard and the result is not reusable, then you restrict your users unnecessarily and deny yourself benefits like accessibility, search engines picking up content etc. While MathML should be the format of choice in any HTML setting, LaTeX or asciimath are important options to re-use material easily.

Oh, and searching the user group gives you the following list.

• http://ubermeatshield.github.com/MathEditor/ (open source)
• wysiwhat (open source, not quite wysiwyg)
• http://www.wiris.net/demo/editor/demo/en/
• http://www.mathicando.com/en/
• http://sharemath.com/en/
• http://www.hostmath.com/Default.aspxen/
• http://formulasheet.com/#r%7Ced

Peter.

[Frédéric WANG]

On 25/01/2013 06:13, Peter Krautzberger wrote:
> two I don't think it can produce MathML. This may seem unnecessary for
> WYSIWYG point of view

Actually I think it is one important point to design the a Web-based
WYSIWYG editor. Currently, from a user point of view, the most obvious
differences between MathQuill and MathJax is that the former renders the
equation faster and has integrated support for input forms while the
latter has a wider input support and a better rendering quality. MathJax
can render the equation at least as fast if the MathML output Jax is
used and the input forms can be implemented via the valid
"<mtext><input></mtext>" syntax (there is a MathJax extension for
HTML-CSS/NativeMathML:
http://leathrum.github.com/mathjax-ext-contrib/forminput/sample.html).
Then of course we have to hope better MathML support in browsers, but
now at least Webkit/Gecko based browsers should support what MathQuill
uses. At the opposite, if one wants to improve MathQuill's input/output
to the same level as MathJax, without relying on MathML at all, then I
can't see how the library could avoid becoming bigger and slower...

--
Fr�d�ric Wang
maths-informatique-jeux.com/blog/frederic

[Jan Pöschko]

Thanks for your quick responses! I've been a MathJax user for quite a while now (with Mathics) and appreciate it a lot. The two things you mention are not so critical to me, though:

1. I would use my own way of letting the user input things anyway (no automatic conversion of e.g. "/" to a fraction, but using a toolbox instead). MathQuill or MathJax would just be used as a way to convert a given math formula/structure to an HTML+CSS structure which should then be editable.

2. I would provide ways to allow for both input and output in various formats, e.g. MathML.

The most important thing is really getting a DOM representation of a formula. While MathQuill's output seems really simple and straight-forward, the markup that MathJax generates is really hard to manipulate to inject input fields in there. To make things worse, simple input fields would not be enough, I would like to allow the user to insert images and all sorts of HTML in the typeset environment.

The MathEditor you mentioned (thanks for the link!) seems to go in the right direction, but appears rather unusable to me, still. The approach seems reasonable, though: Inject an input field into the output of MathJax, and on every change try to update the DOM as nicely as possible on the fly, but do a whole MathJax pass on the whole expression again, updating the DOM and placing the cursor again. However, I could imagine that becoming quite unusable for larger inputs.

Do you know of any examples where MathJax significantly outperforms MathQuill in terms of rendering quality (even after possibly improving fonts and styles), i.e., where the whole way of representing a formula directly using HTML+CSS (without sophisticated JavaScript measurement calculations/artificial DOM nodes) ultimately has to fail (as opposed to a JavaScript "black box" rendering pass)?

Cheers,

Jan

[Mark A. DeMichele]

The issue I have is that I’m converting an application that used to be written in flash and needs to work a certain way.  You can get an idea by going to http://www.interactmath.com, click enter,  choose the Adams: Calculus, 6e book.  Then go to Chapter 0, Section 1 and chose the first exercise.  You should see my flash based equation editor.  You can click on it and start play with it.  It does a lot of stuff.  It has shortcuts, syntax highlighting for functions and other stuff.  Though this example doesn’t show it, it handles matrices and other constructs as well.  We also have a read-only version of this that supports accessibility using MathSpeak.  I’m just scratching the surface though with describing all that it can do. 

 

So I needed to make a decision, do I start with MathQuill which is probably 10-15% there, and then possibly struggle to force fit my needs into MathQuill, or do I just build it from the ground up knowing all the features I would need.  If you read the MathQUill wiki, you’ll see that even the people that wrote MathQuill concede that it needs to be reworked for the future.  Anyway, I decided to just use MathQuill as an inspiration and code it from the ground up maintaining my  original architecture, but tweaking it to build html dom objects with CSS as opposed to Flash display objects.  This made it easier to maintain similar functionality.  Also, I’m using dojo and not jquery to write it since the rest of my app is using dojo.

 

That being said, all was going well until yesterday.  I had my basic version working on desktops and the ipad with safari.  However, I’m having a lot of trouble on android.  It all centers around the hidden textarea control you need to use to proxy the keyboard events.  I plan on verifying all this today, but from what I can tell so far, the android browsers do not have a keypress event.  Also, in the built-in android browser for ICS, it seems that if you have the keyboard open and you’re typing it will not allow you to update the html dom.  It lets you do it, but you don’t see the results until either you do a lot of changes or you leave focus.  This is a huge problem since the entire design depends on updating the html dom.  Today, I plan on making some test pages that use some simple pure javascript to demonstrate some of these issues so I can get a handle on how to deal with them.  I’m very worried right now about my approach because of this dom update issue.  I’m toying with the idea to have a mode that causes my editor to create a canvas or svg  instead of html.  Maybe those will update while the user is typing.  That’s a lot of work though so I’m going to try and deal with the issues using the dom if possible.

 

BTW, if you try the MathQuill demo (with the latest js from github or the demo on the site), it has various troubles as well.

 

Another idea I’m toying around with is it use a “fake” virtual keyboard instead of the real keyboard of the device.  I don’t really like that idea though.

 

Good luck.

[Peter Krautzberger]

@Jan I understand you problems, but these are all solvable -- that's what the MathJax team and our community is here for. In fact, you would be providing us with valuable feedback on what issues you encounter (and what APIs you'd need), while your users get the best rendering quality.

While I think we need (maybe even more) human readable formats, making up yet another one without any means for people to reuse it in the future is, imho, not a good idea. MathML is part of HTML5 and epub3 and is getting a lot of attention from search engines, assistive technology, and publishers. MathJax does a lot of conversion for you and we plan to expand that in the future.

@Mark I've been thinking about the keyboard idea recently and I'm beginning to like it as a fundamental idea. What was your experience with it?

Generally speaking, maybe we should have a google+ hangout about this?

--
 
 

[William F Hammond]

On Sat, Jan 26, 2013 at 5:25 PM, Peter Krautzberger
<p.kraut...@gmail.com> wrote in part:

> While I think we need (maybe even more) human readable formats, making up
> yet another one without any means for people to reuse it in the future is,
> imho, not a good idea. MathML is part of HTML5 and epub3 and is getting a
> lot of attention from search engines, assistive technology, and publishers.
> MathJax does a lot of conversion for you and we plan to expand that in the
> future.

Shouldn't things of this type ideally begin with literate source
markup? Literate source is maximally re-usable.

Short of that: reusable author-level source.

It's debatable whether html5 without MathML is author-level. But
MathML, though human readable in a sense, is not author-level.

See Kaveh Bazargan's talk in London this past December: "XML is not
just a format; it's much, much more",
http://river-valley.tv/xml-is-not-just-a-format-its-much-much-more/
and my TUG 2010 talk "LaTeX Profiles as Objects in the Category of
Markup Languages",
http://river-valley.tv/latex-profiles-as-objects-in-the-%E2%80%9Ccategory%E2%80%9D-of-markup-languages/

-- Bill

--
William F Hammond
http://www.albany.edu/~hammond/
Email: gel...@gmail.com

[Paul Topping]

Bill,

By "MathML is not author-level", do you mean that MathML is not an input language? I respectfully suggest that "input language" or "human input language" is a better phrase than "author-level". In these modern times, authors get involved in so many aspects of publishing these days.

The that I think best describes MathML, and opposes "input language" nicely, is "representation language". The fact that is readable in the sense that is a text file is neither here nor there.

I know you know all this already. I just offer it for the purpose of establishing some good phrases for this kind of discussion. It comes from uncounted times replying to TeX people who look at MathML aghast, wondering why anyone would think typing it would be a good thing.

Paul

[Jan Pöschko]

My concerns are really not so much about the input/output format, and the user would not face that either - if you like, it's the Mathematica (box) language, with buttons for all important constructs such as fractions, matrices, etc. A nice and responsive math editor is the primary goal.

As far as I understand MathJax, letting the user type in some hidden area and render the whole expression for display is the only feasible solution, due to the rendering chain (as opposed to MathQill, where the user types directly in the typeset content). This could work (e.g. CodeMirror works that way too - I don't think it supports Android either, though, which might interest you, Mark). I'm just afraid that this approach would be too slow for large expressions, so the user would not get immediate feedback while typing.

Having a Google hangout is a great idea! What about Tuesday afternoon? I'm on Central Time.

Thanks,

Jan

[William F Hammond]

Hi Paul,

On Sun, Jan 27, 2013 at 9:43 AM, you wrote:

> By "MathML is not author-level", do you mean that MathML is not an input language?

I mean that MathML is not a reasonable input language for human
authors. This, after all, is the reason why MathJax's TeX input is so
useful. I do think, however, that HTML with TeX math is still not
optimal for human authors.

Alas, my previous comment and your reply have taken us now rather far
away from the original topic in this thread. So let me try to
elaborate on my point by giving you this url:

http://www.albany.edu/~hammond/author-level/

[Jan Pöschko]

Thanks for sharing your experience, Mark! So you don't use MathQuill or MathJax at the moment, but your approach is similar to MathQuill's. Using HTML+CSS as much as possible and letting each node control itself sounds very reasonable, that is also how I would approach it.

Besides the keyboard problems on Android, are you missing anything fundamental? It doesn't sound like you are really considering switching to MathJax (or MathQuill) anymore. Although my guess would be that the rendering quality of MathJax is still superior?

How flexible is your approach with regards to the "input fields"? Would it be possible to leave the input and rendering of the basic lines to some other framework via some interface? Specifically, I'm thinking of syntax highlighting while typing (keywords, local variables...) within the math structure.

This is getting really interesting -- let's see when we find a common time to talk about this.

Jan

On Jan 29, 2013, at 12:49 AM, Peter Krautzberger wrote:

> Hi,
>
> I can't make tomorrow -- back to back meetings. To be sure that Davide Cervone (MathJax's mastermind and lead dev) will be there, I'll get in touch with him first and then set up a doodle or something.
>
> I know there are other people interested who've been quiet on this thread, so I'll announce that a bit wider.
>
> Peter.
>
>
>
>
> On Mon, Jan 28, 2013 at 5:13 AM, Mark A. DeMichele <de...@intellipro.com> wrote:
> I’m in meetings all day Tuesday, but I could do Wednesday. I’ve never done a Google hangout though so I may need a little help setting that up. I have a google account. Here’s my situation. I work for a large company (with Lawyers J ). Our product is a commercial product and the “equation editor” portion is a very small portion. I made a request to eventually open source our editor, but I ‘m not sure that will fly. Time will tell, but it will be a long time I think. I have a deadline of this summer to get my entire app working in javascript and html5. It’s currently written in actionscript (flash). The current somewhat working version of my editor (without all the constructs) is in decent shape. Though the code it currently a little messy and undocumented since I’m still experimenting with different techniques. I have various small bugs on different platforms, but it’s a work in progress and this is expected. Since I needed to mimic the existing functionality of my flash player, I adopted a similar architecture which is proving to save time since I’m extremely familiar with the previous code base. My overall philosophy is to create an equation dom made up of rows and nodes (a row is a node as well). This equation dom maps closely to the MathML structure, but to be honest I created this back in the day before MathML (originally in Delphi for an activeX control). Turns out I just got lucky. The idea is that each “node” is responsible for create an HTML dom node (or nodes) and also has optional code to update its own nodes when changes are sensed in child nodes. For instance a sqrt node needs to “stretch” it’s radical symbol when the contents changes size. Similarly, parens need to stretch as well. The resultant HTML is closer to MathQuill than MathJax. Like MathQuill, I’m building html dom structures that maintain themselves for the most part using CSS. I only need to add code to do odd stuff like stretching the height of things. So far it’s working out. However, my architecture is set up so any node can “mess” with the dom as things change so it is open ended.
>
>
>
> The hardest part so far is dealing with the built-in keyboard of mobile devices. I’m really close to getting something working, but I’m relying on some of dojo’s dijit framework to do some of the work so I’m not sure how my design will map to a jquery framework. Dijit’s have a nice feature to hook a “focusNode” to any dom node. This worked out perfectly for making a hidden input and associating with the entire editor. I still have a few bugs to work out, but it seems to be holding together. I haven’t done clipboard stuff yet. That will be challenging I think.
>
>
>
> So in summary, I have no problem sharing some of my knowledge gained by working on my editor, but right now, I don’t think I can share the code.
>
>
>
> Mark