FW: MathJax and epub3

[Robert Miner]

Hi.

 

I’m moving this to the mathj...@googlegroups.com list so it will be archived.

 

@Peter and/or Alfred, on the assumption that you will be able to get things working now with this additional info, would you be willing to write up what all it took to get MathJax running in an epub file on a page on the dev wiki:

 

https://sites.google.com/site/mathjaxproject/

 

You have to have a google account for me to add you with authoring priviledges I think, but that isn’t a very big deal.

 

--Robert

 

From: Peter Krautzberger [mailto:pkra...@umich.edu]
Sent: Saturday, October 01, 2011 4:55 PM
To: Davide Cervone
Cc: Robert Miner; Alfred Wassermann
Subject: Re: MathJax and epub3

 

Davide,

 

Thanks so much for the help!

 

Best,

Peter.

On Sat, Oct 1, 2011 at 5:36 PM, Davide P. Cervone <dp...@union.edu> wrote:

I have listed the files that can be deleted for several other people.  Here is the upshot:

 

You can remove the MathJax/fonts/HTML-CSS/TeX/eot, svg, and png directories (since the WebKit engine will use the otf fonts).  You can also remove the two oft fles that start with MathJax_Win (since they are for Windows).  Then you can remove the MathJax/unpacked directory entirely, since you will be running the compressed versions of the files, and you can delete the MathJax/test and MathJax/docs directories completely.  If you are only using TeX input (not MathML), then I would recommend using the TeX-AMS_HTML-full configuration file.  In that case, you can remove the MathJax/jax/input/MathML and MathJax/jax/output/NativeMML directories, and the MathJax/extensions/mml2jax.js and MathJax/extensions/jsMath2jax.js extensions.  You can remove the FontWarnings and v1.0-warnings extensions as well, and you can delete all the configuration files except the one you are using.  Since you will be using the MathJax fonts, not the STIX fonts, you can remove the MathJax/jax/output/HTML-CSS/fonts/STIX directory and its subdirectories.  If you want to remove every possible file, then you can open the configuration file that you are using and remove the files listed in the MathJax.Ajax.Preloading() call at the top of the file, as those will all be included in the configuration file itself.  So you should be able to get things down to a smaller footprint if you need to.  I was able to get it to about 1.2MB that way.  To get more than that, you would have to limit the features you use.

 

Davide

 

 

On Sep 30, 2011, at 10:59 AM, Robert Miner wrote:

Peter,

 

Why not just ditch the image fonts, and set the config to use only system fonts.  Then you could leave all the images of the individual characters out, and you would only be left with a few dozen (?) js files.

 

--Robert

 

From: Peter Krautzberger [mailto:pkra...@umich.edu]
Sent: Friday, September 30, 2011 9:49 AM
To: Alfred Wassermann
Cc: Robert Miner
Subject: Re: MathJax and epub3

 

Dear Alfred,

 

I finally constructed an almost validating epub file -- including a full mathjax installation.

 

The biggest problem is the manifest. Since iBooks is very strict, all ~30.000 files have to be listed. I took me a while to find a reliable tool to create the manifest of 3.5MB (uncompressed). This, I think, was also the cause for most epub viewers to crash when trying to view the end product -- but surprisingly iBooks survived the test.

 

There are a lot of bugs in the displayed mathematics, but I'll have to test for the causes (on the one hand, the html content in my test file turned out not to be valid, on the other hand $\color{} ...$ seemed to consistently not render at all).

 

In any case, with the advent of epub3, a minimal mathjax focusing on mathml-rendering should make life much, much easier. I'll make both files (full mathjax, reduced mathjax) available after some more testing next week.

 

Best wishes,

Peter.

 

 

 

On Fri, Sep 30, 2011 at 8:29 AM, Alfred Wassermann <alfred.w...@uni-bayreuth.de> wrote:

Am 28.09.2011 05:16, schrieb Peter Krautzberger:

Dear Alfred,

Just a quick reply to your questions.

1) Converting to mathml ahead of time should be easy. I personally use
pandoc to create epubs for my tests which creates very reliable mathml
(out of pretty much anything). Of course, this brings us back to the
lack of mathml support. I'm somewhat in favor of the idea to focus on
successful conversion methods (back and forth).

2) Today, I created an epub with a reduced mathjax copy (of ~1.3MB)
which drops all fonts and produces mathml only. I haven't tested it
properly though. As before, the mathml problems remain.

3) Have you actually attempted to include all of mathjax? I'm curious if
that would actually work on an iPad. From what you wrote, it should.
I'll try that out.

Peter.

Dear Peter,
method 3) has been tested by one of our power users. Unfortunately, without success.
In the mathjax google groups some guys claim they could
include mathjax in an iOS app. But I did nt see the code, yet.
Best wishes,
Alfred

 

 

 

[Peter Krautzberger]

The "almost validated" epub I mentioned is unreliable on the iPad //
in iBooks (which is, for now, the point as the only hardware device
supporting javascript in epub2).

Sometimes, mathjax is being rendered, sometimes it is not -- I'm
guessing the size/structure causes these problems. When rendered,
there were also rendering problems: characters like \varepsilon tended
to be rendered twice (i.e., like \varepsilon \varepsilon).

I have also followed Davide's advice from the above conversation to
reduce the mathjax installation. This has different problems. iBooks
runs mathjax (slowly but reliably) with severe rendering problems: all
rendered results are displayed in the wrong place (starting at the
bottom right of the correct position, but no doubling).

I should add: calibre renders the file just fine. I will create a file
from the mathjax test docs and share it later.

Peter.

[Davide P. Cervone]

Peter:

Thanks for the info. Can you clarify something for me? Is the
positioning problem new to the stripped-down version, or is it also in
the original unreliable one (with the double \varepslion)? I have not
seen the misplacement that you describe before.

Can you send me the epub file so that I can take a look at it? I have
not worked with epub creation before, so if you could also send me
instructions for creating the file, I can see if I can figure out what
is going on.

Davide

[Peter Krautzberger]

Davide,

The two different versions have disjoint rendering problems in iBooks.

The "full" epub (when it was rendered) had things like double
\varepsilon

The "slim" epub (which always rendered) placed the rendered result as
oddly as I tried to describe.

I hope to find the time to write up a how-to with the files later this
week.

Peter.

[Davide P. Cervone]

If the "full" version didn't have the placement problems, then it
looks like something important may have been lost in the removal of
files. Not sure what, though. On the other hand, the fact that the
full version placed thing properly (if unreliably and sometimes
double), then it is hopeful that the problem is not too hard to fix.

Davide

[Peter Krautzberger]

That's a good idea. I'll experiment with deleting less from mathjax to
see how it goes.

[Alfred Wassermann]

I got the same result like Peter. All characters are in the wrong position. Also, the suggestion by Davide
    MathJax.Hub.Config({
      "HTML-CSS": {
        availableFonts: ["TeX"],
        imageFont: null
        }
    });

did not help.

Alfred

[Davide P. Cervone]

When you use MathJax in the epub, do you see the grey messages from
MathJax at the lower left? If so, are there any error messages among
them?

Davide

[Alfred Wassermann]

Yes, I see the grey messages, but no error messages.
But  there are no error messages.
Alfred

[Robert Miner]

I don’t know if this will shed any light, but Sean Hogan has finished up some nice diagnostic tools.  You can see them at

 

http://devel.mathjax.org/mathjax/shogun70/test/test/

 

And in particular, the section on Page Tests, does a lot of cross checking, logging of messages, etc.  They aren’t merged to master at GitHub yet, but you can grab them from Sean’s fork:

 

shogun70/MathJax

 

and I think it’s on his “test” branch: https://github.com/shogun70/MathJax/tree/test/test

 

--Robert

 

From: mathj...@googlegroups.com [mailto:mathj...@googlegroups.com] On Behalf Of Alfred Wassermann
Sent: Wednesday, October 05, 2011 6:01 AM
To: mathj...@googlegroups.com
Subject: Re: MathJax and epub3

 

Yes, I see the grey messages, but no error messages.

[Davide P. Cervone]

Do you see any message about "loading TeX/Main/Italic" or some such things?

I know that you can't access the MathJax menu, or the other usual diagnostic tools, so it is hard to get the details of what is going on, but it appears to not be using the MathJax fonts, and that makes me think something is going wrong with the loading of the fonts or font data.

Davide

[Alfred Wassermann]

It is hard to debug but I see the following sequence of messages (approximately):
* Processing Math: 0%
* Loading Web-font: TeX/Math/Italic
* Loading Web-font: TeX/...
* Processing Math: %
* Loading ???.js
* Processing Math: 100%

By the way in epub3 all the text files have to be XHTML files.

Alfred

[Davide P. Cervone]

OK, thanks. That does look like it is trying to load the MathJax fonts, and it does seem to think they are available (otherwise there would be "Failed to load TeX/math/Italic" and other similar messages). Can you send a higher resolution image of the messed up math (can you zoom in and take the picture)? I want to see the characters more carefully to see if I can tell what font it is. It appears NOT to be the actual MathJax fonts.

I will probably have to look into this myself, since it seems that it isn't going to be just a configuration change.

Davide

[Robert Miner]

Is lack of an iPad part of your problem?

--Robert

> -----Original Message-----
> From: mathj...@googlegroups.com [mailto:mathjax-
> d...@googlegroups.com] On Behalf Of Davide P. Cervone
> Sent: Wednesday, October 05, 2011 12:35 PM
> To: mathj...@googlegroups.com
> Subject: Re: MathJax and epub3
>

[Davide P. Cervone]

> Is lack of an iPad part of your problem?

No, I have one; it is lack of time while teaching and grading. (I had originally left the iPad at home in WV when I started teaching, but I picked it up this past weekend, so will have it to work with now whenever I get the time to do it.)

I suppose I should have said "yes", and you might have offered one. On the other hand, this is an original iPad, not an iPad2. :-)

Davide

[Davide P. Cervone]

I finally got the chance to view you book in iBooks, and I certainly
see the alignment issues that you have been having. I also can
confirm that the fonts in use for your book ARE the MathJax web fonts,
so that is good. But it leaves me in the dark about the problem with
the alignment. I will have to do some experimenting myself so that I
can get some debugging information out of MathJax in the epub file.
Make sure you didn't delete the mathjax/jax/output/HTML-CSS/fonts/TeX
directory (only the STIX one). Can you send me a directory listing of
the MathJax files that are actually included in the epub file (I
haven't yet determined how to unpackage the epub into its component
files).

Davide

[Davide P. Cervone]

OK, I was able to unpack the file, and see that the right files appear
to be there. I will have to look deeper. That means working out how
to create epub files. My initial attempts to unpack your epub file
and then repack it as an epub have failed. I guess I'll have to learn
how to do it properly. Sigh.

Davide

[Alfred Wassermann]

Thank you for putting so much energy into this problem.
The unpacked files of the epub booke are here:
http://jsxgraph.uni-bayreuth.de/~alfred/jsxgui/epub3/IntroJSXGraph/OPS/
For testing I left more files in the repository than you recommended.
I have a simple script to generate the mathjax file listing for the manifest in the file
package.opf:
http://jsxgraph.uni-bayreuth.de/~alfred/jsxgui/epub3/IntroJSXGraph/createManifest.sh
The output of this shell script has to pasted in the manifest part of this file.

package.opf without mathjax looks like

<?xml version="1.0" encoding="UTF-8"?>
<package xmlns="http://www.idpf.org/2007/opf" version="3.0" profile="http://www.idpf.org/epub/30/profile/package/" xml:lang="en" unique-identifier="pub-id">
  <metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
    <dc:title prefer="dcterms-title">Introduction to JSXGraph</dc:title>
    <meta property="dcterms:title" id="dcterms-title">Introduction to JSXGraph</meta>
    <meta about="#dcterms-title" property="title-type">primary</meta>
    <dc:creator prefer="auth">Alfred Wassermann</dc:creator>
    <meta property="dcterms:creator" id="auth">Alfred Wassermann</meta>
    <meta about="#auth" property="file-as">WASSERMANN, ALFRED</meta>
    <meta about="#auth" property="role" id="auth-role">aut</meta>
    <meta about="#auth-role" property="scheme" datatype="xsd:anyURI">http://id.loc.gov/vocabulary/relators</meta>
    <dc:identifier prefer="isbn" id="pub-id">urn:isbn:???</dc:identifier>
    <meta property="dcterms:identifier" id="isbn">urn:isbn:???</meta>
    <meta about="#pub-id" property="scheme" datatype="xsd:string">isbn</meta>
    <meta property="dcterms:language" id="pub-lang">en</meta>
    <dc:language prefer="pub-lang">e-DE</dc:language>
    <meta property="dcterms:modified">2011-10-5T12:00:00Z</meta>
    <meta property="dcterms:publisher">University of Bayreuth</meta>
    <meta property="dcterms:contributor" id="contrib1">Nobody</meta>
    <meta about="#contrib1" property="role" id="contrib1-role">mrk</meta>
    <meta about="#contrib1-role" property="scheme" datatype="xsd:anyURI">http://id.loc.gov/vocabulary/relators</meta>
  </metadata>
  <manifest>
    <item id="toc" properties="nav" href="toc.xhtml" media-type="application/xhtml+xml"/>
    <item id="copyright" href="copyright.xhtml" media-type="application/xhtml+xml"/>
    <item id="titlepage" href="titlepage.xhtml" media-type="application/xhtml+xml"/>
    <item id="cover" href="cover.xhtml" media-type="application/xhtml+xml"/>
    <item id="style" href="css/stylesheet.css" media-type="text/css"/>
    <item id="jsxgraphcore.js" href="javascript/javascript/jsxgraphcore.js" media-type="text/javascript"/>
    <item id="alogo-grayscale.png" href="images/logo-grayscale.png" media-type="image/jpeg"/>
    <item id="aGod_the_Geometer.jpg" href="images/God_the_Geometer.jpg" media-type="image/jpeg"/>
    <item id="xpreface_001" href="preface_001.xhtml" media-type="application/xhtml+xml"/>
    <item id="xchapter_001" href="chap_001.xhtml" media-type="application/xhtml+xml"/>
    <item id="xchapter_002" href="chap_002.xhtml" media-type="application/xhtml+xml"/>
    <item id="xchapter_003" href="chap_003.xhtml" media-type="application/xhtml+xml"/>
    <item id="xchapter_004" href="chap_004.xhtml" media-type="application/xhtml+xml"/>
    <item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>

       <!--
                      Here comes the mathjax part
        -->

  </manifest>
  <spine toc="ncx">
    <itemref idref="cover" linear="no"/>
    <itemref idref="titlepage" linear="yes"/>
    <itemref linear="yes" idref="xpreface_001"/>
    <itemref linear="yes" idref="xchapter_001"/>
    <itemref linear="yes" idref="xchapter_002"/>
    <itemref linear="yes" idref="xchapter_003"/>
    <itemref linear="yes" idref="xchapter_004"/>
    <itemref idref="copyright" linear="yes"/>
  </spine>
</package>

[Robert Miner]

Davide,

 

It looks like Alfred’s script is doing the hard part of making the EPUB.  In general, it isn’t very hard, and I’d be happy to try to help.  I’ve done it by hand with small examples.  The main thing is the manifest has to correspond to the files.  Otherwise, you should just be able to  stick with the file structure and zip it. 

 

--Robert

 

 

From: mathj...@googlegroups.com [mailto:mathj...@googlegroups.com] On Behalf Of Alfred Wassermann
Sent: Thursday, October 06, 2011 2:41 AM
To: mathj...@googlegroups.com
Subject: Re: MathJax and epub3

 

Thank you for putting so much energy into this problem.

[Davide P. Cervone]

Yes, I did figure that out eventually.  (But you do need to make sure the mimetype file is the first one in the epub file, apparently).  In any case, I was able to repack an epub file and view it, so I should be able to modify it and do some debugging.  It looks like it might be the mechanism for measuring the widths and heights isn't working in the ebook, so that could be a problem.  But that is just a guess at the moment.

One thing I'd like to try is the SVG output jax, since that should work more reliably.  If I could just get an hour free to try it!

Davide

[Davide P. Cervone]

Thanks, I'll see what I can do with this.

Davide

[Davide P. Cervone]

I have made some progress. I can now create epub files that include
MathJax, and I wrote a little debugging environment to help me be able
to see what is going on. Crude, but enough to get by. It does look
like there is a problem with the measuring of sizes of things, but I
haven't tracked it down exactly yet. I tried out the v2.0-candidate
branch, and that does a better job, at least, but is still getting
things to be too wide (though the vertical positioning seems better).
But the scaling is wrong (everything is twice as big as it should be),
so there are several things still to work out. In any case, it looks
like it should be able to be made to work, with a little more testing.

I also tried the SVG output, and that worked out better, but is also
twice as large as it should be, so that needs work, too. But I'm
confident that one or the other can be made to do the job. I'm out of
time at the moment, but will look into it further at my next
opportunity. I'm sorry it is not a quick fix.

Davide

On Oct 6, 2011, at 3:41 AM, Alfred Wassermann wrote:

[Peter Krautzberger]

Davide,

Sorry for being so quiet around here. I haven't had time to extend my
how-to on epub publically yet.

I also tried to use other alternatives to otf fonts and nothing
worked. Could you share an example of your SVG solution (just the
configuration or whatever else it needs)?

Your message seems to imply that SVG fonts have the wrong size, but
that's the only problem (i.e., they are placed correctly). That would
be a big step forward, I think.

[Davide P. Cervone]

The SVG code is not yet in the repository, but it is the next thing
that I hope to do when I have enough time to get the final pieces in
place.

Yes, the placement is correct for the SVG code, but there are some
scaling issues (it is not just that the whole expression is too big,
which it is, but the super and subscripts are not being reduced in
size properly, and I haven't had the chance to check out what is
causing that). So it is not quite ready for prime time yet, but does
look good for the future.

On the other hand, the v2.0-candidate code does a better job than
v1.1a, and doesn't have the "piling up" problem that you are seeing.
It is also scaled too large, and it looks like the widths of the
expressions aren't being calculated correctly. It think that the HTML-
CSS output can also be made to work, but it will take some time to
check into it, and I ran out of it. I'm hoping to get a few hours
next week to work on it again.

Davide

[Peter Krautzberger]

I just had the chance to upload a mathml-epub onto an ipad running
iOS5 . It didn't look too shabby. Will test more on the weekend (but
this means my silly how-to draft will be delayed).

Anybody else have a look already?

[Davide P. Cervone]

I did get iOS5, but haven't tried the MathML support. I assume it is
the same as that in Safari, which is better than I thought it would
be, but is not yet fully functional (and not sufficient to handle
MathJax's MathML output).

I had worked out a number of the difficulties with the HTML-CSS output
jax in iBooks 1.3, but the iOS5 update has caused new problems, so I
still have to work harder to figure that out. I have the positioning
and scaling issues worked out, but now the fonts don't seem to be
displaying properly. Not sure what that is about.

I also worked on the SVG problems, and they are straight-forward to
fix, so I do have that working even in iOS5. As soon as I can get a
version that is production ready, I'll let you know.

Davide

[Peter Krautzberger]

In case it's still of interest, I wrote up how I hack together my
epubs -- including two example files based on a blog post of mine.

The post can be found at http://boolesrings.org/krautzberger/2011/10/17/epub-mathjax-and-the-ipad-another-attempt/

Best,
Peter.

[Davide P. Cervone]

Thanks, I appreciate the instructions. I've already worked out my own process, but will keep this for reference for the future.

Davide

[Peter Krautzberger]

I think this fits in here: via twitter I heard about "eqate"

http://www.youtube.com/watch?v=VkQkzRb0e4k
http://www.evolu-group.com/

Has anybody heard of these before?

Peter.

[leathrum]

I posted a note on the MJ Users group earlier today.
It is getting a bit of discussion over there...

On Oct 27, 5:05 pm, Peter Krautzberger <p.krautzber...@googlemail.com>
wrote:

> I think this fits in here: via twitter I heard about "eqate"
>

> http://www.youtube.com/watch?v=VkQkzRb0e4khttp://www.evolu-group.com/