The Conformance Tests and Readium SDK/js

[Ric Wright]

Hello, group.  For those of you who don't know me, I am the Director of Engineering for the Readium Foundation, responsible for managing the Readium SDK, Readium.js and a couple of nascent projects bubbling on the back-burner.  In the past, I was responsible for creating and driving to fruition along with Bill McCoy and a bunch of great engineers and QE, the Adobe eBook effort ( RMSDK, ACS4, Digital Editions).

I am currently looking at the conformance tests in the context of Readium SDK.  I started a Google doc on my efforts with the test web site here:

https://docs.google.com/document/d/1XcVhrY8xSZ1kSpo32ViwxeTsMYZBpcX_qPeH0BL3i0A/edit

I have a number of comments and suggestions about the conformance suite.  I like to keep notes when I am working through these types of efforts.  Markus suggested I log the obvious ones as issues on the conformance github site (https://github.com/mgylling/epub-testsuite), which I am happy to do.

I am going to start YAGD (yet another Google doc) for the conformance test comments since there will be some odd ball comments and questions which aren't really "issues".  Also I am also poking at iBooks (as an example of an EPUB 3 RS which SHOULD be correct (though I have already found some weird bugs).  The comments are here:

https://docs.google.com/document/d/1S1ZpY76_i3hGYYcRFrgbWpcZYBk1lHxphhiFVHwDLK0/edit#

Feel free to add notes directly in the doc, if you like, or send me replies directly.

Thanks

Ric

[Ric Wright]

My apologies.  I did share the document with this mailing group, but I inadvertently misspelled the group's name.  I have now corrected that and also explicitly OK'ed sharing with those who sent an email asking for access.  Sorry for the hassle.

Ric

--
You received this message because you are subscribed to the Google Groups "epub-testsuite-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to epub-testsuite-di...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ori Idan]

Note that the test suite consists of two main parts:

1. The actual tests EPUB

2. The reporting system

From reading your comments it seems that you are talking about the reporting system.

The reporting system is in draft state only and will probably be integrated with a web application in the future.

-- 

Ori Idan

[Ric Wright]

Ori,  

There are two documents up there,  the one about the test site:

https://docs.google.com/document/d/1XcVhrY8xSZ1kSpo32ViwxeTsMYZBpcX_qPeH0BL3i0A/edit

And another about the tests themselves

https://docs.google.com/document/d/1S1ZpY76_i3hGYYcRFrgbWpcZYBk1lHxphhiFVHwDLK0/edit

Ric

[Ori Idan]

Thank you for your comments.

We tried to make as many tests as possible in the same book but it is not possible to make all tests together.

For example FL can not be mixed with RFL.

It can be mixed according to the standard but most readers do not support it.

Also as for iPad, as much as I know iBooks has the best support for EPUB3 but it is not complete.

Unfortunately I am still looking for a reading system to support all EPUB3 features.

I would be happy to work with the Readium foundation to make it support all EPUB3 features.

I think the test suite is only one of the tools to test it.

Currently it is not completed and I would be happy if you can contribute by forking it on GitHub and add your changes, then commit it back to the original repository (I will review them and do the final merge.)

-- 

Ori Idan

[Ric Wright]

Ori,

Thanks.  To be clear, I think the conformance tests AND the website are both awesome.

I will try to make some of the changes.  I have a lot on my plate but I do want to contribute.

The offer to help stretch Readium SDK is great and much appreciated.  Readium SDK does in fact support both fixed and reflowable content in a single EPUB.  It is out goal to support every single feature in 3.0 and then 3.0.1

Thanks

[Ori Idan]

On Wed, Sep 25, 2013 at 8:35 PM, Ric Wright <rkwr...@geofx.com> wrote:

Ori,

Thanks.  To be clear, I think the conformance tests AND the website are both awesome.

Thank you. 

I will try to make some of the changes.  I have a lot on my plate but I do want to contribute.

I look forward to see your comments.

The offer to help stretch Readium SDK is great and much appreciated.  Readium SDK does in fact support both fixed and reflowable content in a single EPUB.  It is out goal to support every single feature in 3.0 and then 3.0.1

I have tried few fixed layout books that worked great on iBooks on readium.js and simple-readium.js I could display one page at a time with many problems. I could not find how to display facing pages.

I might be doing something wrong. Is there anyone I can consult about it?

-- 

Ori Idan

[Ric Wright]

Well, that area of the layout got rewritten recently and the person who did most of the work (Boris Schneiderman) is on vacation til next Monday.  Can you log an issue on github or send me the relevant info and I would be happy to do so.

[Matt Garrish]

To pick back up on this thread, I was looking at the FXL tests and find myself agreeing with what Ric has noted in his comments. I was going to take a stab at some clean up the descriptions and success criteria, but also noticed along the way that they aren’t atomic tests, but are coupling two behaviours (e.g., most of the test require checking that there is no fault with the orientation and no fault with synthetic spreads being generated to pass, but either could fail independently of the other).

 

I’m still okay with doing some editorial work, but I was wondering whether we need to keep all these permutations in the test suite? Wouldn’t it be simpler to test all the global values of rendition:orientation leaving rendition:spread set to auto and then test all the values of rendition:spread leaving rendition:orientation set to auto? We could drop a third of the combinatory tests that way and get the same support information.

 

At least in my mind (which is known to be buggy), if one or the other global setting fails in isolation, it will fail in combination with any other global value, and the we’re not out to test how ugly something might look (e.g., forcing all pages into landscape mode in a synthetic spread when the device is held in portrait mode to see how tiny the pages come out).

 

As it is now, if we keep the current set up, but split the testing within each publication for each global value, we end up testing the same functionality repeatedly.

 

Matt

 

From: Ric Wright

Sent: Tuesday, September 24, 2013 1:34 PM

To: epub-testsu...@googlegroups.com

Subject: The Conformance Tests and Readium SDK/js

 

[Ric Wright]

FWIW, I agree with Matt.  There are probably some edge cases where the atomic/global values fail only in certain combinations, but I agree that we should test them separately first and let the more complex, "real world" cases be tested in subsequent passes (or in real-world examples).

Ric

[Ori Idan]

I think that Matt is right, if one of global settings fail, it will fail the combination so we can make it simpler.

Ori Idan

[Markus Gylling]

Agree this makes sense. Matt, would you be willing to take a stab on this?

/markus

[Matt Garrish]

Sure, I'll make a run through and see what I can do to minimize and tidy...

-----Original Message-----
From: Markus Gylling
Sent: Tuesday, October 22, 2013 1:26 PM
To: epub-testsu...@googlegroups.com

[Matt Garrish]

More a note, but I noticed that all the FXL tests are list as required, but
in fact there are very few "musts" in terms of handling (mostly that the
reading system "should" do what has been set). I'm going to go through and
tweak to reflect the spec wording.

[Strickland, Joel]

Markus,

I could work on refactoring these cases if you'd like.

Thanks,

-Joel

[Matt Garrish]

Hi Joel,

I've committed the changes from below, basically reducing the test documents
by checking each global support property independently (plus some general
editorial stuff). If you want to do a review and see if there are any
additional changes or improvements that can be made, please feel free to.

Regards,

Matt

[Strickland, Joel]

Hi Matt,

Sure, I'll take a look.