Mobile Accessible Book Generator: The 52 hour E-Book Hackathon for Children With Print Disabilities

[Zaid Al-Timimi]

The 52 hour E-Book Hackathon for Children With Print Disabilities

During last weekend's Startup Weekend DC, I worked with a few people on the "Mobile Accessible Book Generator" project which we discussed during the MLK Library Accessibility Hackathon. Our goal was to prove or disprove the design, implementation, and workflow suggested by BookShare.org and further refined during the Accessibility Hackathon.

Eric, an attorney and non-developer, devoted his 52 hours to help me work on the Mobile Accessible Book Generator. A few "hackathon mentors" and an senior developer from Microsoft also provided some input.

Here are our findings:

A. Scanning Process: Using a scanner vs. a mobile phone's camera

Using a Mobile Phone's Camera:

1. For either a scanner or mobile phone use minimal resolution. For a phone, 2 megapixels is sufficient to capture the page's image and text. This means that a iPhone 3GS is a adequate and on Android, the camera app allows you to explicitly select a low resolution such as 2.5 megapixels
  
2. While easy to use, the mobile phone will capture images that include the table upon which the book lies. Though possible to crop the image using the camera app's editor, the images will never be "professional".

3. Since it is not easy to open a book with 1 hand and snap a picture with the other, images will contain a thumb and/or an index finger. Though possible to crop the image using the camera app's editor, the images will never be "professional".

4. On a mobile phone, you will have limited ability to changed the image name after taking the picture. In iOS you do not have access to the file system where the images are store but you do in Android. Nonetheless, it is too tedious to rename each file afterwords.

5. Emailing the images in order of capture is not too difficult but since books for kids probably do not have page numbers so Quality Assurance is dependent on the email order if the QA person does not have a copy of the book.

6. Other problems with using a camera include: glare from overhead lights and shadows and blurred images from jittery hands.

Using a Desktop Scanner:

6. For either a scanner or mobile phone use minimal resolution. For a scanner, never use 300 or 600 dpi and try to get as close to 72 dpi as possible which I think for many scanners will be 150 dpi. An image scan and not character recognition (OCR) is suggested because of the fonts used, irregular placement of text, and since some pages do not have any words whatsoever.

7. There are 3 types of desktop scanners: flatbed, overhead, and sheet-fed. Flatbed scanners are probably the best because some can auto-crop the image to only include the page. You will have to manually scan each page with a flatbed since many people do not want to break the spine or remove the spine of the book.

8. Using a desktop scanner will take 2-3 times longer than using a mobile phone's camera to capture a page but will be more accurate and will have less "noise" from reflections, shadows, fingers, etc

9. Using a desktop scanner usually allows you to name the outputted image files numerically such as 1, 2, 3, etc. and can correspond to the pages. This is very helpful in Quality Assurance and when typing the text and generating the e-book pages in the correct order.

10. It is probably best to create separate folders for each book. Until we create a program to manage the relationship between folders and books, it will be important to manually document the purpose of each folder in a text file and store that file with the images in the appropriate folder.

11. The book information: title, author, number of pages, copyright, publisher, etc should also be documented in a text file in the folder

B. Typing Process:

1. For images with a few lines of text at the bottom of a page, the iPhone/iPod Touch provides enough screen real-estate to see the image and type the corresponding text or dictate using the Siri technology on the iPhone 4S

2. For images with text at the top of the page, you will need to use an iPad and use the spit keyboard to see image which you type. You can still use the iPhone/iPod Touch, but you will have to scroll the editor several times or have a good memory.

If you have access to the book or a print out of the page, then an iPhone/iPod Touch will be adequate.

3. As suggested by the BookShare representative, using a desktop computer with a large screen is great because you can see the image and type with a physical keyboard. Unfortunately, the user has to go through so many more steps with inserting and resizing the images files, inserting a page break between pages, and remembering to save in RTF format. Furthermore, which RTF-compatible editor are we suggesting?

4. There should be some type of Quality Assurance since it is easy to misspell

C. Unresolved Issues:

1. As the BookShare representative correctly stated: some pages in children's books do not any words and only have a scenic image. How should we describe these pages and more generally pages which use uncommon words:

a. Place brackets around the text with a notice: [Editor's note: the following does not appear in the original text: ...] which has no relevance to a child and will be read by a text-to-speech system verbatim.

b. Use modern words and describe the image as if you are a moderator "Voice of God" type... there is probably a standard approach used by Closed Captioning editors and foreign language translators

c. Be unconcerned and just provide a literal transcription
 

2. How should we format the text:

a. With line breaks and quotes as in the original
b. Be unconcerned and just type the text

3. Do we expect the user to scan and type a book by themselves or can we use one group to scan and another to type? Nonetheless the QA step is needed after the scanning stage and the typing stage.

4. Do we expect the user to email directly to BookShare?

D. Conclusions

We were in general agreement that there are really 2 separate use cases:

1. E-Books created for personal use: the seeing person (parent or child) will not really care if the page's image has "noise" because of the greater benefit. These e-books can be created in a library, school, or at home. We have defined a workflow whereby 4 outputs can be generated:

1.1 text for braille devices and printers
1.2 an e-book with or without synthesized text-to-speech or a human voice
1.3 an audio book/file with synthesized text-to-speech or a human voice
1.4 a website with a page for each for book with the images, text, and audio

Is it possible that MLK Library become a clearinghouse for such "less than professionally created" items?

2. Because of its proprietary systems, e-book readers, or just the content in an RTF file created for BookShare will require significant QA and the use of desktop scanners. In other words, MLK Library will have to create a production group which will mostly be dedicated to creating content for BookShare.

3. From our weekend experience, we were in agreement that most people would not agree to spend much time creating content using a desktop computer and generating RTF files. Also, everyone expressed concerns in working for BookShare.org (irrespective of the nobility of the company's mission). Alternatively, everyone was more than willing to use their mobile phones and create e-books or just the e-book content for MLK Library.

There are more issues related to our 52-hours of work on this topic, but I hope that this email provides deeper insight into the possibility of implementing this project.

Thank you for reading this and making any comments.

--Zaid

Zaid Al-Timimi

@TheMashupApp
http://TheMashupApp.com

[Gerardo Capiel]

I'm excited to see momentum on this project.  We should not think of this app as a Bookshare specific app, but an app for creating accessible versions of printed books.  The target accessible format should be DAISY 3 (either text only, which is voiced by TTS, or text/audio).  You can learn more about the DAISY format at http://daisy.org.  Bookshare and many of the libraries of accessible materials around the world have standardized on DAISY.  In the next couple of years, EPUB 3 will replace DAISY, since many of the features found in DAISY 3 are incorporated in EPUB 3, thus instead of DAISY 4 we will have EPUB 3.

I'll have someone comment on our process for working with volunteers.

[Gerardo Capiel]

I should also add that we convert RTFs or EPUBs to DAISY.  The EPUB (v2) format has better support for embedded images, so we would prefer that, if we can't get DAISY.  The EPUB format would also work in many ebook readers, such as FBReader - an popular open source EPUB reader for Android.  We are currently, developing an accessible fork of that app - the project can be found at http://github.com/benetech/fbreaderj.

Also, the DAISY format has support for image descriptions as <PRODNOTES>.  We are currently working on a tool, called Poet, for crowdsourcing image descriptions.  You can learn more about that tool and image accessibility in general at http://diagramcenter.org.  Poet is an open source application.

Bookshare also distributes an iOS app called Read2Go, which allows people with print disabilities to listen to DAISY books.

Finally, books created with the book generator could enter our volunteer workflow via email, but I'll have our team comment on that.

[Zaid Al-Timimi]

Hello Gerardo and thank you for your comments.

I and many hackathon participants are new to the Accessibility community and we very much appreciate your guidance :-)

I have included both your most recent comments to make it easier to reference. Here is my current plan based on last weekend's meeting during the Big Data hackathon, your comments, and the Bookshare.org website:

1. There will be at least 2 phases to the Mobile Accessible Book Generator: a quick proof of concept implementation and a production implementation. 

2. The proof of concept implementation will include the following:

a. Architecture: mobile device centric

b. Book Selection: random children's books

c. Image capture: less than professional image capture, most likely using a desktop scanner for small books and camera images for over-sized books and pop-out pages which will not be edited

d. Image transfer to Text Caption Editor: manually transfer via iTunes File Sharing or email

e. Text Caption Editor: mobile device-based editor supports Speech-to-Text dictation + manually typing + bluetooth keyboard

f. Audio generation optional: CMU Flite and will accordingly use robotic voices

g. Output Format: individual pages of image + text + optional audio in HTML5 for incremental Quality Assurance + web publishing, iBooks compatible EPUB

h. Output Transfer: manually transfer via iTunes File Sharing or email 

3. The production implementation will include a workflow process defined in January 2012 and continuously refined subsequently

a. Architecture: TBD but probably some combination of cloud + desktop + mobile

b. Book Selection: TBD but probably curated list of children's books probably based on the Bookshare.org list as described on Bookshare.org 

c. Image capture: TBD but probably professional image capture, most likely using a desktop scanner for small-to-average sized books and camera images for over-sized books and pop-out pages which will need to be edited

d. Image transfer to Text Caption Editor: TBD but probably secure cloud transfer from desktop admin station

e. Text Caption Editor: TBD but probably mobile editor and/or desktop editor

f. Audio generation optional: TBD but probably commercial text-to-speech engine or human audio recording

g. Output Format: TBD but probably DAISY 3 + EPUB

h. Output Transfer: TBD but probably secure cloud transfer to desktop admin station

4. It would be great to run the e-books generated during the proof of concept phase through the Bookshare.org workflow.

5. It would be great to test the e-books generated during the proof of concept phase on the popular EPUB readers used in the community. We should probably create another category in the Wikimedia DC Accessibility pages for e-book readers much like we have a Section 508 category

6. In January/February the Lab can subsequently determine how it wants to proceed: either by augmenting the proof of concept process (for example, with professional image capture) and/or build the production implementation. 

... just some thoughts and I hope that everyone will provide guidance, comments, and ideas

Thank you.

--Zaid

Zaid Al-Timimi
@TheMashupApp
http://TheMashupApp.com

On Dec 5, 2011, Gerardo Capiel <gca...@gmail.com> wrote:

I'm excited to see momentum on this project.  We should not think of this app as a Bookshare specific app, but an app for creating accessible versions of printed books.  The target accessible format should be DAISY 3 (either text only, which is voiced by TTS, or text/audio).  You can learn more about the DAISY format at http://daisy.org.  Bookshare and many of the libraries of accessible materials around the world have standardized on DAISY.  In the next couple of years, EPUB 3 will replace DAISY, since many of the features found in DAISY 3 are incorporated in EPUB 3, thus instead of DAISY 4 we will have EPUB 3.

I'll have someone comment on our process for working with volunteers.