OCR on Internet archive.

[विश्वासो वासुकिजः (Vishvas Vasuki)]

+ shankara, a frequent uploader, so that he may specify good language metadata

Lots of old sanskrit texts are uploaded to Internet archive (archive.org), and such uploads can be very convenient for future proofreading on wikisource (through various import tools which match the OCR text with the right page).

But the OCR for devanAgarI is just gibberish.

​So, I was wondering what software is used for OCR by the Internet Archive, and if we can somehow​ help improve it. So far, I discern the following:

• While uploading: To ensure good OCR, one must specify the right languages in the metadata [ https://archive.org/post/1052575/ocr-scanning ] Even if it is not useful for sanskrit right now, it will be useful in the future.
• Sometimes OCR does not happen automatically, in such cases, one may need to rederive the item (with the help of archive.org people?). [ https://archive.org/post/1053966/how-to-make-sure-a-scanned-pdf-file-will-be-ocred ]
• As of 2014 it used LuraTech software for pdf compression, and derived OCR text with the included ABBYY FineReader (which excludes several other modules generally available from ABBYY directly). [ https://archive.org/post/299315/ocr-on-fraktur ]
• ABBYY Finereader does not seem to have good devanAgarI (and CJK support as of 2010 - https://archive.org/post/299315/ocr-on-fraktur ].
• There have been proposals to use Tesseract in certain cases, but they have not fructified. [ https://archive.org/post/1010389/using-tesseract-to-improve-ocr-for-some-languages  ]

--

--
Vishvas /विश्वासः

[विश्वासो वासुकिजः (Vishvas Vasuki)]

Also: 

https://archive.org/about/faqs.php#1165 :

Why is OCR so bad? Why do the epub, djvu, mobi, text files have garbled or missing text?

OCR (Optical Character Recognition) is inexact. Sometimes it can be poor. It largely relies on factors of the physical book such as type font, color, cleanliness of the page, language (some are not OCRable at this time), and page orientation (sometimes charts are turned at 90°). At this time we do not offer a way for you to either correct bad OCR or add your own corrected OCR file. Several of the derived file formats such as mobi, epub and djvu rely on OCR. So, if the OCR is poor, those files will also have garbled or misspelled words.

Also added an inquiry here: https://archive.org/post/1056091/indian-language-ocr-no-good-can-we-help