We need a data set to run tests on for accuracy of our OCR
Debayan Banerjee
I think its time we started looking for a set of images covering most
Indian scripts that we can run tests on. In the longer run this set
should serve as the standard benchmark for all Indic OCRs. I know some
institutes in India have such sets but they are all restrictively
licensed.
There are people on this list who knwo what kind of images we should
be looking for. I request those list members to chip in with such
data.
--
Debayan Banerjee
satyaakam goswami
M.N.S.Rao
Will the scanned pages in kannada be enough or are you looking for custom
chars for test. If scanned pages I have a lot that I sent to YAGPO project.
Pl feel free to ask for scannig of kannada pages for test.
MNS Rao
Ashish Mahabal
Hi,
I have pdf pages from marathi vishvakosh. Some are scanned pages and
some are from machine conversions. The later should be easy if the OCR
is good, and the former will be a good test.
Let me know where I can deposit them.
Cheers,
ashish
http://www.astro.caltech.edu/~aam
Ashish Mahabal http://www.astro.caltech.edu/~aam
Abhaya Agarwal
- Recent Laser printed material on white paper makes for the easiest data set. This can be generated in as much amount as required and pretty quickly. See my comment at the end of the mail.
- Scanned pages of recently offset printed books. - Hard to get properly licensed data
- scanned pages of old books: this will be hardest. Apart from the age factor, the quality of printing in Indian language books is usually very bad with uneven ink spread. If you look closely at one of the attachments I sent earlier, you will also find very uneven inter word spacing. Getting the images for various languages is not an issue here, we can find sufficient number of out of copyright books in the DLI database and collect them.
- If someone already has scanned pages + the digital version of the same : let us collect all that. Then we can build a small interface to allow anyone to come and align it.
- If someone only has the scanned pages, word level alignments can be generated right at the time of typing. We automatically analyze the image and break into words and then show one word at a time for entry.
-------------------------------------------------
blog: http://abhaga.blogspot.com
Twitter: http://twitter.com/abhaga
-------------------------------------------------
M.N.S.Rao
M.N.S.Rao
----- Original Message -----From: Abhaya Agarwal
Sent: Wednesday, April 20, 2011 9:37 AM
Sankarshan Mukhopadhyay
<abhaya....@gmail.com> wrote:
> So primarily 2 tasks:
>
> If someone already has scanned pages + the digital version of the same : let
> us collect all that. Then we can build a small interface to allow anyone to
> come and align it.
> If someone only has the scanned pages, word level alignments can be
> generated right at the time of typing. We automatically analyze the image
> and break into words and then show one word at a time for entry.
archive.org seems to have copies of books that are essentially scanned
pages. Does that help as one of the sources for obtaining data ?
--
sankarshan mukhopadhyay
<http://sankarshan.randomink.org/blog/>
M.N.S.Rao
Anyway I want to say that :
Selecting a data file for preparing a language traineddata file for indic
languages has become a challenging job as it is nearly impossible to include
all possible combintions of the characters by the very nature of the
structures of these languages. It is very necessary to take the assistance
from experts in the field of linguistics in this context. Some thought may
be given on this aspect.
MNS Rao
----- Original Message -----
From: "Dr. Atul Negi" <atul...@gmail.com>
To: "indic-ocr" <indi...@googlegroups.com>
Sent: Wednesday, April 20, 2011 8:04 PM
Subject: [Indic-OCR] Re: We need a data set to run tests on for accuracy of
our OCR
> Here are my views. I chose to comment with this post since it has
> several points that
> echo mine so I am just reinforcing what has been said.
>
> On Apr 20, 9:07 am, Abhaya Agarwal <abhaya.agar...@gmail.com> wrote:
>> We should collect the data of different types separately.
>>
>> 1. Recent Laser printed material on white paper makes for the easiest
>> data set. This can be generated in as much amount as required and
>> pretty
>
> I very much agree with this point, however the laser print data needs
> to be generated
> systematically so that the entire character set with all the
> variations of maatraas, combinations
> of consonants and sanyukt akshar etc etc is generated. Now this has to
> be done intelligently
> and not by brute force. We need to take the common sets of words.
> Therefore we need expert
> lingusitic knowledge to give this set of data.
>
> Therefore start by generating print outs for aksharmala with all
> barakhadi. Follow this up with words
> that are used in the KG and 1st class books that are used for a
> systematic introduction to the language.
> In telugu there is a book called as "pedda bala shikshaa" where the
> choice of words ensures an exposure
> to all the aksharaa mostly atleast and also simple well known words.
>
> We should or can use the 1st class or kg books themselves for the OCR
> training data.
> Or at least the books of upto middle schools may be used for the
> simpler vocabulary.
>
> Note for a full fledged test we need to take up the text in various
> fonts and styles like
> bold and italics.
>
>
>> 2. Scanned pages of recently offset printed books. - Hard to get
>> properly
>> licensed data
> Here I think it means we need to get those books which dont have
> copyright,
>
> Old books can satisfy the condition. They are out of copyright.
>
> Religious texts are out of copyright so they have been printed by many
> many printers
> in various fonts.
>
>
>> 3. scanned pages of old books: this will be hardest. Apart from the
>> age
>> factor, the quality of printing in Indian language books is usually
>> very bad
>> with uneven ink spread. If you look closely at one of the attachments
>> I sent
>> earlier, you will also find very uneven inter word spacing. Getting
>> the
>> images for various languages is not an issue here, we can find
>> sufficient
>> number of out of copyright books in the DLI database and collect them.
>
> you said it ! ink spread etc makes it real tough.
>
>> Apart from scanned pages, we will also need the ground truth for these
>> images. The simplest ground truth is the text version of the scanned
>> pages.
>
> What ground truth really requires is that not just the text but it
> needs a mapping
> stating that for the given rectangular area of the image, what is the
> associated text.
>
> There are several approaches for creating of ground truth data. The
> simplest is like in
> with XML tags for the text and giving the bounding box for the text
> content. Tags are needed
> for like <page> <page no.>, <header> <footer> <line>, <word>,
> <picture>, <non-ocr symbol>, etc etc.
> Punctuation is very important to capture and keep in the ground truth
> data.
> For example is there a dot at the end of the word or is it some
> noise ?
>
>> Although ideally, we would want word level mappings also. Character level
>> mappings would be cherry on the top. We would also need to decide on data
>> format for storing the word and character level mappings. Does Tesseract
>> already has something?
>
>
> character or akshara level mappings is quite difficult to create.
> Since I dont know anything
> how Tesseract does things more informed folks can give their view.
>
> I would like to point out that IL OCR needs almost a parallel project
> that generates a critical mass of
> test sets and ground truths without which things will not work.
> Ideally somebody needs to donate
> the effort to scan a book out of copyright and then to create the
> ground truth. It is a work of good magnitude.
>
>
>> So primarily 2 tasks:
>>
>> - If someone already has scanned pages + the digital version of the
>> same
>> : let us collect all that. Then we can build a small interface to
>> allow
>> anyone to come and align it.
>> - If someone only has the scanned pages, word level alignments can be
>> >> debaya...@gmail.com>
Dr. Atul Negi
several points that
echo mine so I am just reinforcing what has been said.
On Apr 20, 9:07 am, Abhaya Agarwal <abhaya.agar...@gmail.com> wrote:
>
to be generated
systematically so that the entire character set with all the
variations of maatraas, combinations
of consonants and sanyukt akshar etc etc is generated. Now this has to
be done intelligently
and not by brute force. We need to take the common sets of words.
Therefore we need expert
lingusitic knowledge to give this set of data.
Therefore start by generating print outs for aksharmala with all
barakhadi. Follow this up with words
that are used in the KG and 1st class books that are used for a
systematic introduction to the language.
In telugu there is a book called as "pedda bala shikshaa" where the
choice of words ensures an exposure
to all the aksharaa mostly atleast and also simple well known words.
We should or can use the 1st class or kg books themselves for the OCR
training data.
Or at least the books of upto middle schools may be used for the
simpler vocabulary.
Note for a full fledged test we need to take up the text in various
fonts and styles like
bold and italics.
> licensed data
copyright,
Old books can satisfy the condition. They are out of copyright.
Religious texts are out of copyright so they have been printed by many
many printers
in various fonts.
> with uneven ink spread. If you look closely at one of the attachments I sent
> earlier, you will also find very uneven inter word spacing. Getting the
> images for various languages is not an issue here, we can find sufficient
> number of out of copyright books in the DLI database and collect them.
> images. The simplest ground truth is the text version of the scanned pages.
needs a mapping
stating that for the given rectangular area of the image, what is the
associated text.
There are several approaches for creating of ground truth data. The
simplest is like in
with XML tags for the text and giving the bounding box for the text
content. Tags are needed
for like <page> <page no.>, <header> <footer> <line>, <word>,
<picture>, <non-ocr symbol>, etc etc.
Punctuation is very important to capture and keep in the ground truth
data.
For example is there a dot at the end of the word or is it some
noise ?
> mappings would be cherry on the top. We would also need to decide on data
> format for storing the word and character level mappings. Does Tesseract
> already has something?
Since I dont know anything
how Tesseract does things more informed folks can give their view.
I would like to point out that IL OCR needs almost a parallel project
that generates a critical mass of
test sets and ground truths without which things will not work.
Ideally somebody needs to donate
the effort to scan a book out of copyright and then to create the
ground truth. It is a work of good magnitude.
> So primarily 2 tasks:
>
> anyone to come and align it.
Debayan Banerjee
I have created separate repository for this purpose at https://github.com/debayan/Indic-OCR-Benchmark-Data . Here is how we are going to add data to it:
1) Person interested in contributing will first upload relevant pages to some 3rd party image hosting sites, like picasa or flickr, and pass a link to this list. If uploading to a 3rd party site is not possible, the interested person may adopt any other method to upload the image to a publicly accessible location which all list members can access.
2) On verifying the image's quality and suitability one of the people with commit access to this repository will commit the pertinent images to the right folder.
3) If a particular person is sending in useful images on a regular basis s/he will gain commit access to the repository.
All the images being donated should obviously belong to some Creative Commons or similar suitably open license.
Now the question is what kind of images do we need, and what should be the folder structure of the repository.
The top level of the directory will have folders belonging to different languages. Each language specific folder will have the following folders:
1) Laser Printed Images -> Will contain scanned laser printed images
2) Typeset Images -> Will contain scanned images of not-so-old books which have been typeset
3) Old Typeset Images -> Will contain scanned images of old pages with ink spread.
4) Parking Folder -> This will contain images that are good but do not have associated ground truth yet
For each folder above we will have the following sub-folders:
1) Noisy
2) Clean
The noisy folder may contain images of different kinds of noise.
Each image must be accompanied with corresponding metadata, or as is called ground truth. What format should we follow? I recommend the simple yet effective "box file" format.
A box file is nothing but a file containing each character per line and then a 4 tuple mentioning the co-ordinates on which that character appears in the corresponding image.
This is the format Tesseract uses, and there are several tools for automating your task of creating this data. Find out more about this at http://code.google.com/p/tesseract-ocr/wiki/TrainingTesseract3#Box_File_Editors
I invite comments and suggestions.
--
Debayan Banerjee
M.N.S.Rao
----- Original Message -----From: Debayan Banerjee
Debayan Banerjee
I have uploaded some kannada pages at https://github.com/mnsrao/Indic-OCR
I dont see anything there. It says "This repository's default branch is empty!" on the home page.
--
Debayan Banerjee
M.N.S.Rao
----- Original Message -----From: Debayan BanerjeeSent: Friday, April 22, 2011 2:52 AMSubject: Re: [Indic-OCR] Re: We need a data set to run tests on for accuracy of our OCR
Debayan Banerjee
Sorry; Some mistake.Material available at
I have to create a Windows Live ID to access these files. Please upload them to something like Picasa or Flickr instead.
--
Debayan Banerjee
M.N.S.Rao
----- Original Message -----From: Debayan BanerjeeSent: Friday, April 22, 2011 3:04 AMSubject: Re: [Indic-OCR] Re: We need a data set to run tests on for accuracy of our OCR
Debayan Banerjee
Text attached as picasa doesnot accept txt
Great! Thanks for your patience.
--
Debayan Banerjee
Abhaya Agarwal
archive.org seems to have copies of books that are essentially scanned
pages. Does that help as one of the sources for obtaining data ?