Fine tuning with existing box/tiff pairs in Tesseract 4.0

[an-an-ko...@yandex.ru]

Hello everyone,

I'm experimenting with handwriting recognition using Tesseract 4.0. More concrete, I want to train Tesseract to recognize one particular Russian handwriting.

So, I wanted to add the "new font" (based on a bunch of tiff-images, which are a part of scanned archive, and box files) to already existing rus.traineddata using fine tuning.

I've prepared tiff/box pairs and then tried this script:

training/lstmtraining --model_output /.../rus_new/ --continue_from /.../rus.lstm --train_listfile /.../list_of_files.txt --eval_listfile /.../list_of_files.txt --max_iterations 5000

Where "list_of_files.txt' looked like:

/.../rus.Eskal_Font4You.exp0.tif
/.../rus.Eskal_Font4You.exp0.box

...and it ended up with this error:

First document cannot be empty!!
num_pages_per_doc_ > 0:Error:Assert failed:in file imagedata.cpp, line 655

What I am missing? 

Thanks in advance.

[bmwmine]

you are missing the .lstmf files 

[ShreeDevi Kumar]

When using pre-existing box tiff pairs, you have to add a box with tab character to mark end of line and also add boxes with spaces after every word. 

You then need to generate the .lstmf files - please see training/tesstrain.sh for details.

ShreeDevi
____________________________________________________________
भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

On Sat, May 6, 2017 at 4:40 PM, bmwmine <bmw...@gmail.com> wrote:

you are missing the .lstmf files 

--
You received this message because you are subscribed to the Google Groups "tesseract-ocr" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tesseract-ocr+unsubscribe@googlegroups.com.
To post to this group, send email to tesser...@googlegroups.com.
Visit this group at https://groups.google.com/group/tesseract-ocr.
To view this discussion on the web visit https://groups.google.com/d/msgid/tesseract-ocr/47875785-3322-4d5d-89fd-1818c2c06bc2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[an-an-ko...@yandex.ru]

Should I add boxes with spaces before punctuation marks?

Also I've found this discussion:

https://github.com/tesseract-ocr/tesseract/issues/841

It helped me a lot, but I still got questions.

What should I put in rus.training_text, if I want to generate .lstmf files from my own box/tiff pairs? Texts from images?

суббота, 6 мая 2017 г., 17:13:29 UTC+5 пользователь shree написал:

When using pre-existing box tiff pairs, you have to add a box with tab character to mark end of line and also add boxes with spaces after every word. 

You then need to generate the .lstmf files - please see training/tesstrain.sh for details.

ShreeDevi
____________________________________________________________
भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

On Sat, May 6, 2017 at 4:40 PM, bmwmine <bmw...@gmail.com> wrote:

you are missing the .lstmf files 

--
You received this message because you are subscribed to the Google Groups "tesseract-ocr" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tesseract-oc...@googlegroups.com.

To post to this group, send email to tesser...@googlegroups.com.
Visit this group at https://groups.google.com/group/tesseract-ocr.

[ShreeDevi Kumar]

On Sat, May 6, 2017 at 8:24 PM, <an-an-ko...@yandex.ru> wrote:

Should I add boxes with spaces before punctuation marks?

​If there is a space before the punctuation mark in text, then you should. ​

Also I've found this discussion:

https://github.com/tesseract-ocr/tesseract/issues/841

It helped me a lot, but I still got questions.

What should I put in rus.training_text, if I want to generate .lstmf files from my own box/tiff pairs? Texts from images?

​If you want to train solely based on your images, just use the default training text from langdata and one font, and use the modified shell script to train with copied box/tiff pairs and this file.

Then as described in the discussion at 841, build your training_files.txt file by using appropriate names for your box/tiff pairs only.

 eg. my copied box/tiff pairs were named as eng.1852nyd*.tif, while the generated file was eng.Arial.exp0.tif. By using the following command, I created the training_files,txt.

sudo ls -1 ~/tesstutorial/nydFT/eng.1852*.lstmf > ~/tesstutorial/nydFT/nyd.training_files.txt

​Possibly, you can directly edit that file and only keep the files you want on the list, but on my PC I haven't been able to get i to work - the edited file gives error, hence I use `ls` command to build it.

Experiment with a small file and adjust to get the output you want.

суббота, 6 мая 2017 г., 17:13:29 UTC+5 пользователь shree написал:

When using pre-existing box tiff pairs, you have to add a box with tab character to mark end of line and also add boxes with spaces after every word. 

You then need to generate the .lstmf files - please see training/tesstrain.sh for details.

ShreeDevi
____________________________________________________________
भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

On Sat, May 6, 2017 at 4:40 PM, bmwmine <bmw...@gmail.com> wrote:

you are missing the .lstmf files 

--
You received this message because you are subscribed to the Google Groups "tesseract-ocr" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tesseract-oc...@googlegroups.com.
To post to this group, send email to tesser...@googlegroups.com.
Visit this group at https://groups.google.com/group/tesseract-ocr.
To view this discussion on the web visit https://groups.google.com/d/msgid/tesseract-ocr/47875785-3322-4d5d-89fd-1818c2c06bc2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tesseract-ocr" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tesseract-ocr+unsubscribe@googlegroups.com.

To post to this group, send email to tesser...@googlegroups.com.
Visit this group at https://groups.google.com/group/tesseract-ocr.

To view this discussion on the web visit https://groups.google.com/d/msgid/tesseract-ocr/35ecde41-d654-408d-bd98-7de37fc6684a%40googlegroups.com.

[an-an-ko...@yandex.ru]

Thanks a lot for your help, everything worked, but now I have another problem.
I generated .lstmf files with boxtrain.sh from six box-tiff pairs and everything was alright.
I've added a few more pairs and now I get the "No block overlapping textline" error for almost every line in new box files.
They look exactly like the old ones, with tabs and everything.
What could go wrong?

Thanks in advance.