Tensorflow.js for OCR

[Gaurav Srivastava]

Is it possible to use Tensorflow.js for  real-time OCR as I am willing to make client side native desktop application running in offline mode.? Motivation behind it is to avoid unnecessary network resource consumption and have higher level of security.Inspiration behind it is google speech to text recognition in android phones.I tried bundling Tesseract.js but its not real time and there is no much activity in respective forum for a longer.time.Any pointer in this regard would be a great help.Thanks in advance.

[Nikhil Thorat]

Yes it is possible!

The techniques that would be used in Keras / Tensorflow can also be used in TensorFlow.js.

Here is a Keras example that may be helpful for you: https://github.com/keras-team/keras/blob/master/examples/image_ocr.py

You may try to port this model using our Keras model converter: https://js.tensorflow.org/tutorials/import-keras.html

On Mon, May 14, 2018 at 2:31 PM, Gaurav Srivastava <srigau...@gmail.com> wrote:

Is it possible to use Tensorflow.js for  real-time OCR as I am willing to make client side native desktop application running in offline mode.? Motivation behind it is to avoid unnecessary network resource consumption and have higher level of security.Inspiration behind it is google speech to text recognition in android phones.I tried bundling Tesseract.js but its not real time and there is no much activity in respective forum for a longer.time.Any pointer in this regard would be a great help.Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+unsubscribe@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/f31cb834-76f7-44e0-bc6a-d262add9005e%40tensorflow.org.

[Shanqing Cai]

+1 to what Nikhil said.

TensoFlow.js is a generic machine learning / deep learning library. OCR is one of the sweet spots of ML/DL. In fact, several of tensorflow.js's official examples are about the classical MNIST digit recognition:

https://github.com/tensorflow/tfjs-examples/tree/master/mnist-core

https://github.com/tensorflow/tfjs-examples/tree/master/mnist-transfer-cnn

https://github.com/tensorflow/tfjs-examples/tree/master/mnist

If you are looking for more advanced OCR like those with language modeling, you can check out existing TensorFlow or Keras implementations and try converting the trained models to tensorflow.js for use in the browser:

https://github.com/pannous/tensorflow-ocr

[Gaurav Srivastava]

Hi Shanqing and Nikhil,

Sincere thanks for suggestions.I shall try out what Nikhil has shared, My major concern is "real time OCR" on non-GPU machines ( normal laptops)and I am interested in language models. Yes I already checked MNIST model for digit recognition but it wasn't my requirement.

Already I use tesseract.js for the word recognition on images created from  video[screen-sharing] captured using webrtc and drawn on canvas but recognition is considerably slow ~30 sec( after removing noise, adding contrast and upscaling image to 3x of screen size,still sometimes miss desired word) but this is not real time. Will Tensorflow.js  help me in making it in real time recognition ? As tesseract.js is quite old and no significant changes are done in recent years but Tensorflow.js is quite new( last month) and must be supporting latest version of required technologies. I am using electron 1.7, just adding for better picture.

Please let me know if i was unable to make clear picture of my problem.

Thanks All.

--

You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+unsubscribe@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/f55b72af-8794-4fdd-be13-771c16f3ac44%40tensorflow.org.

--

Gaurav

[Preyash Patel]

Hello Dear,

I am working on it and one part of Project is ready that is Single character recognition

here is the link to the project: https://preyash2047.github.io/textdetection.html

Note: Kindly use console for model loading status, if above link not working navigate through https://preyash2047.github.io

Now I am working for Document, converting a document into the computerised text and set text position as per original image's character position.  

once it is ready it will be hosted at https://preyash2047.github.io

Project repo available at https://github.com/preyash2047/preyash2047.github.io

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.

Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

[Preyash Patel]

Any suggestion regarding tensorflow.js for  converting a document into the computerised text and set text position as per original image's character position  

[Jason Mayes]

Thanks for sharing your project - excited to see how it evolves and copes with variety of fonts etc. Would be great to see this working on whole image and then making text selectable. Using approach similar to this: https://www.youtube.com/watch?v=cOC-ad0BsY0 to extract the characters in order and then place back in original position  etc.