Tesseract doesn't recognize some numbers from an image

3,641 views
Skip to first unread message

javalover

unread,
Apr 7, 2022, 1:19:40 AM4/7/22
to tesseract-ocr
I'm deskewing an image containing a number using Projection profile based skew estimation algorithm and extracting it through OCR.

In order to calculate the correct skew angle, we compare the maximum difference between peaks and using this skew angle, thus rotate the image using the correct angle to correct the skew.

Each image (which has a single number) has prefixed background and foreground (number) color:

  • If the first pixel of the background is black, the foreground is dark gray.
  • If the first pixel of the background is white, the foreground is dark white.

Here are some sample images:

1.jpg 2.jpg 3.jpg 4.jpg

which all of them get successfully deskewed to these:

1.png 2.png 3.png 4.png

After it's been deskewed, I've tried with no luck to improve image quality to let OCR (PyTesseract) recognize the numbers.


import cv2
import numpy as np
import scipy.ndimage
import pytesseract
from PIL import Image, ImageEnhance, ImageFilter


from scipy.ndimage import interpolation as inter

def correct_skew(image, delta=6, limit=150):
    def determine_score(arr, angle):
        data = inter.rotate(arr, angle, reshape=False, order=0)
        histogram = np.sum(data, axis=1)
        score = np.sum((histogram[1:] - histogram[:-1]) ** 2)
        return histogram, score
 
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)                              
    blur = cv2.medianBlur(gray, 21)                                            
    thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]


    scores = []
    angles = np.arange(-limit, limit + delta, delta)
    for angle in angles:
        histogram, score = determine_score(thresh, angle)
        scores.append(score)

    best_angle = angles[scores.index(max(scores))] + 90

    (h, w) = image.shape[:2]
    center = (w // 2, h // 2)
    M = cv2.getRotationMatrix2D(center, best_angle, 1.0)
    rotated = cv2.warpAffine(image, M, (w, h), flags=cv2.INTER_CUBIC)


    ## Image processing to improve OCR accuracy
    """
    #rotated = cv2.medianBlur(rotated, 20)
   
    rotated = rotated.astype(np.float) / 255.

    # Calculate channel K:
    rotated = 1 - np.max(rotated, axis=2)

    # Convert back to uint 8:
    rotated = (255 * rotated).astype(np.uint8)

    binaryThresh = 190
    _, binaryImage = cv2.threshold(rotated, binaryThresh, 255, cv2.THRESH_BINARY)

    binaryThresh = 190
    _, binaryImage = cv2.threshold(rotated, binaryThresh, 255, cv2.THRESH_BINARY)

    # Use a little bit of morphology to clean the mask:
    # Set kernel (structuring element) size:
    kernelSize = 3
    # Set morph operation iterations:
    opIterations = 2
    # Get the structuring element:
    morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
    # Perform closing:
    rotated = cv2.morphologyEx(binaryImage, cv2.MORPH_CLOSE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
    """

    return best_angle, rotated

if __name__ == '__main__':
    image = cv2.imread('number.jpg')
    if image[0][0][0] > 128: image = cv2.bitwise_not(image)
    angle, rotated = correct_skew(image)
    print(angle)
    cv2.imshow('rotated', rotated)
    cv2.imwrite('rotated.png', rotated)
    pytesseract.pytesseract.tesseract_cmd = r'C:\\Program Files\\Tesseract-OCR\\tesseract.exe'

    text = pytesseract.image_to_string(rotated, config="-c tessedit_char_whitelist=0123456789")
    print("number:", text)
    cv2.waitKey()


This code (PyTesseract) recognizes the first and the second number, while not the others. Why?

Zdenko Podobny

unread,
Apr 7, 2022, 9:14:45 AM4/7/22
to tesser...@googlegroups.com
It does not matter how many times you post this (SO, issue tracker) - answer will be same: read and follow instruction in 

Zdenko


št 7. 4. 2022 o 7:19 javalover <gamerit...@gmail.com> napísal(a):
--
You received this message because you are subscribed to the Google Groups "tesseract-ocr" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tesseract-oc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tesseract-ocr/b6d80f87-893b-40de-8067-a0cb1b0865d2n%40googlegroups.com.

javalover

unread,
Apr 7, 2022, 6:56:56 PM4/7/22
to tesseract-ocr
@zdenop: it doesn't matter how much arrogant/ignorant you can be, you wouldn't have never understood that the reason I'm posting here is just because of this.
Thus, if you had actually read the page you linked me (Still having problems? paragraph), you would have come to the conclusion that all proposed solutions don't work:

If you've tried the above and are still getting low accuracy results, ask on the forum for help, ideally posting an example image.

that page suggested to ask here for further problems. And as you've already said, somebody has already suggested me to work on these image processing procedures before feeding it to Tesseract:

Zdenko Podobny

unread,
Apr 9, 2022, 3:40:21 AM4/9/22
to tesser...@googlegroups.com
Well  https://github.com/tesseract-ocr/tessdoc/blob/main/ImproveQuality.md


Rescaling -> Optimal resolution suggests that the size of input letters should be 30-33 points.  YOU IGNORED it - your has  240  points!!!
"for 4.x version use dark text on light background" ->  YOU IGNORED IT.
Border -> you did not remove them

Conclusion: you are ignoring documentation. Good luck with asking for support while blaming others for ignorance.

Zdenko


pi 8. 4. 2022 o 0:56 javalover <gamerit...@gmail.com> napísal(a):

javalover

unread,
Apr 10, 2022, 7:37:29 PM4/10/22
to tesseract-ocr

 @zdenop: the code I posted in the @OP doesn't have do the rescaled image and removed borders but I've already tried with no luck.
I excuse myself if I called you ignorant, but doesn't your sentence "It does not matter how many times you post this (SO, issue tracker)..." feel a little bit aggressive for no reason against me...? Even if I try to improve quality, it doesn't mean it has to work at all.
:-)
I acknowledge that you may have contributed to the Tesseract project, but could we please keep a non arrogant tone? I'm not familiar with Tesseract. It's the first time I'm using it, maybe I'm doing something wrong.
Did you actually find the solution to my problem (does it work for you by rescaling/removing borders)? If yes, I'd ask if you can post the code you used, because it didn't work for me.

Thanks.

Rodhad

unread,
Mar 24, 2023, 7:40:50 AM3/24/23
to tesseract-ocr
Hi, i'm actually trying to do the same thing that you've already tried on the same kind of images. Unfortunately with no results, i've even tried to train a neural network to recognize the numbers but even in that case with bad results.
I know that have been passed a bit from your last post, but i really would like to know if you have found a solution.

Thanks in advance.
Reply all
Reply to author
Forward
0 new messages