Tesseract fails to recognize digits, even with rescaling, char white_listing and filtering - tesseract

Question

For an open source pokerbot I'm trying to recognize images as implemented here. I have tried the following with an example image that I'd like tesseract to recognize:
pytesseract.image_to_string(img_orig)
Out[32]: 'cies TE'
pytesseract.image_to_string(img_mod, 'eng', config='--psm 6 --oem 1 -c tessedit_char_whitelist=0123456789.$£B')
Out[33]: ''
So then let's use some more sophisticated methods by scaling::
basewidth = 200
wpercent = (basewidth / float(img_orig.size[0]))
hsize = int((float(img_orig.size[1]) * float(wpercent)))
img_resized = img_orig.convert('L').resize((basewidth, hsize), Image.ANTIALIAS)
if binarize:
img_resized = binarize_array(img_resized, 200)
Now we end up with an image looking like this:
Let's see what comes out:
pytesseract.image_to_string(img_resized)
Out[34]: 'Stee'
pytesseract.image_to_string(img_resized, 'eng', config='--psm 6 --oem 1 -c tessedit_char_whitelist=0123456789.$£B')
Out[35]: ''
Ok, that didn't work. Let's try applying some filers:
img_min = img_resized.filter(ImageFilter.MinFilter)
img_mod = img_resized.filter(ImageFilter.ModeFilter)
img_med = img_resized.filter(ImageFilter.MedianFilter)
img_sharp = img_resized.filter(ImageFilter.SHARPEN)
pytesseract.image_to_string(img_min)
Out[36]: ''
pytesseract.image_to_string(img_mod)
Out[37]: 'oe Se'
pytesseract.image_to_string(img_med)
Out[38]: 'rete'
pytesseract.image_to_string(img_sharp)
Out[39]: 'ry'
Or maybe binarize will help?
numpy_array = np.array(image)
for i in range(len(numpy_array)):
for j in range(len(numpy_array[0])):
if numpy_array[i][j] > threshold:
numpy_array[i][j] = 255
else:
numpy_array[i][j] = 0
img_binarized = Image.fromarray(numpy_array)
pytesseract.image_to_string(img_binarized)
Out[42]: 'Sion'
pytesseract.image_to_string(img_binarized, 'eng', config='--psm 6 --oem 1 -c tessedit_char_whitelist=0123456789.$£B')
Out[44]: '0'
Again, all totally wrong.
What else can I do?
Any suggestions would be greatly appreciated.
Add on example:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
threshold_img = cv2.threshold(gray, 100, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
im_pil = cv2_to_pil(threshold_img)
pytesseract.image_to_string(im_pil)
Out[5]: 'TUM'
or trying another suggested algo for:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
threshold_img = cv2.threshold(gray, 100, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
im_pil = cv2_to_pil(threshold_img)
pytesseract.image_to_string(im_pil, 'eng', config='--psm 7')
Out[5]: '$1.99'

Answer 1

I think you are making it too complicated here. I did simple Otsu thresholding on the image that you've provided and was able to get the output.
image_path = r'path/to/image'
img = cv2.imread(image_path)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 100, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
cv2.imwrite('thresh.png', thresh)
detected_text = pytesseract.image_to_string(Image.open(image_path))
print(detected_text)
The image that I got after thresholding was
Tesseract was easily able to detect it and the output I got: $0.51

Answer 2

You can do it with or without Otsu. The trick for me was to invert the image so that it's black text on white background (which Tesseract seems to prefer).
EDIT One more trick for Tesseract is to add a border around the image. Tesseract does not like for the text to be too close to the edge.
import cv2
import pytesseract
import numpy as np
img = cv2.imread('one_twenty_nine.png', cv2.IMREAD_GRAYSCALE)
thresh = cv2.threshold(img, 100, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)[1]
cv2.imwrite('thresh.png', thresh)
detected_text = pytesseract.image_to_string(thresh, config = '--psm 7')
print(detected_text)
which gives
$1.29