[Python] Problem I Face while trying Number Recognition Using Tensorflow MNIST

In my OpenCV project, I'm aiming to read a Sudoku table, divide it into individual cells, and recognize the numbers. The rest is mostly Sudoku solving.

I'm stuck on recognizing the numbers part. I decided to use MNIST dataset to train the model. For a reason that I couldn't figure out, the program always recognizes the digit wrong. I'll explain further with my code below.

This is my code for the recognition part:

(ds_train, ds_test), ds_info = tfds.load(
'mnist',
split=['train', 'test'],
shuffle_files=True,
as_supervised=True,
with_info=True,
)


def normalize_img(image, label):
"""Normalizes images: `uint8` -> `float32`."""
return tf.cast(image, tf.float32) / 255., label


ds_train = ds_train.map(normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_train = ds_train.cache()
ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
ds_train = ds_train.batch(128)
ds_train = ds_train.prefetch(tf.data.AUTOTUNE)

ds_test = ds_test.map(normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_test = ds_test.batch(128)
ds_test = ds_test.cache()
ds_test = ds_test.prefetch(tf.data.AUTOTUNE)

model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28, 1)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(10)
])
model.compile(
optimizer=tf.keras.optimizers.legacy.Adam(0.001),
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
)

model.fit(
ds_train,
epochs=6,
validation_data=ds_test,
)
sample = cv.cvtColor(sample, cv.COLOR_BGR2GRAY)
sample = cv.resize(sample, (28, 28))

view_image(sample)
sample = np.invert(np.array([sample]))

prediction = model.predict(sample)
print(np.argmax(prediction))

sample = np.invert(np.array([sample])).reshape((28, 28, 1))
cv.imshow("test", sample)
cv.waitKey(0)


This is my image before the processing and after.

First form of my image:



My image after rescaling to 28x28:



The model predicts my image as 3 constantly. I don't know the reason behind this.

I tried to implement a solution in which I trained the model with a size of 300x300 and converted my image to 300x300 rather than 28x28, but the result didn't change.

I don't know if I implemented that solution wrong, but I hope that you can help me with this.

EDIT

size = 300
(ds_train, ds_test), ds_info = tfds.load(
'mnist',
split=['train', 'test'],
shuffle_files=True,
as_supervised=True,
with_info=True,
)


def resize_img(image, label):
return tf.image.resize(image, (size, size)), label


ds_train = ds_train.map(resize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_test = ds_test.map(resize_img, num_parallel_calls=tf.data.AUTOTUNE)


def normalize_img(image, label):
"""Normalizes images: `uint8` -> `float32`."""
return tf.cast(image, tf.float32) / 255., label


ds_train = ds_train.map(normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_train = ds_train.cache()
ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
ds_train = ds_train.batch(128)
ds_train = ds_train.prefetch(tf.data.AUTOTUNE)

ds_test = ds_test.map(normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_test = ds_test.batch(128)
ds_test = ds_test.cache()
ds_test = ds_test.prefetch(tf.data.AUTOTUNE)

model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(size, size, 1)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(10)
])
model.compile(
optimizer=tf.keras.optimizers.legacy.Adam(0.001),
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
)

model.fit(
ds_train,
epochs=6,
validation_data=ds_test,
)
sample = cv.cvtColor(sample, cv.COLOR_BGR2GRAY)
sample = cv.resize(sample, (size, size))

view_image(sample)
sample = np.invert(np.array([sample]))

prediction = model.predict(sample)
print(np.argmax(prediction))

for class_index, prob in enumerate(prediction[0]):
print(f'Class {class_index}: Probability {prob}')

sample = np.invert(np.array([sample])).reshape((size, size, 1))
cv.imshow("test", sample)
cv.waitKey(0)


This is the code that I wrote to process all of the images in 300x300 format. Of course, it's slower than the 28x28, but I thought that It would increase the accuracy. Unfortunately, it didn't change anything.

Continue reading...