Supervised learning: deep learning¶

In this practical, we will create a feed-forward neural network as well as a convolutional neural network to analyze the famous MNIST dataset.

Setup: installing TensorFlow¶

TensorFlow does not yet publish packages for very new Python versions (e.g. 3.13/3.14) — pip install tensorflow will fail on those. Use a dedicated conda environment pinned to Python 3.11 (any version in the 3.9–3.11 range works):

conda create -n daml python=3.11 -y
conda activate daml
pip install numpy matplotlib tensorflow
conda install jupyter -y
jupyter notebook
In [1]:
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

Let's set the seed value and use the same number as below to reproduce the same results.

In [2]:
tf.random.set_seed(45)
np.random.seed(45)

Data preparation¶

1. Load the built-in MNIST dataset by running the following code. Then, describe the structure and contents of the resulting objects.¶

In [3]:
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
In [4]:
print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)
print(x_train.min(), x_train.max())
print(np.unique(y_train, return_counts=True))

# x_train/x_test: (N, 28, 28) arrays of integers 0-255
# y_train/y_test: a single integer (0-9) label per image
(60000, 28, 28) (60000,)
(10000, 28, 28) (10000,)
0 255
(array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8), array([5923, 6742, 5958, 6131, 5842, 5421, 5918, 6265, 5851, 5949]))

2. Plotting is very important when working with image data. We have defined a convenient plotting function for you. Use the plot_img() function below to plot the first training image. The img parameter has to be a 2D array with shape (28, 28).¶

In [5]:
def plot_img(img, cmap="gray_r", **kwargs):
    plt.imshow(img, cmap=cmap, **kwargs)
    plt.axis("off")
    plt.show()
In [6]:
im = x_train[0]
plot_img(im)
No description has been provided for this image

It is usually a good idea to normalize your features to have a manageable, standard range before entering them in neural networks.

3. As a preprocessing step, ensure the brightness values of the images in the training and test set are in the range (0, 1).¶

In [7]:
x_train = x_train / 255
x_test = x_test / 255

Multi-layer perceptron: multinomial logistic regression¶

The simplest neural network model is a multi-layer perceptron where we have no hidden layers and only input and output layers. We can call this a multinomial logistic regression model, where we have no hidden layers and 10 outputs (0-1) for our mnist data. That model is shown below.

In [8]:
multinom = keras.Sequential([
    keras.Input(shape=(28, 28)),
    layers.Flatten(),
    layers.Dense(10, activation="softmax")
])

multinom.compile(
    loss="sparse_categorical_crossentropy",
    optimizer="adam",
    metrics=["accuracy"]
)

4. Display a summary of the multinomial model using .summary(). Describe why this model has 7850 parameters.¶

In [9]:
multinom.summary()
# parameters: 784*10 weights + 10 biases at output nodes = 7850
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten (Flatten)               │ (None, 784)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 10)             │         7,850 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 7,850 (30.66 KB)
 Trainable params: 7,850 (30.66 KB)
 Non-trainable params: 0 (0.00 B)

5. Train the model for 5 epochs using the code below. What accuracy do we obtain in the validation set?¶

In [10]:
multinom.fit(x_train, y_train, epochs=5, validation_split=0.2, verbose=2)
Epoch 1/5
1500/1500 - 1s - 552us/step - accuracy: 0.8684 - loss: 0.5047 - val_accuracy: 0.9128 - val_loss: 0.3174
Epoch 2/5
1500/1500 - 1s - 375us/step - accuracy: 0.9117 - loss: 0.3180 - val_accuracy: 0.9195 - val_loss: 0.2884
Epoch 3/5
1500/1500 - 1s - 534us/step - accuracy: 0.9185 - loss: 0.2940 - val_accuracy: 0.9225 - val_loss: 0.2779
Epoch 4/5
1500/1500 - 1s - 556us/step - accuracy: 0.9207 - loss: 0.2821 - val_accuracy: 0.9243 - val_loss: 0.2724
Epoch 5/5
1500/1500 - 1s - 525us/step - accuracy: 0.9231 - loss: 0.2745 - val_accuracy: 0.9253 - val_loss: 0.2692
Out[10]:
<keras.src.callbacks.history.History at 0x125c59cd0>

Answer: about 92.69% accuracy on the validation set.

6. Train the model for another 5 epochs. What accuracy do we obtain in the validation set?¶

In [11]:
multinom.fit(x_train, y_train, epochs=5, validation_split=0.2, verbose=2)
# about 92.98% accuracy on the validation set. Slightly improved but not much!
Epoch 1/5
1500/1500 - 1s - 479us/step - accuracy: 0.9247 - loss: 0.2690 - val_accuracy: 0.9263 - val_loss: 0.2672
Epoch 2/5
1500/1500 - 1s - 414us/step - accuracy: 0.9264 - loss: 0.2647 - val_accuracy: 0.9266 - val_loss: 0.2658
Epoch 3/5
1500/1500 - 1s - 670us/step - accuracy: 0.9274 - loss: 0.2613 - val_accuracy: 0.9271 - val_loss: 0.2650
Epoch 4/5
1500/1500 - 1s - 599us/step - accuracy: 0.9285 - loss: 0.2584 - val_accuracy: 0.9276 - val_loss: 0.2644
Epoch 5/5
1500/1500 - 1s - 580us/step - accuracy: 0.9292 - loss: 0.2560 - val_accuracy: 0.9282 - val_loss: 0.2641
Out[11]:
<keras.src.callbacks.history.History at 0x12624b750>

Deep feed-forward neural networks¶

7. Create and compile a feed-forward neural network with the following properties. Ensure that the model has 50890 parameters.¶

  • sequential model
  • flatten layer
  • dense layer with 64 hidden units and "relu" activation function
  • dense output layer with 10 units and softmax activation function
In [12]:
ffnn = keras.Sequential([
    keras.Input(shape=(28, 28)),
    layers.Flatten(),
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax")
])

ffnn.compile(
    loss="sparse_categorical_crossentropy",
    optimizer="adam",
    metrics=["accuracy"]
)
ffnn.summary()
Model: "sequential_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten_1 (Flatten)             │ (None, 784)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 64)             │        50,240 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 10)             │           650 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 50,890 (198.79 KB)
 Trainable params: 50,890 (198.79 KB)
 Non-trainable params: 0 (0.00 B)

8. Train the model for 10 epochs. What do you see in terms of validation accuracy, also compared to the multinomial model?¶

In [13]:
ffnn.fit(x_train, y_train, epochs=10, validation_split=0.2, verbose=2)
# higher accuracy! ~97.12%
Epoch 1/10
1500/1500 - 2s - 1ms/step - accuracy: 0.9078 - loss: 0.3295 - val_accuracy: 0.9486 - val_loss: 0.1809
Epoch 2/10
1500/1500 - 1s - 625us/step - accuracy: 0.9537 - loss: 0.1604 - val_accuracy: 0.9590 - val_loss: 0.1407
Epoch 3/10
1500/1500 - 1s - 591us/step - accuracy: 0.9652 - loss: 0.1182 - val_accuracy: 0.9631 - val_loss: 0.1239
Epoch 4/10
1500/1500 - 1s - 592us/step - accuracy: 0.9726 - loss: 0.0937 - val_accuracy: 0.9660 - val_loss: 0.1169
Epoch 5/10
1500/1500 - 1s - 538us/step - accuracy: 0.9775 - loss: 0.0770 - val_accuracy: 0.9668 - val_loss: 0.1136
Epoch 6/10
1500/1500 - 1s - 563us/step - accuracy: 0.9811 - loss: 0.0644 - val_accuracy: 0.9689 - val_loss: 0.1096
Epoch 7/10
1500/1500 - 1s - 549us/step - accuracy: 0.9844 - loss: 0.0545 - val_accuracy: 0.9699 - val_loss: 0.1071
Epoch 8/10
1500/1500 - 1s - 523us/step - accuracy: 0.9870 - loss: 0.0465 - val_accuracy: 0.9709 - val_loss: 0.1075
Epoch 9/10
1500/1500 - 1s - 532us/step - accuracy: 0.9895 - loss: 0.0393 - val_accuracy: 0.9707 - val_loss: 0.1085
Epoch 10/10
1500/1500 - 1s - 868us/step - accuracy: 0.9913 - loss: 0.0332 - val_accuracy: 0.9705 - val_loss: 0.1115
Out[13]:
<keras.src.callbacks.history.History at 0x12627b750>

9. Create predictions for the test data using the two trained models (using the function below). Create a confusion matrix and compute test accuracy for these two models.¶

In [14]:
def class_predict(model, x):
    return model.predict(x).argmax(axis=1)
In [15]:
pred_multinom = class_predict(multinom, x_test)
pred_ffnn = class_predict(ffnn, x_test)

ctab_multinom = tf.math.confusion_matrix(y_test, pred_multinom).numpy()
ctab_ffnn = tf.math.confusion_matrix(y_test, pred_ffnn).numpy()

print(ctab_multinom)
print(ctab_ffnn)
print(np.trace(ctab_multinom) / ctab_multinom.sum())
print(np.trace(ctab_ffnn) / ctab_ffnn.sum())
  1/313 ━━━━━━━━━━━━━━━━━━━━ 4s 13ms/step

262/313 ━━━━━━━━━━━━━━━━━━━━ 0s 192us/step

313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 237us/step
  1/313 ━━━━━━━━━━━━━━━━━━━━ 3s 13ms/step

234/313 ━━━━━━━━━━━━━━━━━━━━ 0s 215us/step

313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 258us/step
[[ 959    0    1    2    0    7    7    2    2    0]
 [   0 1116    3    2    0    1    4    2    7    0]
 [   4   11  925   14    6    3   12    8   46    3]
 [   3    1   21  907    0   25    3   10   33    7]
 [   1    2    8    1  908    0   10    4   11   37]
 [   8    3    3   25    6  778   16    5   41    7]
 [  13    3    9    1    6    7  915    1    3    0]
 [   1    8   22    8    5    1    0  943    3   37]
 [   5    9    6   14    7   17    9    7  893    7]
 [  10    7    1    8   22    6    0   16   12  927]]
[[ 967    0    1    1    1    3    3    2    2    0]
 [   0 1128    4    0    0    1    1    1    0    0]
 [   5    0 1003    2    2    0    3   10    7    0]
 [   0    0   10  983    0    4    0    3    8    2]
 [   3    0    4    1  957    1    4    1    1   10]
 [   3    0    0   14    1  852    9    0    8    5]
 [   5    3    1    1    1    4  943    0    0    0]
 [   1    6    8    4    0    0    2 1002    0    5]
 [   8    4    6    9    4    3    4    3  930    3]
 [   6    5    0    7   10    1    0   13    4  963]]
0.9271
0.9728

10. Create and estimate (10 epochs) a deep feed-forward model with the following properties. Compare this model to the previous models on the test data.¶

  • sequential model
  • flatten layer
  • dense layer with 128 hidden units and "relu" activation function
  • dense layer with 64 hidden units and "relu" activation function
  • dense output layer with 10 units and softmax activation function
In [16]:
dffnn = keras.Sequential([
    keras.Input(shape=(28, 28)),
    layers.Flatten(),
    layers.Dense(128, activation="relu"),
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax")
])

dffnn.compile(
    loss="sparse_categorical_crossentropy",
    optimizer="adam",
    metrics=["accuracy"]
)
dffnn.summary()
dffnn.fit(x_train, y_train, epochs=10, validation_split=0.2, verbose=2)
Model: "sequential_2"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten_2 (Flatten)             │ (None, 784)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_3 (Dense)                 │ (None, 128)            │       100,480 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_4 (Dense)                 │ (None, 64)             │         8,256 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_5 (Dense)                 │ (None, 10)             │           650 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 109,386 (427.29 KB)
 Trainable params: 109,386 (427.29 KB)
 Non-trainable params: 0 (0.00 B)
Epoch 1/10
1500/1500 - 2s - 1ms/step - accuracy: 0.9193 - loss: 0.2737 - val_accuracy: 0.9613 - val_loss: 0.1278
Epoch 2/10
1500/1500 - 1s - 784us/step - accuracy: 0.9664 - loss: 0.1122 - val_accuracy: 0.9660 - val_loss: 0.1083
Epoch 3/10
1500/1500 - 1s - 849us/step - accuracy: 0.9774 - loss: 0.0748 - val_accuracy: 0.9682 - val_loss: 0.1055
Epoch 4/10
1500/1500 - 1s - 887us/step - accuracy: 0.9837 - loss: 0.0532 - val_accuracy: 0.9703 - val_loss: 0.1108
Epoch 5/10
1500/1500 - 1s - 783us/step - accuracy: 0.9883 - loss: 0.0405 - val_accuracy: 0.9724 - val_loss: 0.1051
Epoch 6/10
1500/1500 - 1s - 858us/step - accuracy: 0.9901 - loss: 0.0329 - val_accuracy: 0.9753 - val_loss: 0.0990
Epoch 7/10
1500/1500 - 1s - 857us/step - accuracy: 0.9913 - loss: 0.0278 - val_accuracy: 0.9745 - val_loss: 0.1030
Epoch 8/10
1500/1500 - 1s - 821us/step - accuracy: 0.9927 - loss: 0.0228 - val_accuracy: 0.9735 - val_loss: 0.1118
Epoch 9/10
1500/1500 - 1s - 828us/step - accuracy: 0.9931 - loss: 0.0205 - val_accuracy: 0.9710 - val_loss: 0.1342
Epoch 10/10
1500/1500 - 1s - 845us/step - accuracy: 0.9938 - loss: 0.0184 - val_accuracy: 0.9737 - val_loss: 0.1241
Out[16]:
<keras.src.callbacks.history.History at 0x126660850>
In [17]:
pred_dffnn = class_predict(dffnn, x_test)
ctab_dffnn = tf.math.confusion_matrix(y_test, pred_dffnn).numpy()
print(ctab_dffnn)
print(np.trace(ctab_dffnn) / ctab_dffnn.sum())
# this one does even better!
  1/313 ━━━━━━━━━━━━━━━━━━━━ 4s 16ms/step

199/313 ━━━━━━━━━━━━━━━━━━━━ 0s 253us/step

313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 298us/step
[[ 972    1    1    1    1    0    2    1    1    0]
 [   1 1124    4    2    0    0    1    1    2    0]
 [   6    0 1005    1    3    0    2   10    5    0]
 [   1    0    4  981    1    3    0    4    4   12]
 [   3    0    3    0  955    0    1    3    1   16]
 [   2    1    0   10    3  860    7    1    4    4]
 [   6    2    1    1    5    2  941    0    0    0]
 [   1    2    9    0    1    0    0 1011    2    2]
 [   3    0    3    4    3    0    1    6  951    3]
 [   3    2    0    1   13    2    0   17    1  970]]
0.977

Convolutional neural network¶

For each example, they need a (width, height, channels) array (tensor). For a colour image with 28*28 dimension, that shape is usually (28, 28, 3), where the channels indicate red, green, and blue. MNIST has no colour info, but we still need the channel dimension to enter the data into a convolution layer with shape (28, 28, 1). The training dataset x_train should thus have shape (60000, 28, 28, 1).

11. Add a "channel" dimension to the training and test data using the following code. Plot an image using the first channel of the 314th training example (this is a 9).¶

In [18]:
x_train = x_train[..., np.newaxis]
x_test = x_test[..., np.newaxis]
In [19]:
plot_img(x_train[313, :, :, 0])
No description has been provided for this image

12. Create and compile a convolutional neural network using the following code. Describe the different layers in your own words.¶

In [20]:
cnn = keras.Sequential([
    keras.Input(shape=(28, 28, 1)),
    layers.Conv2D(filters=6, kernel_size=(5, 5)),
    layers.MaxPooling2D(pool_size=(4, 4)),
    layers.Flatten(),
    layers.Dense(units=32, activation="relu"),
    layers.Dense(10, activation="softmax")
])

cnn.compile(
    loss="sparse_categorical_crossentropy",
    optimizer="adam",
    metrics=["accuracy"]
)
In [21]:
cnn.summary()
# First, we have the input layer which gets the images and the first channel (28, 28, 1)
# then, there is a 2d convolution layer with 6 filters, and a kernel size of 5 (in each direction)
# then, we max-pool the resulting 6 maps to reduce their size by 4 in each direction
# afterwards, we flatten
# then comes a dense hidden layer with 32 units and a relu activation function
# lastly, the output layer is the same as before
Model: "sequential_3"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                 │ (None, 24, 24, 6)      │           156 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d (MaxPooling2D)    │ (None, 6, 6, 6)        │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_3 (Flatten)             │ (None, 216)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_6 (Dense)                 │ (None, 32)             │         6,944 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_7 (Dense)                 │ (None, 10)             │           330 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 7,430 (29.02 KB)
 Trainable params: 7,430 (29.02 KB)
 Non-trainable params: 0 (0.00 B)

13. Fit this model on the training data (10 epochs) and compare it to the previous models.¶

In [22]:
cnn.fit(x_train, y_train, epochs=10, validation_split=0.2, verbose=2)
Epoch 1/10
1500/1500 - 3s - 2ms/step - accuracy: 0.8961 - loss: 0.3548 - val_accuracy: 0.9600 - val_loss: 0.1352
Epoch 2/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9640 - loss: 0.1210 - val_accuracy: 0.9672 - val_loss: 0.1063
Epoch 3/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9714 - loss: 0.0930 - val_accuracy: 0.9714 - val_loss: 0.0930
Epoch 4/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9762 - loss: 0.0783 - val_accuracy: 0.9754 - val_loss: 0.0827
Epoch 5/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9789 - loss: 0.0683 - val_accuracy: 0.9767 - val_loss: 0.0769
Epoch 6/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9812 - loss: 0.0607 - val_accuracy: 0.9778 - val_loss: 0.0718
Epoch 7/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9833 - loss: 0.0546 - val_accuracy: 0.9790 - val_loss: 0.0680
Epoch 8/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9847 - loss: 0.0495 - val_accuracy: 0.9795 - val_loss: 0.0654
Epoch 9/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9864 - loss: 0.0452 - val_accuracy: 0.9802 - val_loss: 0.0630
Epoch 10/10
1500/1500 - 3s - 2ms/step - accuracy: 0.9877 - loss: 0.0415 - val_accuracy: 0.9817 - val_loss: 0.0616
Out[22]:
<keras.src.callbacks.history.History at 0x1270c0810>
In [23]:
pred_cnn = class_predict(cnn, x_test)
ctab_cnn = tf.math.confusion_matrix(y_test, pred_cnn).numpy()
print(ctab_cnn)
print(np.trace(ctab_cnn) / ctab_cnn.sum())
# about the same performance as the deep feed-forward model, with far fewer parameters!
  1/313 ━━━━━━━━━━━━━━━━━━━━ 6s 21ms/step

 78/313 ━━━━━━━━━━━━━━━━━━━━ 0s 656us/step

154/313 ━━━━━━━━━━━━━━━━━━━━ 0s 660us/step

231/313 ━━━━━━━━━━━━━━━━━━━━ 0s 659us/step

308/313 ━━━━━━━━━━━━━━━━━━━━ 0s 658us/step

313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 722us/step
[[ 977    0    1    0    0    0    0    1    1    0]
 [   1 1129    3    0    0    0    0    1    1    0]
 [   3    1 1014    3    1    0    0    4    6    0]
 [   1    0    1  998    0    4    0    2    3    1]
 [   1    2    0    0  970    0    0    2    2    5]
 [   2    0    1    6    1  870    4    1    5    2]
 [  14    4    0    0    7    3  925    0    5    0]
 [   2    3    8    2    1    0    0 1008    1    3]
 [   7    0    2    3    0    0    2    4  954    2]
 [   3    4    0    4   10    4    0    4    6  974]]
0.9819

14. Create another CNN which has better performance within 10 epochs.¶

Here are some things you could do:

  • Reduce the convolution filter size & the pooling size and add a second convolutional & pooling layer with double the number of filters
  • Add a dropout layer after the flatten layer
  • Look up on the internet what works well and implement it!
In [24]:
cnn_2 = keras.Sequential([
    keras.Input(shape=(28, 28, 1)),
    layers.Conv2D(filters=6, kernel_size=(3, 3)),
    layers.MaxPooling2D(pool_size=(2, 2)),
    layers.Conv2D(filters=12, kernel_size=(3, 3)),
    layers.MaxPooling2D(pool_size=(2, 2)),
    layers.Flatten(),
    layers.Dropout(rate=0.2),
    layers.Dense(units=32, activation="relu"),
    layers.Dense(10, activation="softmax")
])

cnn_2.compile(
    loss="sparse_categorical_crossentropy",
    optimizer="adam",
    metrics=["accuracy"]
)
cnn_2.fit(x_train, y_train, epochs=10, validation_split=0.2, verbose=2)
Epoch 1/10
1500/1500 - 5s - 3ms/step - accuracy: 0.8852 - loss: 0.3719 - val_accuracy: 0.9660 - val_loss: 0.1205
Epoch 2/10
1500/1500 - 5s - 3ms/step - accuracy: 0.9602 - loss: 0.1319 - val_accuracy: 0.9772 - val_loss: 0.0819
Epoch 3/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9687 - loss: 0.0972 - val_accuracy: 0.9790 - val_loss: 0.0724
Epoch 4/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9735 - loss: 0.0818 - val_accuracy: 0.9795 - val_loss: 0.0726
Epoch 5/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9761 - loss: 0.0737 - val_accuracy: 0.9818 - val_loss: 0.0623
Epoch 6/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9786 - loss: 0.0661 - val_accuracy: 0.9804 - val_loss: 0.0652
Epoch 7/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9811 - loss: 0.0597 - val_accuracy: 0.9847 - val_loss: 0.0535
Epoch 8/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9814 - loss: 0.0573 - val_accuracy: 0.9838 - val_loss: 0.0538
Epoch 9/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9833 - loss: 0.0511 - val_accuracy: 0.9852 - val_loss: 0.0511
Epoch 10/10
1500/1500 - 4s - 3ms/step - accuracy: 0.9839 - loss: 0.0487 - val_accuracy: 0.9855 - val_loss: 0.0515
Out[24]:
<keras.src.callbacks.history.History at 0x126683750>
In [25]:
pred_cnn_2 = class_predict(cnn_2, x_test)
ctab_cnn_2 = tf.math.confusion_matrix(y_test, pred_cnn_2).numpy()
print(ctab_cnn_2)
print(np.trace(ctab_cnn_2) / ctab_cnn_2.sum())
  1/313 ━━━━━━━━━━━━━━━━━━━━ 7s 24ms/step

 68/313 ━━━━━━━━━━━━━━━━━━━━ 0s 752us/step

135/313 ━━━━━━━━━━━━━━━━━━━━ 0s 751us/step

201/313 ━━━━━━━━━━━━━━━━━━━━ 0s 753us/step

267/313 ━━━━━━━━━━━━━━━━━━━━ 0s 756us/step

313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 816us/step

313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 833us/step
[[ 971    0    0    0    0    0    2    4    3    0]
 [   0 1133    1    1    0    0    0    0    0    0]
 [   1    4 1020    1    1    0    0    4    1    0]
 [   1    0    3  997    0    5    0    1    2    1]
 [   0    2    1    0  976    0    0    0    0    3]
 [   3    0    1    6    0  878    2    1    1    0]
 [   7    3    1    0    3    2  939    0    3    0]
 [   0    4    5    0    0    0    0 1016    0    3]
 [   3    0    3    0    0    0    3    3  957    5]
 [   3    4    1    2    6    1    0    5    0  987]]
0.9874