fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(16, 5))
nn_plot(hist[15], 100, subplot=ax1, loss=True, title="Reference: full training set / hidden layer: 30 nodes")
nn_plot(hist[20], 100, subplot=ax2, loss=True, title="full training set / hidden layer: 100 nodes")

fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(16, 5))
nn_plot(hist[15], 100, subplot=ax1, acc=True, title="Reference: full training set / hidden layer: 30 nodes")
nn_plot(hist[20], 100, subplot=ax2, acc=True, title="full training set / hidden layer: 100 nodes")

By increasing from 30 to 100 nodes, the model has increased the number of parameters considerably, demanding more than double the time for execution. As results for ours metrics, the model improved a little and also didn't reach overfitting.

# Evaluate the model on the test data using 'evaluate'
print("Evaluate on test data")
results.append(model[-1].evaluate(x_test, y_test, batch_size=64))
print("test loss, test acc:", results[2])

Evaluate on test data
157/157 [==============================] - 1s 4ms/step - loss: 0.2191 - sparse_categorical_accuracy: 0.9628
test loss, test acc: [0.2190646231174469, 0.9628000259399414]

The accuracy over the test set improved from 95.29% to 96.28%.

Full training set: 2 hidden layers: 100 nodes each

Our next topology to analyze will be two hidden layers, with 100 nodes each, over the entire training set.

model.append(tf.keras.models.Sequential([
    tf.keras.layers.Dense(100, kernel_regularizer=tf.keras.regularizers.l2(0.01),
                          activation='relu', input_shape=(784,), name='hidden_1_layer'),
    tf.keras.layers.Dense(100, kernel_regularizer=tf.keras.regularizers.l2(0.01),
                          activation='relu', name='hidden_2_layer'),
    tf.keras.layers.Dense(10, activation='softmax', name='output_layer')
]))

nn_compile(model[-1], params[3])