K-nearest neighbors (Knn)

The knn technique utilizes the concept of distance between data.
It calculates the distance between observations of the features and find the best estimate of the conditional probability, based on the neighborhood points.
To ilustrate this we first analyze the distance between the data, for samples of outcomes 2 and 7.

ind <- which(mnist$train$labels %in% c(2,7)) %>% sample(1000)
x <- mnist$train$images[ind,]
y <- mnist$train$labels[ind]

The distance between each row shows how each outcome is related to others.
The red squares demonstrate the digits that are the same.
The 7s are more uniform and close to each other than the 2s.

d <- dist(x)
options(repr.plot.width=10, repr.plot.height=7)
image(as.matrix(d)[order(y), order(y)], col = hcl.colors(12, "YlOrRd", rev = FALSE))

The distance between each column shows how each predictor is related to the others.
We can see, for a random predictor, the spatial pattern between pixels that are physically close.

d <- dist(t(x))
d <- as.matrix(d)[sample(1:784, 1),]
options(repr.plot.width=7, repr.plot.height=7)
image(1:28, 1:28, matrix(d, 28, 28))

Applying the knn technique, we start by applying k-nearest neighbor as 5.
Below are the model, confusion matrix and conditional probability.