Matrix Factorization

In previous analysis we verified that there is also strong effect of movie genre over the ratings. The residuals are obtained from the previous model:
$r_{u,i} = y_{u,i} - \hat{b}_{i} - \hat{b}_{u}$
We proceed now analyzing the data based on their rating to highlight this correlation.
It is important to note that we cannot use the rating as predictor, in case we want to fit a model, since the rating is our goal.
Our first step is to make some preprocessing on the data.

The residuals are calculated by removing movies (columns) and users (rows) averages.

The following graphic shows the correlation between two movies of the same title, in case, the movies Godfather and Godfater II. It shows that users that liked the Godfather more than the model prediction also liked the Godfather II more than the prediction.

We can show the correlation between The Godfather and Goodfellas, movies in the same genre, gangsters. They still have some correlation.

If we compare movies of another genre, like romantic comedies, you can also see a correlation between them.

Comparing these movies, the correlation between movies of the same genre are positive while movies from different genres show negative correlation, showing that users that like one group tend to not like the other group.

With matrix factorization we are able to find patterns in grouped data, like groups of movies or groups of users. This technique allows us to approach a structure, by applying factor analysis.

We define factors that model the entire residuals dataset into two vectors {p} and {q}:
$r_{u,i} \approx p_{u}q_{i}$
The new model includes the residuals explained by factorization:
$Y_{u,i} = \mu + b_{i} + b_{u} + p_{u}q_{i} + \varepsilon_{u,i}$

Singular value decomposition (SVD) and Principal Component Analysis (PCA)

The factors from the data can be estimated by applying single value decomposition (SVD) and principal component analysis (PCA).
Single value decomposition is a method that decomposes a matrix of data into principal components, that allows us to study the residuals variance. SVD estimates the vectors $p$ and $q$ by writing the residuals as $m$ rows and $n$ columns as:
$r_{u,i} = p_{u,1}q_{1,i} + p_{u,2}q_{2,i} + \cdots + p_{u,m}q_{m,i}$
With this it is possible to find important structure and variability in the data.

The function prcomp calculates the principal components, linear orthogonal transformation which maximizes the variability of a matrix. In this example we have 671 principal components.
The following graphic shows the variability of each principal component.

By plotting its cumulative sum, we can see that with just a few of principal components, we can explain a large percent of the data. With over 60 principal components we are able to explain 50% of the variability and with over 200 principal components we are able to explain about 80% of the variability out of a total of 671 principal components.

In order to investigate the importance of the first principal components, we can plot the first two, by labelling the points to their corresponding movies.
For this analysis we separate a small training set, only movies and users that have 50 or more ratings, and redo the fitting.

By investigating the first components, we can see that the first principal component shows the difference between critically acclaimed movies on one side and Hollywood blockbusters on the other side.

The second principal component captures a different structure in the data. It groups on one side artsy independent films and the nerd movies on the other side.