Advanced Machine Learning
02: Linear Algebra Recap
Outline for the lecture
- Linear Algebra from scratch
- Linear Algebra In Pictures
- A short demo
Linear Algebra
from scratch
Document-term table
Written as a matrix
Some operations on documents
Some operations on words
Movie preferences
Factoring a matrix
Factoring a matrix
Your turn!
factor this as a product of 2 vectors
Is this your solution?
How about this?
Matrix factorization problem
Additive features
- Features are non- negative and only add up
- Features are unknown: data comes as their combination
Multiplicative updates
- Setting the learning rates: \begin{align} \bm{\eta}_{\bm{H}} &= \frac{\bm{H}}{\bm{W}^T\bm{W}\bm{H}}\\ \bm{\eta}_{\bm{W}} &= \frac{\bm{W}}{\bm{W}^T\bm{H}\bm{H}^T}\\ \end{align}
- Results in updates: \begin{align*} \bm{H} &=& \bm{H}\odot \frac{\bm{W}^{T}\bm{X}} {\bm{W}^{T}\bm{W}\bm{H}}\\ \bm{W} &=& \bm{W}\odot \frac{\bm{X}\bm{H}^{T}} {\bm{W}\bm{H}\bm{H}^{T}} \end{align*}
Advantages:
- automatic non-negativity constraint satisfaction
- adaptive learning rate
- no parameter setting
the movie ranking
Rank 1 factorization
Rank 1 factorization: residuals
Rank 2 factorization
genres
digression: matrix multiplication
digression: matrix multiplication
digression: matrix multiplication
Genres
