Skip to content
Last Dictionary
Plane
Machine LearningMathematics & Statistics

matrix factorization

nounAlso: matrix decomposition, latent factor model, Funk SVD, alternating least squares, ALS, non-negative matrix factorization, NMF

In plain wordsBreaking a big table of numbers, such as which users liked which films, into two smaller tables of hidden traits, so missing entries can be filled in by multiplying them back together.

  1. The approximation of a matrix by a product of lower-rank factor matrices. In recommender systemsrecommender system, n.: Software that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on., the sparse user–item rating or interaction matrix is approximated by the product of a user-factor matrix and an item-factor matrix, giving every user and item a vector of k latent factors (often 10–200); a predicted preference is the dot productdot product, n.: A score of how well two lists of numbers line up, found by multiplying matching entries and adding the products. It is large when the lists point the same way and zero when they are at right angles. of the two vectors, usually plus user and item bias terms. Unlike the classical singular value decompositionsingular value decomposition, n.: A way of rewriting any grid of numbers as three simple moves in turn — turn, stretch along perpendicular axes, turn again — which reveals the directions that matter most in the data., which needs a complete matrix, recommender factorization is fitted only to observed entries, with regularizationregularization, n.: Any technique that stops a model from memorizing its training examples too closely — often by penalizing complexity — so that it works better on new data., by stochastic gradient descentstochastic gradient descent, n.: A faster version of gradient descent that estimates which way is downhill from a small random sample of the training data at each step, instead of checking the whole dataset. or alternating least squares; weighted variants handle implicit feedbackimplicit feedback, n.: Clues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give.. The learned factors often line up with recognizable traits such as genre or audience. More broadly the term covers decompositions used throughout machine learning: SVD, principal component analysisprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores., non-negative matrix factorization (whose nonnegative factors yield additive, parts-based representations used in topic modeling and audio separation) and the low-rank approximationlow-rank approximation, n.: Replacing a big grid of numbers with a slimmer one that keeps most of the same information, the way a few pencil strokes can capture a face. It saves memory and computation. behind low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves..

    Matrix factorization with 64 latent factors beat the neighborhood model on held-out ratings.

min⁡P, Q∑(u,i)∈Ω(rui−pu⊤qi)2+λ(∥pu∥2+∥qi∥2)\min_{P,\,Q} \sum_{(u,i) \in \Omega} \big(r_{ui} - \mathbf{p}_u^{\top} \mathbf{q}_i\big)^2 + \lambda \big(\lVert \mathbf{p}_u \rVert^2 + \lVert \mathbf{q}_i \rVert^2\big)

Origin

Matrix decompositions are classical linear algebra. Their use for rating prediction took off during the Netflix Prize after Simon Funk (pseudonym of Brandyn Webb) described a simple incremental SVD-style method in a December 2006 blog post; Yehuda Koren, Robert Bell and Chris Volinsky, members of the winning team, summarized the approach in IEEE Computer in 2009. Daniel Lee and Sebastian Seung popularized non-negative matrix factorization in Nature in 1999.

Usage note

In recommender jargon 'SVD' often means Funk-style factorization of the observed entries only, which is not a true singular value decomposition.

Related

See also
  • collaborative filteringcollaborative filtering, n.: Recommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'.
  • recommender systemrecommender system, n.: Software that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on.
  • singular value decompositionsingular value decomposition, n.: A way of rewriting any grid of numbers as three simple moves in turn — turn, stretch along perpendicular axes, turn again — which reveals the directions that matter most in the data.
  • low-rank approximationlow-rank approximation, n.: Replacing a big grid of numbers with a slimmer one that keeps most of the same information, the way a few pencil strokes can capture a face. It saves memory and computation.
  • embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.
  • two-tower modeltwo-tower model, n.: A design for recommendation and search in which one neural network turns users or queries into lists of numbers and another does the same for items, so good matches can be found by quick comparison.
  • Netflix PrizeNetflix Prize, prop. n.: A 2006–2009 contest in which Netflix offered a million dollars to anyone who could predict customers' movie ratings 10 percent better than its own system. It popularized machine-learning competitions and model blending.

Sources

  1. Koren, Y., Bell, R. & Volinsky, C. (2009). Matrix Factorization Techniques for Recommender Systems. Computer 42(8).
  2. Lee, D. D. & Seung, H. S. (1999). Learning the Parts of Objects by Non-negative Matrix Factorization. Nature 401.
  3. Hu, Y., Koren, Y. & Volinsky, C. (2008). Collaborative Filtering for Implicit Feedback Datasets. Proc. 8th IEEE International Conference on Data Mining (ICDM).

Mentioned in

recommender systemrecommender system, n.: Software that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on., collaborative filteringcollaborative filtering, n.: Recommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'., imputationimputation, n.: Filling in missing values in a dataset with sensible estimates, such as the typical value or a prediction from the other columns, so the data can be analyzed or used for training., low-rank approximationlow-rank approximation, n.: Replacing a big grid of numbers with a slimmer one that keeps most of the same information, the way a few pencil strokes can capture a face. It saves memory and computation., topic modeltopic model, n.: A method that reads a large pile of documents and discovers the themes running through them, describing each theme by its typical words and each document by its mix of themes., two-tower modeltwo-tower model, n.: A design for recommendation and search in which one neural network turns users or queries into lists of numbers and another does the same for items, so good matches can be found by quick comparison., implicit feedbackimplicit feedback, n.: Clues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give., Netflix PrizeNetflix Prize, prop. n.: A 2006–2009 contest in which Netflix offered a million dollars to anyone who could predict customers' movie ratings 10 percent better than its own system. It popularized machine-learning competitions and model blending.