Skip to content
Last Dictionary
Plane
Machine LearningSociety, Ethics & EconomyAI Engineering & MLOps
legal

recommender system

nounAlso: recommendation system, recommendation engine, recommender, recsys, recommendation algorithm

In plain wordsSoftware that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on.

  1. A system that estimates how relevant items are to a user and presents a ranked selection, using the user's past behavior, the items' attributes, context such as time and device, and the behavior of similar users. The main approaches are collaborative filteringcollaborative filtering, n.: Recommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'., which exploits patterns across many users' interactions; content-based filteringcontent-based filtering, n.: Recommending items that resemble what you already liked, judged by the items' own features, such as a film's genre, cast and plot, rather than by other people's choices., which matches item features to a user profile; and hybrids of the two. Industrial systems usually run as a multi-stage pipeline: candidate generation narrows millions of items to hundreds, often with a two-tower modeltwo-tower model, n.: A design for recommendation and search in which one neural network turns users or queries into lists of numbers and another does the same for items, so good matches can be found by quick comparison. and approximate nearest neighbor searchapproximate nearest neighbor search, n.: A family of shortcuts for finding the most similar items in a huge collection without checking every one — accepting an occasional miss in exchange for answers thousands of times faster.; a heavier ranking model scores the candidates; and a re-ranking stage applies business rules, diversity and freshness. Models learn mostly from implicit feedbackimplicit feedback, n.: Clues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give. such as clicks and watch time, are evaluated offline with ranking metrics such as recall@krecall@k, n.: A score for search systems: of all the right answers that exist, what fraction showed up in the top k results? Higher means fewer relevant items were missed. and NDCGnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1., and are then tested online with A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better.. Persistent problems include the cold start problemcold start problem, n.: The trouble a recommendation system has with brand-new users or items: with no history to learn from, it has little basis for good suggestions., feedback loops in which recommendations shape the very data they learn from, popularity bias, and choosing objectives that reflect what users actually value.

    The recommender system now blends predicted watch time with a diversity penalty in the final re-ranking stage.

Origin

Elaine Rich's Grundy (1979), which suggested library books from user stereotypes, is often cited as the first recommender. Xerox PARC's Tapestry (1992) introduced collaborative filtering, and Paul Resnick and Hal Varian popularized the term 'recommender systems' in a 1997 special issue of Communications of the ACM. Amazon's item-to-item recommendations and the 2006–2009 Netflix Prize brought the field wide attention.

Usage note

Among the most consequential deployed AI, recommenders shape what billions of people read, watch and buy. Critics link engagement optimizationengagement optimization, n.: Designing apps and AI to keep people clicking, watching or chatting for as long as possible because that earns money, even when it is not what is good for them. to filter bubblesfilter bubble, n.: A situation in which you see mostly news and opinions you already agree with, because apps choose what to show you based on what you clicked before. and harms to wellbeing. The EU Digital Services Act (2022) requires platforms to explain the main parameters of their recommender systems and very large platforms to offer an option not based on profiling.

Related

See also
  • collaborative filteringcollaborative filtering, n.: Recommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'.
  • content-based filteringcontent-based filtering, n.: Recommending items that resemble what you already liked, judged by the items' own features, such as a film's genre, cast and plot, rather than by other people's choices.
  • matrix factorizationmatrix factorization, n.: Breaking a big table of numbers, such as which users liked which films, into two smaller tables of hidden traits, so missing entries can be filled in by multiplying them back together.
  • two-tower modeltwo-tower model, n.: A design for recommendation and search in which one neural network turns users or queries into lists of numbers and another does the same for items, so good matches can be found by quick comparison.
  • cold start problemcold start problem, n.: The trouble a recommendation system has with brand-new users or items: with no history to learn from, it has little basis for good suggestions.
  • implicit feedbackimplicit feedback, n.: Clues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give.
  • learning to ranklearning to rank, n.: Training a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often.
  • filter bubblefilter bubble, n.: A situation in which you see mostly news and opinions you already agree with, because apps choose what to show you based on what you clicked before.

Sources

  1. Resnick, P. & Varian, H. R. (1997). Recommender Systems. Communications of the ACM 40(3).
  2. Rich, E. (1979). User Modeling via Stereotypes. Cognitive Science 3(4).
  3. Covington, P., Adams, J. & Sargin, E. (2016). Deep Neural Networks for YouTube Recommendations. Proc. 10th ACM Conference on Recommender Systems (RecSys).

Defined in law or standards

EU Digital Services Act (Reg. (EU) 2022/2065), Art. 3(s); obligations in Arts. 27 and 38

Mentioned in

embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance., misinformationmisinformation, n.: False or misleading information that spreads, whether or not the people sharing it know it is wrong. AI can produce it by mistake and makes it cheaper to create., association rule learningassociation rule learning, n.: Finding 'if this, then often that' patterns in large records of transactions, such as noticing that shoppers who buy pasta and tomatoes often buy parmesan too., ByteDanceByteDance, prop. n.: The Chinese company that owns TikTok, which also runs Doubao, China's most popular AI chatbot, and builds image and video generators., click-through rate predictionclick-through rate prediction, n.: Estimating how likely a particular person is to click on a particular ad, search result or recommended item: a prediction made billions of times a day to decide what each user is shown., cold start problemcold start problem, n.: The trouble a recommendation system has with brand-new users or items: with no history to learn from, it has little basis for good suggestions., collaborative filteringcollaborative filtering, n.: Recommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'., confounding variableconfounding variable, n.: A hidden third factor that influences two things at once and makes them look linked. Ice-cream sales and drownings both rise in hot weather; neither causes the other., content-based filteringcontent-based filtering, n.: Recommending items that resemble what you already liked, judged by the items' own features, such as a film's genre, cast and plot, rather than by other people's choices., engagement optimizationengagement optimization, n.: Designing apps and AI to keep people clicking, watching or chatting for as long as possible because that earns money, even when it is not what is good for them., feedback loopfeedback loop, n.: A cycle in which what a system produces comes back to shape its own future input, like a microphone howling near its loudspeaker. In AI, a model's predictions can shape the very data it later learns from., filter bubblefilter bubble, n.: A situation in which you see mostly news and opinions you already agree with, because apps choose what to show you based on what you clicked before., graph neural networkgraph neural network, n.: A neural network for data shaped like a web of connections, such as molecules, social networks or road maps, in which each point repeatedly gathers information from its neighbors., information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up., low-rank approximationlow-rank approximation, n.: Replacing a big grid of numbers with a slimmer one that keeps most of the same information, the way a few pencil strokes can capture a face. It saves memory and computation., matrix factorizationmatrix factorization, n.: Breaking a big table of numbers, such as which users liked which films, into two smaller tables of hidden traits, so missing entries can be filled in by multiplying them back together., multi-armed banditmulti-armed bandit, n.: A puzzle named after a gambler choosing between slot machines: you must keep picking among options whose payoffs are unknown, learning which is best while wasting as few tries as possible on the others., Section 230Section 230, prop. n.: A 1996 US law that generally protects websites and apps from being sued over what their users post, and lets them remove content in good faith. Whether it protects what AI chatbots themselves say is unsettled., surveillance capitalismsurveillance capitalism, n.: A business model in which companies collect huge amounts of data about people's behavior and use it to predict and influence what they will do, mainly to sell advertising., two-tower modeltwo-tower model, n.: A design for recommendation and search in which one neural network turns users or queries into lists of numbers and another does the same for items, so good matches can be found by quick comparison., algorithm filingalgorithm filing, n.: China's requirement that companies register certain algorithms — such as recommendation feeds and chatbots — with the internet regulator, describing what they do and how their risks are controlled., Digital Services ActDigital Services Act, prop. n.: The EU law that sets rules for online platforms and search engines — how they handle illegal content, explain their recommendation algorithms and manage the risks their services create for society., generative recommendationgenerative recommendation, n.: A newer approach to recommendation that uses language-model-style AI to predict the next item a person will want, one step at a time, much as a chatbot predicts the next word., implicit feedbackimplicit feedback, n.: Clues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give.