small language model
In plain wordsA language model built to be compact enough to run cheaply and quickly, even on a laptop or phone, trading some knowledge and skill for speed, privacy and lower cost.
A language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. with relatively few parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. — in common but informal usage, from a few hundred million up to roughly 10–15 billion — designed for low latencylatency, n.: How long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system., low inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text. or on-deviceon-device AI, n.: AI that runs directly on your own phone, laptop, car or headset rather than on a company's distant servers, so it can work offline, respond instantly and keep your data on the device. deployment. Small models narrow the gap with larger ones through heavily curated or synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars., training on far more tokens than compute-optimal recipes prescribe (overtrainingovertraining, n.: Deliberately training a model on much more data than would be most efficient for its training budget, so that a smaller model reaches good quality and is cheaper to run afterwards.), knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run. from larger teacher models, and quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.. They usually trail large modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. in world knowledge and complex multi-step reasoning, but are widely fine-tuned for narrow tasks, used as routers, classifiers and guard models, and deployed where data must stay local. In agent systems a small model often handles routine subtasks while a larger model plans.
A 3-billion-parameter small language model handles intent detection on the phone, so no audio leaves the device.
Origin
The contrast with large models appears in Timo Schick and Hinrich Schütze's 2020 paper 'It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners'. The abbreviation SLM spread in 2023–2024 with Microsoft's Phi series: phi-1 (1.3 billion parameters, 2023) and Phi-3 (2024), whose 3.8-billion-parameter 'mini' model was presented as small enough to run on a phone.
Usage note
Size labels drift: GPT-2's 1.5 billion parameters counted as large in 2019, but models several times bigger are marketed as small today. 'Small' is relative to the frontier, not an absolute class.
Related
- large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.
- knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.
- on-device AIon-device AI, n.: AI that runs directly on your own phone, laptop, car or headset rather than on a company's distant servers, so it can work offline, respond instantly and keep your data on the device.
- edge AIedge AI, n.: Doing AI's computing on or near the devices that collect the data, such as cameras, cars, factory machines, phones or local servers, instead of sending everything off to distant data centers.
- quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.
- model routingmodel routing, n.: Automatically choosing which AI model should answer each request, a cheap quick one for easy questions and a bigger slower one for hard ones, to save money without hurting quality.
- parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them.
- large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.
- frontier modelfrontier model, n.: One of the most capable AI models in existence at a given time, built by a handful of leading labs at great cost, and the main target of AI safety rules.
Sources
- Schick, T. & Schütze, H. (2021). It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners. NAACL 2021. arxiv.org/abs/2009.07118
- Gunasekar, S. et al. (2023). Textbooks Are All You Need. arXiv:2306.11644. arxiv.org/abs/2306.11644
- Abdin, M. et al. (2024). Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv:2404.14219. arxiv.org/abs/2404.14219
Mentioned in
frontier modelfrontier model, n.: One of the most capable AI models in existence at a given time, built by a handful of leading labs at great cost, and the main target of AI safety rules., knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run., large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., on-device AIon-device AI, n.: AI that runs directly on your own phone, laptop, car or headset rather than on a company's distant servers, so it can work offline, respond instantly and keep your data on the device., GemmaGemma, prop. n.: Google's family of free-to-download AI language models: smaller relatives of its Gemini systems, compact enough to run on a single computer and, in some versions, on a phone., Microsoft ResearchMicrosoft Research, prop. n.: Microsoft's research division, with laboratories on several continents, whose scientists have produced influential work on speech, vision and language AI., model routingmodel routing, n.: Automatically choosing which AI model should answer each request, a cheap quick one for easy questions and a bigger slower one for hard ones, to save money without hurting quality., overtrainingovertraining, n.: Deliberately training a model on much more data than would be most efficient for its training budget, so that a smaller model reaches good quality and is cheaper to run afterwards., parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them., GPU poorGPU poor, adj.: Slang for researchers, startups or countries without access to the huge numbers of expensive chips needed to build the most advanced AI, as opposed to the GPU-rich big labs., small AIsmall AI, n.: Cheap, compact AI tools built for one practical job — like advising farmers or screening patients — that poorer countries can afford to run, instead of giant all-purpose models., tiny recursive modeltiny recursive model, n.: A very small AI model that solves hard puzzles by going over its own answer again and again, improving it each time, instead of relying on sheer size.