Transformers
In plain wordsA free software library from Hugging Face that lets programmers download and use thousands of ready-trained AI models for text, images and sound with a few lines of code.
Hugging FaceHugging Face, prop. n.: A company whose website is the main public library for sharing AI models and datasets — often called the GitHub of machine learning — and which makes widely used free AI software.'s open-source Python library of model implementations, chiefly transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it. architectures, together with the code to fetch pretrained weights from the Hugging Face Hub and to run inferenceinference, n.: Drawing a conclusion from evidence. In today's AI it most often means running an already-trained model to get an answer, as opposed to training it. with them or fine-tunefine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. them. Auto classes such as AutoModel and AutoTokenizer load the right architecture and tokenizertokenizer, n.: The component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text. from a model identifier with from_pretrained; the high-level pipeline function wraps preprocessing, model call and postprocessing for common tasks; and a Trainer class handles training loops. It covers language, vision, speech and multimodal models — more than 400 architectures by late 2025 — and its model definitions serve as reference implementations on which serving engines such as vLLMvLLM, prop. n.: A free, open-source program for running large language models on servers efficiently, so one machine can answer many more users at once. Many companies use it to host AI models. and SGLangSGLang, prop. n.: A free, open-source engine for running AI language and image models on servers, built to be fast when many requests begin the same way, as in chat and agent workloads., training libraries and conversion tools rely. The library favors readable, self-contained model files over shared abstractions, a choice that eases experimentation at the cost of duplicated code. Version 5 (December 2025) made PyTorch its sole backend, dropping TensorFlow and Flax.
Load the checkpoint with Transformers, check a few generations, then hand it to vLLM for serving.
Origin
Began in November 2018 as pytorch-pretrained-bert, Thomas Wolf's PyTorch port of Google's BERTBERT, prop. n.: An influential 2018 Google language model that learned to understand text by filling in hidden words. It made search engines and many language tools much better at grasping meaning in context.; it was renamed pytorch-transformers in July 2019 and Transformers in September 2019, when TensorFlow support was added. Wolf and colleagues described the system in an EMNLP 2020 demonstration paper.
Usage note
Capitalized and plural, it names the library; lowercase 'transformer' names the neural-network architecture. Models that ship their own code load only with the option trust_remote_code, which runs that code locally and should be reserved for trusted sources.
Related
- Hugging FaceHugging Face, prop. n.: A company whose website is the main public library for sharing AI models and datasets — often called the GitHub of machine learning — and which makes widely used free AI software.
- transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
- PyTorchPyTorch, prop. n.: The most widely used free software toolkit for building and training neural networks, especially in research; programmers write ordinary Python and it handles the calculus and the graphics chips.
- safetensorssafetensors, n.: A file format for storing an AI model's learned numbers that cannot hide harmful programs and loads very quickly, replacing older formats that could run code when opened.
- tokenizertokenizer, n.: The component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text.
- fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
- transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
Sources
- Wolf, T. et al. (2020). Transformers: State-of-the-Art Natural Language Processing. Proceedings of EMNLP 2020: System Demonstrations.
- Hugging Face (2025). Transformers v5: Simple model definitions powering the AI ecosystem. Hugging Face blog, 1 December 2025. huggingface.co/blog/transformers-v5
Mentioned in
Hugging FaceHugging Face, prop. n.: A company whose website is the main public library for sharing AI models and datasets — often called the GitHub of machine learning — and which makes widely used free AI software., attention maskattention mask, n.: A grid of yes-or-no switches telling an attention layer which words each word may look at — for example, ignore blank padding, or do not peek at words that come later., causal language modelcausal language model, n.: A language model that reads text only from left to right, predicting each word from the words before it. The models behind today's chatbots are of this kind., machine learning pipelinemachine learning pipeline, n.: An automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way., PyTorchPyTorch, prop. n.: The most widely used free software toolkit for building and training neural networks, especially in research; programmers write ordinary Python and it handles the calculus and the graphics chips., chat templatechat template, n.: The exact text layout a chat AI was trained on for marking who is speaking — system, user or assistant. Software must reproduce it precisely, or the model performs noticeably worse.