fully connected layer

A layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage. that applies an affine map y = Wx + b to its whole input vector, usually followed by an activation functionactivation function, n.: The simple rule each artificial neuron applies to its total input to decide its output, such as 'keep positive values, turn negatives into zero'. Without it, a deep network would be no smarter than a single layer.: every output unit is connected to every input unit by its own weight, so a layer from n inputs to m outputs has m·n weights and m biases. Stacked, such layers form a multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.; in CNNsconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. they traditionally formed the final classification head, and in transformerstransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it. they make up the attention projections and the feed-forward networkfeed-forward network, n.: The part of each transformer layer that works on every word separately, after attention has mixed in context. It holds most of a model's adjustable numbers and is thought to store much of its knowledge. sublayers, which together hold most of the parameters. PyTorch calls it Linear and Keras Dense. Because it imposes no structure on its input, it carries no built-in inductive biasinductive bias, n.: The assumptions built into a learning method that decide how it fills the gaps between the examples it was shown, such as that similar inputs get similar answers. Every learner needs some to cope with new cases. such as locality or weight sharingweight sharing, n.: Reusing the same learned numbers in many places in a network, for example one edge detector applied at every spot in an image, so the network needs fewer parameters and learns from less data.; in a convolutional network it is equivalent to a convolution whose kernel covers the whole input.

The classification head is a single fully connected layer from 768 features to 10 classes.