partial derivative

The rate at which a function of several variables changes as one input varies while all the others are frozen, written ∂f/∂xᵢ — an ordinary derivativederivative, n.: How fast something changes as one of its inputs changes — the steepness of a curve at a point. Training an AI means repeatedly following such slopes downhill. taken along a single coordinate direction. A model's lossloss function, n.: A score that measures how wrong a model's predictions are. Training means adjusting the model, step by step, to push this number down. depends on millions or billions of parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned., and each partial derivative says how the loss would change if that one parameter alone were nudged; the gradientgradient, n.: A list of slopes showing which way, and how steeply, a quantity rises as each of its inputs changes. Training nudges an AI's settings in the opposite direction, downhill. collects them into a vector, and the JacobianJacobian, n.: A table listing how each output of a process responds to a small change in each input. It describes how a transformation locally stretches, squeezes and turns space. arranges the partial derivatives of a vector-valued function into a matrix. Second-order and mixed partial derivatives form the HessianHessian, n.: A table of 'slopes of slopes' that describes how a surface curves: whether a point is the bottom of a bowl, the top of a hill or a mountain pass.. Backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction. computes every partial derivative of a scalar loss in a single backward sweep, at a cost comparable to one forward evaluation, by applying the chain rulechain rule, n.: A rule for working out how a change ripples through a chain of steps by combining the effect at each step. It is how AI training traces each error back through every layer. through the computational graphcomputational graph, n.: A map of a calculation drawn as boxes and arrows: each box is a simple operation and each arrow carries a number to the next step. AI software uses it to trace errors backward..

The partial derivative of the loss with respect to that bias is exactly zero, so it never moves.