Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Basin broadness depends on the size and number of orthogonal features, published by TheMcDouglas on August 27, 2022 on The AI Alignment Forum. TL;DR For neural networks trained to perfect loss, the broadness of optima in parameter space is given by the number and norm of independent orthogonal features the neural network has. The inner product that defines this "feature norm" and independence/orthogonality is the L2 product of Hilbert space. Introduction - why do we care about broadness? Recently, there's been some discussion of what determines the broadness of optima of neural networks in parameter space. People care about this because the size of the optimum basin may influence how easy it is for gradient descent (or other local optimisation algorithms which you might use to train a neural network) to find the corresponding solution, since it's probably easier to stumble across a broader optimum's attractor. People also care about this because there's some hypotheses floating around saying that the broadness of optima is correlated with their generality somehow. The broader the basin of your solution is in parameter space, the more likely it seems to be that the network will successfully generalise to new data in deployment. So understanding what kinds of circuit configurations are broad seems like it might help us grasp better how AIs learn things, and how an AGI might really work internally. Meaning, understanding what kind of solutions are broad is probably relevant to start developing general predictive theories of AI and AI training dynamics, which we'd like to have so we can answer questions like "What goals is my AGI going to develop, given that it has architecture A, and I'm training it with loss function L on data set D?" Measuring broadness Warning: calculus and linear algebra incoming! Say we have a loss function L(f,x,y) for a neural network f(x,Θ), with some data points x and labels y(x), and we've trained the network to perfect loss, setting the parameters to some set ^Θ. L=∑xl(f(x,Θ),y(x))dL(^Θ)dΘ=0 Now, lets try to look at how much the loss changes when we make small changes to the parameters ^Θ. To do this, we're going to perform a polynomial expansion of L up to quadratic order. This isn't going to describe the function behaviour perfectly, but within a small neighbourhood of ^Θ, it'll be mostly accurate. L(Θ)−L(^Θ)=dL(^Θ)dΘ(Θ−^Θ)+(Θ−^Θ)Td2L(^Θ)d2Θ(Θ−^Θ)+O((Θ−^Θ)3)=(Θ−^Θ)Td2L(^Θ)d2Θ(Θ−^Θ)+O((Θ−^Θ)3)=:δΘTd2L^Θ)d2ΘδΘ+O(δΘ3) The first order term vanished because dL(^Θ)dΘ=0 (since we're at an optimum). δΘ:=(Θ−^Θ) is a vector with #params entries, specifying how much we're perturbing each parameter in the network from its value at the optimum ^Θ. d2L(^Θ)d2Θ is going to be a matrix with (#params,#params) entries. So far so good. Now, if we wanted to work out how broad the basin is by looking at this matrix, how would we do it? The matrix is filled with second derivatives. Or in other words, curvatures. If you only had one parameter, this matrix would be a single number (the second derivative), telling you the curvature of the parabola you can fit to the optimum. And the curvature of a parabola determines its broadness, the closer the curvature is to zero, the broader the parabola, and with it the optimum, is. If you have two parameters, you can imagine the optimum as a little valley in the loss function landscape. The valley has two curvatures that characterise it now, but those don't need to line up with our coordinate axes x and y. You could, for example, have one "principal direction" of curvature lie in the (1,1) direction of parameter space, and one in the (1,−1) direction. To find these principal directions and their curvatures, you perform an eigendecomposition of the matrix. The two eigenvalues of the matrix will be the curvatures, and the two e...