Thread by @j_bertolotti

#PhysicsFactlet (342) Lagrange multipliers
Strictly speaking Lagrange multipliers are not "Physics", but they are so useful to solve so many Physical problems, that it is definitively worth looking at them.
1/

Before we even introduce them, let's solve a super-simple problem, which will form the basis for our motivation to look into Lagrange multipliers:
Find the minimum of the function f=x²+y².

Yes, I can hear you shouting x=y=0, but let's still do the calculation.
2/

The way you find the minimum of a function is to check the points where all the partial derivatives are zero (in this case we have 2 variables, so we will look at the partial derivatives with respect to x and y): df/dx=2 x, df/dy=2y --> 2x=0, 2y=0 --> x=y=0.
3/

Obviously this doesn't tell you whether what we found is a minimum or a maximum. For that we need the second derivatives (spoiler: we found a minimum 🙃).

What if we had a constrained minimization problem? E.g. find the minimum of the function f=x²+y² such that x²+y=1.
4/

This particular problem is simple enough that we could just brute-force it (substitute y=1-x² into f=x²+y², differentiate with respect to x and equal it to zero). But it is useful to find a simpler way to solve this, so we can use it when the problem gets uglier.
5/

Let's make a "contour plot" of our surface, where the lines represent constant values of the function, and plot the constraint on top of it.
We are looking for the "smallest" contour line that just touches the constraint (if there is a smallest one, we are not on our minimum).
6/

At the point where the contour lines touch the constraint, the two curves are tangent to each other, i.e. their gradients are parallel. Calling λ the proportionality coefficient, we can write the relation between the gradients.
7/

Together with the constraint equation this gives us a system of 3 equations in 3 variables, which we can solve to find our solution(s).
8/

Why is this useful?
The point is that we just saw that asking what is the minimum of the function f, subject to the constraint g=c, is the same as writing the equality ∇ f = λ ∇ g, which is the same as ∇ (f +λ g)=0, which is the same as minimizing f +λ g.
9/

Where λ is called a "Lagrange multiplier".

This method is very powerful, as it allows us to transform a constrained minimization into an unconstrained minimization, which is usually much easier to solve.
10/

Let's look at a more complicated problem, which will highlight both the power and the limitations of this method. Say we have a point (x₁,y₁,z₁), what is the closest point on the torus [√(x² + y²) − 5]² + z² = 4?
11/

To avoid a lot of square roots, we will minimize the distance squared instead of the distance, so the function we want to minimize is
(x-x₁)²+(y-y₁)²+(z-z₁)²+λ [(√(x² + y²) − 5)² + z²].
12/

We have a functional to be minimized, we have a constraint, we put them together (with one Lagrange multiplier per constraint), and now we have a function to minimize without having to bother with the constraint anymore! Easy!
13/

Obviously the problem is that the system of equations we get when we make the partial derivatives with respect to x, y, and z (plus the constraint equation) is not linear, so unless we are very lucky we need to solve it numerically.
That said, the recipe works!
14/

And beside just working, the system you have to solve is simpler than what you would have obtained if you brute-force substituted the constraint inside the functional!
15/15