Lagrange Multipliers for Constrained Optimization

Hello! Welcome to our next lesson.

In our last session, we formally derived the method of Lagrange multipliers, establishing the profound connection between a constrained optimum, tangent level sets, and parallel gradient vectors. We concluded that solving a constrained optimization problem is equivalent to finding the unconstrained critical points of the Lagrangian function, $L(\mathbf{x},\lambda )=f(\mathbf{x})-\lambda (g(\mathbf{x})-c)$.

Today, we transition from theory to practice. Our objective is to solve a simple equality-constrained optimization problem using Lagrange multipliers. We will walk through the systematic process of applying the method to concrete examples, translating abstract calculus into tangible solutions. This will solidify the powerful theoretical tools we developed in the last lesson.

The process we will follow is:

1. Formulate the problem: Identify the objective function $f(\mathbf{x})$ and the constraint $g(\mathbf{x})=c$.
2. Construct the Lagrangian: Define $L(\mathbf{x},\lambda )=f(\mathbf{x})-\lambda (g(\mathbf{x})-c)$.
3. Find the critical points: Solve the system of equations given by $\nabla L(\mathbf{x},\lambda )=0$.
4. Evaluate and interpret: Plug the candidate points back into the objective function $f(\mathbf{x})$ to identify the maximum and minimum values.

---

Part 1: A Complete Walkthrough of a 2D Problem

Let's begin with a classic two-dimensional problem that clearly illustrates every step of the process.

Problem: Find the maximum and minimum values of the function $f(x,y)=xy$ subject to the constraint that the points $(x,y)$ must lie on the unit circle $x^2+y^2=1$.

This is a slightly simplified version of the problem in the video we will watch, making the algebra more direct.

Step 1: Formulate the Problem

• Objective function: We want to optimize $f(x,y)=xy$.
• Constraint function: The points must satisfy $g(x,y)=x^2+y^2=1$.

Step 2: Construct the Lagrangian

Using our standard form, the Lagrangian is:
$L(x,y,\lambda )=f(x,y)-\lambda (g(x,y)-1)$
$L(x,y,\lambda )=xy-\lambda (x^2+y^2-1)$
Note: You may see other sources define the Lagrangian as $f+\lambda g$. This is also correct; it simply inverts the sign of the Lagrange multiplier $\lambda$. Consistency is key.

Step 3: Find the Critical Points by Solving $\nabla L=0$

We need to compute the partial derivatives of $L$ with respect to $x$, $y$, and $\lambda$ and set them all to zero.

1. $\frac{\partial L}{\partial x}=y-2\lambda x=0\Rightarrow y=2\lambda x$
2. $\frac{\partial L}{\partial y}=x-2\lambda y=0\Rightarrow x=2\lambda y$
3. $\frac{\partial L}{\partial \lambda }=-(x^2+y^2-1)=0\Rightarrow x^2+y^2=1$

We now have a system of three equations with three unknowns $(x,y,\lambda )$. Solving such systems is often the most challenging part of a Lagrange multiplier problem. A good strategy is to use the first two equations to find a relationship between $x$ and $y$.

Substitute the expression for $y$ from equation (1) into equation (2):
$x=2\lambda (2\lambda x)=4{\lambda }^{2}x$
Rearranging gives:
$x-4{\lambda }^{2}x=0\Rightarrow x(1-4{\lambda }^2)=0$
This equation implies that either $x=0$ or $1-4{\lambda }^2=0$. Let's analyze these two cases.

• Case A: $x=0$
  If $x=0$, equation (2) tells us $0=2\lambda y$. This means either $\lambda =0$ or $y=0$.
  However, if we plug $x=0$ and $y=0$ into the constraint (equation 3), we get $0^2+0^2\ne 1$. So, the point $(0,0)$ is not on the feasible set.
  Therefore, we must have $\lambda =0$, which from eq (1) means $y=0$. Again, $(0,0)$ is not a solution. Wait, let's re-trace. If $x=0$, from the constraint $x^2+y^2=1$, we must have $y^2=1$, so $y=\pm 1$. If $x=0$ and $y=\pm 1$, equation (1) gives $\pm 1-2\lambda (0)=\pm 1=0$, which is impossible.
  So, the case $x=0$ (and by symmetry, $y=0$) yields no solutions. We can conclude that $x\ne 0$ and $y\ne 0$.

• Case B: $1-4{\lambda }^2=0$
  This implies ${\lambda }^2=1/4$, so $\lambda =\pm 1/2$.

  • If $\lambda =1/2$, equation (1) gives $y=2(\frac{1}{2})x\Rightarrow y=x$.
    Plugging this into the constraint (3): $x^2+x^2=1\Rightarrow 2x^2=1\Rightarrow x=\pm \frac{1}{\sqrt{2}}$.
    Since $y=x$, our candidate points are: $(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$ and $(-\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$.
  • If $\lambda =-1/2$, equation (1) gives $y=2(-\frac{1}{2})x\Rightarrow y=-x$.
    Plugging this into the constraint (3): $x^2+(-x{)}^2=1\Rightarrow 2x^2=1\Rightarrow x=\pm \frac{1}{\sqrt{2}}$.
    Since $y=-x$, our candidate points are: $(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$ and $(-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$.

We have found four candidate points.

Step 4: Evaluate and Interpret

Finally, we evaluate our objective function $f(x,y)=xy$ at each of these four points.

• $f(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})=(\frac{1}{\sqrt{2}})(\frac{1}{\sqrt{2}})=1/2$
• $f(-\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})=(-\frac{1}{\sqrt{2}})(-\frac{1}{\sqrt{2}})=1/2$
• $f(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})=(\frac{1}{\sqrt{2}})(-\frac{1}{\sqrt{2}})=-1/2$
• $f(-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})=(-\frac{1}{\sqrt{2}})(\frac{1}{\sqrt{2}})=-1/2$

Comparing these values, we conclude:

• The maximum value is $1/2$, occurring at $(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$ and $(-\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$.
• The minimum value is $-1/2$, occurring at $(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$ and $(-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$.

The following video solves a very similar problem and provides excellent visual and algebraic intuition for the entire process.

Test your understanding!

Let's solve a simple word problem using the same method. You are given 180 yards of fencing to enclose a rectangular area. One side of the area is against a long building, so it does not need a fence. What are the dimensions of the field with the largest possible area?

Use the Lagrange multiplier method to find the optimal dimensions $x$ and $y$.

Show answer

1. Formulate:

   • Objective: $f(x,y)=xy$
   • Constraint: $g(x,y)=x+2y-180=0$

2. Lagrangian:
   $L(x,y,\lambda )=xy-\lambda (x+2y-180)$

3. Solve $\nabla L=0$:

   • $\frac{\partial L}{\partial x}=y-\lambda =0\Rightarrow y=\lambda$
   • $\frac{\partial L}{\partial y}=x-2\lambda =0\Rightarrow x=2\lambda$
   • $\frac{\partial L}{\partial \lambda }=-(x+2y-180)=0\Rightarrow x+2y=180$

   Now, substitute the expressions for $x$ and $y$ in terms of $\lambda$ into the constraint equation:
   $(2\lambda )+2(\lambda )=180$
   $4\lambda =180\Rightarrow \lambda =45$

4. Find dimensions:

   • $y=\lambda =45$ yards
   • $x=2\lambda =90$ yards

   The dimensions that maximize the area are 90 yards by 45 yards. The maximum area is $90\times 45=4050$ square yards.

---

Part 2: Solving a Problem in 3D

The method extends directly to higher dimensions. Let's tackle a classic problem in three variables, which also has a clear physical interpretation.

Problem: Find the dimensions of a rectangular box with the largest possible volume if its total surface area is fixed at 64 cm².

This is a quintessential application of Lagrange multipliers. I will guide you through the setup, and then you will study the detailed solution from a resource.

Step 1: Formulate the Problem

Let the dimensions of the box be $x,y,z$. We can assume $x,y,z>0$.

• Objective function: We want to maximize the volume, $V(x,y,z)=xyz$.
• Constraint function: The total surface area is given by the sum of the areas of the six faces: $2xy+2xz+2yz=64$. We can simplify this to $g(x,y,z)=xy+xz+yz=32$.

Step 2: Construct the Lagrangian

$L(x,y,z,\lambda )=xyz-\lambda (xy+xz+yz-32)$

Step 3: Set up the System of Equations

Taking the partial derivatives and setting them to zero gives:

1. $\frac{\partial L}{\partial x}=yz-\lambda (y+z)=0$
2. $\frac{\partial L}{\partial y}=xz-\lambda (x+z)=0$
3. $\frac{\partial L}{\partial z}=xy-\lambda (x+y)=0$
4. $\frac{\partial L}{\partial \lambda }=-(xy+xz+yz-32)=0$

This system is non-linear and looks more intimidating than our 2D example. However, its symmetry provides a path to a solution. A common technique is to multiply the first equation by $x$, the second by $y$, and the third by $z$, and then see what relationships emerge. The following resource provides a very clear, step-by-step walkthrough of this exact process.

The key result from solving this system is that $x=y=z$, meaning the box with the maximum volume for a fixed surface area is a cube.

---

Part 3: A Second Example Solved

To see another detailed application from start to finish, the MIT video from our last lesson contains an excellent example that is solved on the blackboard. Watching a problem being solved in real-time can be very helpful for understanding the flow of the algebraic steps.

Problem: Find the point on the hyperbola $xy=3$ that is closest to the origin.

This is equivalent to minimizing the squared distance $f(x,y)=x^2+y^2$ subject to the constraint $g(x,y)=xy=3$.

---

Conclusion

In this lesson, we have put the theory of Lagrange multipliers into practice. We successfully solved both 2D and 3D optimization problems with equality constraints by following a systematic, four-step process.

Key Takeaways:

• The method of Lagrange multipliers provides a robust algorithm: formulate the problem, construct the Lagrangian, solve $\nabla L=0$, and evaluate the candidate solutions.
• The most difficult step is often solving the resulting system of (usually non-linear) equations. Algebraic techniques and exploiting problem symmetry are crucial skills.
• The method finds candidate points for extrema. To classify them as minima, maxima, or saddle points, you must evaluate the objective function at these points. For problems on a closed, bounded domain, the existence of a max and min is guaranteed by the Extreme Value Theorem.

Preview of the Next Lesson:
Our journey into constrained optimization is not yet complete. Equality constraints are powerful, but many real-world problems involve inequalities (e.g., "budget must be less than or equal to $1000," or "variable must be non-negative"). In our next lesson, we will formulate an optimization problem with inequality constraints, which will set the stage for the more general Karush-Kuhn-Tucker (KKT) conditions.