Solving the Algebraic Riccati Equation for LQR

Hello! Welcome to the next lesson in your journey into multivariable control.

In our last two lessons, we've built the foundation for LQR control. First, we formulated the optimization problem with its quadratic cost function. Then, we explored the art of tuning the weighting matrices, $Q$ and $R$ , to shape the system's response. We've been treating commands like MATLAB's lqr as a "black box" that magically produces the optimal gain matrix $K$ .

Today, we're going to open that black box. This lesson reveals the mathematical engine at the heart of LQR: the Algebraic Riccati Equation (ARE). You will learn how solving this equation provides the key to directly calculating the optimal state feedback gain, $K$ . Understanding this process will demystify how LQR controllers are synthesized and give you a deeper appreciation for the tools you're using.

1. The LQR Solution: A Direct Formula for K

Let's start with the answer. For the infinite-horizon LQR problem we've defined, with the cost function:
$J = \int_{0}^{\infty} (x^{T} Q x + u^{T} R u) d t$
and system dynamics $x ˙ = A x + B u$ , some very clever mathematicians proved that the control law that minimizes $J$ is a simple state feedback controller:
$u (t) = - K x (t)$
The revolutionary part is that the optimal gain matrix, $K$ , can be calculated directly using a formula. It's not found by trial-and-error, but through a deterministic computation. The formula is:
$K = R^{- 1} B^{T} S$
This is a powerful result. It tells us that if we can find this special matrix, $S$ , we can immediately compute our optimal controller gain. So, the entire problem now boils down to one question: What is $S$ ?

2. The Algebraic Riccati Equation (ARE)

The matrix $S$ is the unique, symmetric, positive-definite solution to the Continuous-time Algebraic Riccati Equation (CARE), often just called the ARE. For a system defined by matrices $A, B, Q, R$ , the ARE is:

$A^{T} S + S A - S B R^{- 1} B^{T} S + Q = 0$

Let's break this down:

This is a matrix equation. The unknown we are solving for is the entire $n \times n$ matrix $S$ , where $n$ is the number of states.
It's non-linear. Notice the $S B R^{- 1} B^{T} S$ term, which is quadratic in $S$ . This is why it's more complex than the linear systems you're used to.
It's algebraic because it doesn't involve any derivatives of $S$ .

The workflow for designing an LQR controller is therefore not about guessing $K$ , but about solving this equation for $S$ and then using $S$ to find $K$ .

To get a bit of intuition about where this equation originates, the following video provides a nice conceptual derivation without getting lost in the full mathematical proof.

Why the Riccati Equation Is important for LQR Control

To understand where this equation comes from without delving into a deep mathematical proof, this video from the MATLAB channel offers an excellent conceptual derivation. It uses a technique known as 'completing the square' to cleverly rearrange the cost function.

Watch from 7:12 to 11:20. Focus on the main steps: 1) introducing the unknown matrix P (which we are calling S), 2) expanding the cost function, and 3) rearranging it into a form where two parts can be driven to zero. Don't worry about memorizing the matrix algebra; just follow the logic of how the final two key equations emerge.

As the video explains, by manipulating the cost function, we arrive at two conditions for optimality:

The matrix $S$ must satisfy the ARE, which makes one part of the cost function zero.
The control input $u$ must be chosen to make the other part of the cost function zero, which leads directly to the state feedback law $u = - K x$ , where $K$ is defined using $S$ .

3. Solving the ARE: A Step-by-Step Procedure

Since the ARE is a quadratic matrix equation, solving it isn't as simple as taking an inverse. It can have multiple solutions. However, for a controllable system, there is only one solution for $S$ that is positive-definite and results in a stable closed-loop system $A - B K$ . This is the "stabilizing solution" we are looking for.

The following video provides an outstanding walkthrough of the practical procedure for solving the ARE and finding the correct gain $K$ .

Introduction to Linear Quadratic Regulator (LQR) Control

Now that we know what the Algebraic Riccati Equation is, let's see how it's solved in practice. This video from Christopher Lum walks through a clear, five-step procedure for finding the optimal gain K, starting from the system model and the ARE.

Watch from 41:30 to 1:00:49. The video uses a simple mass-damper system as an example. Pay close attention to the procedure demonstrated in Mathematica: 1. Defining A, B, Q, R. 2. Setting up and solving the ARE for S, which yields multiple solutions. 3. Calculating the gain K for each solution. 4. Checking the eigenvalues (poles) of the closed-loop system for stability. 5. Selecting the single K that stabilizes the system. This process reveals exactly what happens inside a command like lqr.

To summarize the procedure shown in the video:

Define System and Costs: Start with your system matrices $A$ and $B$ , and your chosen weighting matrices $Q$ and $R$ .
Solve the ARE for S: Solve the equation $A^{T} S + S A - S B R^{- 1} B^{T} S + Q = 0$ . This will generally yield multiple solutions for the matrix $S$ .
Compute Potential Gains (K): For each solution $S_{i}$ , calculate a corresponding gain matrix $K_{i} = R^{- 1} B^{T} S_{i}$ .
Check Stability: For each gain $K_{i}$ , form the closed-loop system matrix $A_{c l, i} = A - B K_{i}$ . Calculate its eigenvalues (poles).
Select the Stabilizing Gain: The correct, optimal gain $K$ is the one that results in a stable closed-loop system, meaning all eigenvalues of $A_{c l}$ have negative real parts. There will be only one such gain.

4. From Manual Calculation to Software Tools

As you saw in the video, solving the ARE manually, even for a simple 2-state system, is quite tedious. This is where software tools become indispensable. A single command can execute that entire five-step process.

Let's look at the official MATLAB documentation for the lqr function to confirm this.

lqr - Linear-Quadratic Regulator (LQR) design - MATLAB

The MATLAB lqr command automates the entire process of solving the ARE and finding the stabilizing gain. This documentation page shows exactly what the function does.

First, review the 'Syntax' and 'Description' sections to see that lqr can return not just the gain K, but also the Riccati solution S and the closed-loop poles P. Then, scroll down to the 'Algorithms' section to see the exact continuous-time ARE that MATLAB solves. Finally, examine the example output under '[K,S,P] = lqr(sys,Q,R)' to see how K and S are returned for a sample system.

The syntax [K,S,P] = lqr(A,B,Q,R) perfectly maps to the procedure we just learned. The function:

Solves the ARE for the stabilizing solution S.
Calculates K using S.
Returns the resulting closed-loop poles P for you to inspect.

This automates the laborious and error-prone parts of the design, letting you focus on the high-level task of choosing $Q$ and $R$ .

Test your understanding!

Consider a simple system with:
$A = [0010], B = [01], Q = [1000], R = [1]$

Solving the ARE, $A^{T} S + S A - S B R^{- 1} B^{T} S + Q = 0$ , gives two symmetric solutions for $S$ :

$S_{1} = [2112], S_{2} = [- 2 1 1 - 2]$

Which one is the correct stabilizing solution? To find out:

Calculate the gain $K_{1}$ from $S_{1}$ and $K_{2}$ from $S_{2}$ .
Determine the eigenvalues of the closed-loop system $A - B K_{1}$ and $A - B K_{2}$ .
Identify the optimal gain $K$ .

Show answer

Calculate Gains: The formula is $K = R^{- 1} B^{T} S$ .
- For $S_{1}$ : $K_{1} = [1]^{- 1} [01] [2112] = [12] \approx [1 1.414]$
- For $S_{2}$ : $K_{2} = [1]^{- 1} [01] [- 2 1 1 - 2] = [1 - 2] \approx [1 - 1.414]$
Check Stability (Eigenvalues):
- For $K_{1}$ : $A - B K_{1} = [0010] - [01] [12] = [0 - 1 1 - 2]$ . The eigenvalues are approximately $- 0.707 \pm 0.707 j$ . Both have negative real parts, so this system is stable.
- For $K_{2}$ : $A - B K_{2} = [0010] - [01] [1 - 2] = [0 - 1 12]$ . The eigenvalues are approximately $+ 0.707 \pm 0.707 j$ . Both have positive real parts, so this system is unstable.
Identify Optimal Gain:
The stabilizing solution is $S_{1}$ . Therefore, the optimal LQR gain is $K = K_{1} = [12]$ .

Conclusion

You have now completed the core theory of LQR design. You've seen that the "magic" of the lqr command is rooted in solid mathematics.

Key Takeaways:

The optimal LQR control law is a state feedback law $u = - K x$ .
The gain $K$ is computed from a matrix $S$ using the formula $K = R^{- 1} B^{T} S$ .
The matrix $S$ is the unique stabilizing solution to the Algebraic Riccati Equation (ARE): $A^{T} S + S A - S B R^{- 1} B^{T} S + Q = 0$ .
Software tools automate the process of solving the ARE and selecting the correct solution, but understanding this underlying process is key to being an effective control designer.

Preview of the Next Lesson:
Our LQR controller is excellent at regulation—that is, driving the system's states to zero and keeping them there. However, for many applications, including your rotary inverted pendulum, you'll want the system to follow a non-zero command (e.g., move the cart to a new position). In the next lesson, we will build on our LQR controller to add reference tracking, enabling your system to follow setpoints and trajectories.

Recommended resources

1. The LQR Solution: A Direct Formula for K

2. The Algebraic Riccati Equation (ARE)

3. Solving the ARE: A Step-by-Step Procedure

4. From Manual Calculation to Software Tools

Conclusion