LQR Time-Domain Simulation and Performance Analysis

1. Simulating the Closed-Loop System

Once we have the optimal gain matrix $K$ from solving the ARE, we can define our control law as:
$u(t)=-Kx(t)$
This is a state feedback law. To simulate how the system behaves with this controller, we substitute the control law back into the original state equation:
$\dot{x}(t)=Ax(t)+Bu(t)$
$\dot{x}(t)=Ax(t)+B(-Kx(t))$
$\dot{x}(t)=(A-BK)x(t)$

This gives us the equation for the closed-loop system. We define the closed-loop state matrix as $A_{cl}=A-BK$. The system dynamics are now described by the simple linear equation:
$\dot{x}(t)=A_{cl}x(t)$

By solving for $K$ such that $A_{cl}$ is stable (which the LQR algorithm guarantees), we ensure that if the system is disturbed from its equilibrium point $x=0$, it will return to it. Simulating the response of this closed-loop system from a given initial condition $x(0)$ is how we verify the controller's performance.

2. The Art of the Trade-Off: Tuning Q and R

The real design work in LQR comes from choosing the weighting matrices $Q$ and $R$. These matrices define what "optimal" means for your specific problem. There is no single "correct" set of $Q$ and $R$; they are tuning knobs that you use to balance competing objectives.

The following video provides an excellent high-level explanation of this trade-off using a very intuitive analogy.

As the video demonstrates, the core LQR trade-off is:

• High Q / Low R: This corresponds to an aggressive controller. You are heavily penalizing state errors ($x^{T}Qx$) and placing little penalty on control effort ($u^{T}Ru$).
  • Expected Result: A fast response, with states returning to zero quickly. The cost is a large control signal $u(t)$, which might demand a powerful, expensive actuator or even exceed physical limits (saturation).
• Low Q / High R: This corresponds to a conservative controller. You are more tolerant of state errors but heavily penalize control effort.
  • Expected Result: A slower, more sluggish response. The benefit is a small control signal $u(t)$, which is "cheaper" in terms of energy, fuel, or actuator size.

This iterative process of adjusting $Q$ and $R$ and then simulating the response is the standard workflow for LQR design.

3. Practical Walkthrough: LQR for the Inverted Pendulum

Let's apply these concepts directly to the inverted pendulum, a system highly relevant to your project interests. We will see how tuning $Q$ and $R$ affects the stabilization of the pendulum.

First, watch this short segment that demonstrates the effect of changing the control weighting, R, in a MATLAB simulation of an inverted pendulum.

Step-by-Step Simulation and Analysis

Now, let's walk through a complete example of designing, simulating, and refining an LQR controller for an inverted pendulum on a cart. The following tutorial from the Control Tutorials for MATLAB and Simulink (CTMS) is a classic resource that does exactly this.

This walkthrough encapsulates the entire process:

1. Initial Design: Start with a simple choice for Q and R (e.g., based on the outputs or identity matrices).
2. Simulate: Run the simulation and analyze the time-domain response.
3. Analyze and Refine: Identify which aspects of the response are unsatisfactory (e.g., settling time of the cart, overshoot of the pendulum).
4. Tune: Adjust the diagonal elements of Q or R to address the specific performance issues. Increasing a Q element corresponding to a specific state tells the controller to prioritize regulating that state. Increasing R tells the controller to use less overall effort.
5. Iterate: Repeat the simulation and analysis until the performance meets your requirements.

Conclusion

In this lesson, you have moved from the theory of LQR to its practical application. You now understand how to simulate an LQR-controlled system and, more importantly, how to interpret the results to make informed design decisions.

Key Takeaways:

• The closed-loop system with LQR control is described by $\dot{x}=(A-BK)x$. Simulating this system shows how the controller regulates the states to zero.
• LQR design is an iterative process of tuning the weighting matrices $Q$ and $R$ to achieve a desired balance between performance and control effort.
• Increasing diagonal elements in $Q$ makes the controller prioritize stabilizing the corresponding states.
• Increasing elements in $R$ makes the controller more conservative, reducing the magnitude of the control input $u(t)$ at the expense of a slower response.

Preview of the Next Lesson:
Our LQR controller excels at regulation—driving states to zero. However, what if you want to move the cart to a new position (e.g., $x=0.5$ meters) and hold it there? The current controller, $u=-Kx$, will always try to drive $x$ back to 0. In the next lesson, we will introduce reference tracking, augmenting our controller to make the system follow non-zero setpoints.