Optimizing Solution Amplitude with Root and Initial Condition Constraints

Hello! Welcome to the third and final lesson of our first module.

In our previous lessons, we built a solid foundation for understanding solutions to homogeneous linear difference equations. We saw that any solution $x_t$ is a linear combination of terms derived from the characteristic roots ($z_i$), and we specifically investigated how repeated roots introduce polynomial terms ($t^k$), which can lead to amplitude growth even when the root's magnitude is fixed.

Today, we will synthesize these concepts to achieve our final learning outcome for this module:
Formulate the paper's optimization problem: maximizing the solution amplitude $|x_t|$ given constraints on the characteristic roots ($|z|\le r$) and initial conditions.

We will move from a qualitative understanding to a precise mathematical statement, defining the objective, the variables we can control, and the rules that constrain them. This formalization is the starting point for the entire analysis in the paper.

1. The Anatomy of the Optimization Problem

At its core, the paper addresses a "worst-case scenario" question. If you have a linear discrete-time system, but you don't know the exact system parameters, only that they lie within some family, what is the largest possible amplitude you can observe?

Any optimization problem can be broken down into three components:

1. Objective Function: The quantity we want to maximize or minimize.
2. Variables: The parameters we are allowed to change or choose.
3. Constraints: The rules that limit the values our variables can take.

Let's apply this framework to the problem described in the paper's abstract:

• Objective: Maximize the amplitude $|x_t|$ at a specific future time $t$.
• Variables: We have two sets of choices:
  1. The difference equation itself. An $n$-th order equation is defined by its characteristic polynomial, which is equivalent to choosing the set of its $n$ characteristic roots, $\{z_1,z_2,\dots ,z_n\}$.
  2. The specific solution to that equation. This is determined by the $n$ initial conditions, $(x_0,x_1,\dots ,x_{n-1})$.
• Constraints: Our choices are not unlimited:
  1. Root Constraint: The system must be r-stable, meaning all its characteristic roots must lie within the closed disk of radius $r$ in the complex plane. That is, $|z_i|\le r$ for all $i=1,\dots ,n$.
  2. Initial Condition Constraint: The initial state of the system is bounded. The paper's "vanilla" version assumes the initial values are in the unit disk, i.e., $|x_k|\le 1$ for all $k=0,\dots ,n-1$.

The difficulty of this problem, as the paper notes, lies in the fact that while $x_t$ depends linearly on the initial conditions, its dependence on the characteristic roots is highly non-linear.

2. Understanding the Constraints

r-Stability

The constraint on the characteristic roots, $|z_i|\le r$, is a generalization of the standard concept of stability for discrete-time systems.

You may recall from our previous lesson or from general systems theory that the long-term behavior of a solution is dictated by the magnitude of its roots. The Wikipedia article on linear recurrences provides a good summary of this.

The standard definition of asymptotic stability requires all roots to be inside the open unit disk ($|z_i|<1$). The paper generalizes this by introducing a radius $r$.

• If $r<1$, we are looking at a class of asymptotically stable systems.
• If $r=1$, we include systems that might have bounded or polynomially growing solutions (if roots are repeated on the boundary).
• If $r>1$, we are explicitly allowing for exponentially unstable systems.

The problem is to find the maximum amplitude irrespective of which stability regime we are in.

Bounded Initial Conditions

The constraint $|x_k|\le 1$ for $k=0,\dots ,n-1$ defines the set of allowed initial conditions as a unit polydisc in ${\mathbb{C}}^n$. This is a standard way to normalize the problem, essentially asking: "For a system with a bounded initial state or energy, what is the maximum amplification we can see at time $t$?"

3. The Formal Problem Statement

Now that we have the conceptual pieces, let's see how the paper formalizes this using precise notation. This is a crucial step in translating a research question into a solvable mathematical problem.

Let's dissect the notation from the paper to ensure it's perfectly clear. The authors build the problem in layers:

1. Maximum Amplitude for a Fixed Equation:
   $M_t(f|U):=\sup \{|x_t|:x\in X(f),(x_0,\dots ,x_{n-1})\in U\}$

   • Here, $f$ is a fixed characteristic polynomial.
   • $U$ is the set of allowed initial conditions (e.g., the unit polydisc).
   • This expression asks: For a given difference equation, what is the maximum amplitude $|x_t|$ we can get by choosing the best possible initial conditions from the set $U$?

2. Maximum Amplitude over a Class of Equations:
   $M_t(Z|U):={\sup }_{f\in P_n(Z)}{M}_t(f|U)$

   • This is the main quantity of interest. It wraps the previous definition in another supremum.
   • $Z$ is the set of allowed n-tuples of roots. In our case, it's the set of all $(z_1,\dots ,z_n)$ where each $|z_i|\le r$.
   • $P_n(Z)$ is the set of all characteristic polynomials whose roots are in $Z$.
   • This expression asks: What is the maximum possible amplitude $|x_t|$ if we are free to choose both the best equation from our allowed class and the best initial conditions for that equation?

3. Maximum Peak Amplitude:
   $M^{\ast }(Z|U):={\sup }_{t\ge n}{M}_t(Z|U)$

   • Finally, this takes the supremum not just over the equations and initial conditions, but also over all future time steps $t\ge n$. It asks for the highest peak the system can ever reach.

The paper's main goal is to find a closed-form expression for $M_t(Z|U)$.

Conclusion

In this lesson, we have formally defined the central optimization problem of the paper. This is the last piece of the puzzle for understanding the problem statement.

Key Takeaways:

• The problem is to maximize the solution amplitude $|x_t|$ of an $n$-th order linear difference equation at a given time $t$.
• This is a nested optimization problem involving a search over both the characteristic roots and the initial conditions.
• The search space is constrained: the characteristic roots must lie in a disk of radius $r$ ($|z_i|\le r$), and the initial values must lie in the unit disk ($|x_k|\le 1$).
• The paper's notation, particularly $M_t(Z|U)$, provides a precise mathematical formulation for this nested "worst-case" optimization.

Preview of the Next Module:

We have now fully defined the problem. The next logical step is to ask: how can we solve it? The search space—all polynomials with roots in a disk—is enormous and complex. In our next lesson, which begins Module 2, we will introduce a powerful tool from complex analysis, the Maximum Modulus Principle. This principle will allow us to drastically simplify the problem by proving that the optimal roots must lie on the boundary of the disk, i.e., where $|z_i|=r$. This is the first major step towards finding the solution.