Summation and Product Notation

The Language of Sums: Sigma Notation

Often in mathematics, we need to add up a long list of numbers that follow a specific pattern. For instance, imagine summing the first 100 square numbers: $1^2+2^2+3^2+\cdots +100^2$. Writing this out is cumbersome. Summation notation provides a concise and precise way to represent such sums.

The notation uses the capital Greek letter Sigma, $\Sigma$, to signify "sum." Let's break down its components.

Here's a breakdown of the parts:

• The Index of Summation (e.g., $n$ or $i,j,k$): A variable that takes on integer values.
• The Lower Limit: The starting integer value for the index.
• The Upper Limit: The final integer value for the index.
• The Expression (Summand): The formula that is evaluated for each value of the index.

The notation ${\sum }_{i=1}^5{i}^2$ instructs us to:

1. Start with the index $i=1$.
2. Evaluate the expression $i^2$, which gives $1^2$.
3. Increment the index to $i=2$ and evaluate $i^2$, giving $2^2$.
4. Continue this process until the index reaches the upper limit, $i=5$.
5. Sum all the results: $1^2+2^2+3^2+4^2+5^2=1+4+9+16+25=55$.

The following video provides a friendly introduction to the concept of a series and how sigma notation is used to represent it.

Properties of Summations

To "manipulate" series, we need to understand the algebraic rules they follow. These properties are direct consequences of the familiar laws of addition and multiplication (like associativity and distributivity). They allow us to break down complex sums into simpler, more manageable parts.

The main properties are:

1. Factoring out a constant: A constant multiplier inside a sum can be factored out.
   ${\sum }_{i=1}^{n}c\cdot a_i=c{\sum }_{i=1}^n{a}_i$
2. Splitting sums and differences: A sum of two expressions can be split into two separate sums.
   ${\sum }_{i=1}^n(a_i+b_i)={\sum }_{i=1}^n{a}_i+{\sum }_{i=1}^n{b}_i$
3. Summing a constant: The sum of a constant $c$ repeated $n$ times is simply $n\cdot c$.
   ${\sum }_{i=1}^{n}c=n\cdot c$

Since you appreciate formal rigor, it's worth noting that these properties can be formally proven using mathematical induction. The resource below provides such a proof for the third property.

A Toolbox of Standard Formulas

While the properties help us simplify the structure of a sum, they don't always help us compute the final value. For that, mathematicians have derived closed-form formulas for several common series. Knowing these is essential.

The following video introduces the most important formulas and, for the sum of the first $n$ integers, provides the elegant intuitive proof famously discovered by a young Carl Friedrich Gauss.

Here is a summary of the key formulas for sums starting at $i=1$:

• Sum of the first $n$ integers:
  ${\sum }_{i=1}^{n}i=\frac{n(n+1)}{2}$
• Sum of the first $n$ squares:
  ${\sum }_{i=1}^n{i}^2=\frac{n(n+1)(2n+1)}{6}$
• Sum of the first $n$ cubes:
  ${\sum }_{i=1}^n{i}^3={\left(\frac{n(n+1)}{2}\right)}^2$

Manipulating Complex Series

The real power of this framework comes when we combine the properties and formulas to solve complex sums without calculating each term. The strategy is to:

1. Use the properties to break the complex sum into simpler sums.
2. Factor out any constants.
3. Apply the standard formulas to each simple sum.
4. Combine the results.

Professor Leonard's video provides an excellent, detailed walkthrough of this exact process. He tackles a difficult-looking series and systematically simplifies it.

Beyond the Basics: Changing Indices and Telescoping Series

Two other powerful manipulation techniques are worth knowing.

1. Changing the Index of Summation
Sometimes a sum doesn't start at 1, which prevents us from using our standard formulas directly. We can adjust the limits by making a substitution. For example, to evaluate ${\sum }_{k=4}^{10}(k-1{)}^2$, we can let $i=k-1$.

• When $k=4$, $i=3$.
• When $k=10$, $i=9$.
  So the sum becomes ${\sum }_{i=3}^9{i}^2$. This is now in a form we can work with. We can calculate ${\sum }_{i=1}^9{i}^2$ and subtract the terms we don't want: ${\sum }_{i=1}^2{i}^2$.

2. Telescoping Series
This is a special type of series where most intermediate terms cancel each other out, leaving only the first few and last few terms.

The Language of Products: Pi Notation

Just as sigma notation represents repeated addition, Pi notation ($\Pi$) represents repeated multiplication. The structure is identical to sigma notation, with an index, limits, and an expression.

The most famous example of product notation is the factorial function:
$n!={\prod }_{i=1}^{n}i=1\cdot 2\cdot 3\cdot \cdots \cdot n$

A crucial distinction lies with empty operations. An empty sum, like ${\sum }_{i=1}^{0}i$, is defined as 0, the additive identity. An empty product, ${\prod }_{i=1}^{0}i$, is defined as 1, the multiplicative identity. This convention ensures that mathematical rules remain consistent even in these edge cases.

Conclusion

In this lesson, you've learned the essential notation for representing and working with series. These tools are not just for mathematical convenience; they are the building blocks for expressing complex operations in machine learning, statistics, and calculus.

Key Takeaways:

• Summation ($\Sigma$) and Product ($\Pi$) notations provide a compact way to express repeated addition and multiplication.
• They consist of an index, lower and upper limits, and an expression to be evaluated.
• Summations have properties of linearity that allow you to split them up and factor out constants.
• A "toolbox" of closed-form formulas exists for common series (sums of integers, squares, cubes, geometric series).
• By combining properties and formulas, you can manipulate and evaluate complex series systematically.
• An empty sum is 0 (the additive identity), while an empty product is 1 (the multiplicative identity).

In our next lesson, we will explore function composition and inverses. We will see how functions can be combined or "undone," which are concepts that underpin the architecture of neural networks and the logic of feature transformations in data science.