Differentiation Rules: Chain, Product, and Quotient

Hello! Welcome to your next lesson on Calculus.

In our last session, we learned the fundamental rules for finding derivatives of individual functions, like polynomials, exponentials, and trigonometric functions. We saw that the derivative tells us the instantaneous rate of change of a function.

However, most functions you'll encounter in data science aren't simple; they are combinations of other functions. For example, a cost function might involve a product of terms or one function nested inside another.

Today, we'll learn how to handle these combinations. Our learning outcome is to apply the chain rule, product rule, and quotient rule for differentiation. Mastering these three rules will allow you to differentiate almost any single-variable function, a critical skill for understanding how optimization algorithms work in machine learning.

1. The Product Rule: Differentiating a Multiplication

What happens when you need to differentiate a function that is the product of two other functions, like $f (x) = x^{2} sin (x)$ ? You might guess that you just multiply the derivatives, but it's a bit more involved.

To build a strong intuition for where the product rule comes from, we'll start with a brilliant visualization.

Visualizing the chain rule and product rule | Chapter 4, Essence of calculus

This video from the 3Blue1Brown channel provides an excellent visual derivation of the product rule by representing the product of two functions as the area of a rectangle whose sides are changing.

Watch the segment from 04:11 to 08:23. Focus on how a small change in 'x' affects the total area, and see how the two main parts of the new area correspond directly to the terms in the product rule.

As the video demonstrates, the rule for differentiating a product $f (x) \cdot g (x)$ is:

The Product Rule

\frac{d}{d x} [f (x) g (x)] = f^{'} (x) g (x) + f (x) g^{'} (x)

In words: "The derivative of the first function times the second, plus the first function times the derivative of the second." A common mnemonic is "Left d-Right, plus Right d-Left".

Let's see it in action with our example, $h (x) = x^{2} sin (x)$ .
Here, $f (x) = x^{2}$ and $g (x) = sin (x)$ .
From the previous lesson, we know:

$f^{'} (x) = 2 x$
$g^{'} (x) = cos (x)$

Applying the rule:
$h^{'} (x) = (2 x) (sin (x)) + (x^{2}) (cos (x)) = 2 x sin (x) + x^{2} cos (x)$

For more examples and a clear presentation of the formula, let's turn to a practical, problem-solving-focused video.

Differentiation | Class 11 | JEE | PACE SERIES

This video from Physics Wallah explains the product rule formula and demonstrates its application, including how to extend it to three or more functions.

Watch from 00:28:43 to 00:30:24. Pay attention to how the formula is applied in the examples.

2. The Quotient Rule: Differentiating a Division

Next, we need a rule for functions that are divided, like $h (x) = \frac{e ^{x}}{x ^{3}}$ . This is handled by the quotient rule.

The Quotient Rule

\frac{d}{d x} [\frac{f ( x )}{g ( x )}] = \frac{f ^{'} ( x ) g ( x ) - f ( x ) g ^{'} ( x )}{[ g ( x ) ] ^{2}}

This one can be tricky to memorize, but a popular mnemonic helps: "Low d-High, minus High d-Low, square the bottom and away we go!"

"Low" refers to the denominator, $g (x)$ .
"High" refers to the numerator, $f (x)$ .
"d" means "the derivative of".

So, the rule becomes: $\frac{( Low ) ( d-High ) - ( High ) ( d-Low )}{( Low ) ^{2}}$ .

Let's apply this to $h (x) = \frac{e ^{x}}{x ^{3}}$ .

High: $f (x) = e^{x} ⟹ f^{'} (x) = e^{x}$
Low: $g (x) = x^{3} ⟹ g^{'} (x) = 3 x^{2}$

Applying the rule:

h^{'} (x) = \frac{( x ^{3} ) ( e ^{x} ) - ( e ^{x} ) ( 3 x ^{2} )}{( x ^{3} ) ^{2}} = \frac{x ^{3} e ^{x} - 3 x ^{2} e ^{x}}{x ^{6}}

We can simplify this by factoring out $x^{2} e^{x}$ from the numerator:

h^{'} (x) = \frac{x ^{2} e ^{x} ( x - 3 )}{x ^{6}} = \frac{e ^{x} ( x - 3 )}{x ^{4}}

For a concise explanation and example, you can refer to the following resource.

Derivative Rules

This webpage from MathsIsFun provides a clear definition of the quotient rule, including the helpful mnemonic.

Read the section on the 'Quotient Rule'. Focus on the formula and the worked example for cos(x)/x.

Test your understanding!

Find the derivative of the function $k (x) = x^{4} cos (x)$ .

Show answer

This requires the product rule. Let $f (x) = x^{4}$ and $g (x) = cos (x)$ .
Then $f^{'} (x) = 4 x^{3}$ and $g^{'} (x) = - sin (x)$ .

Applying the product rule:
$k^{'} (x) = f^{'} (x) g (x) + f (x) g^{'} (x)$
$k^{'} (x) = (4 x^{3}) (cos (x)) + (x^{4}) (- sin (x))$
$k^{'} (x) = 4 x^{3} cos (x) - x^{4} sin (x)$

3. The Chain Rule: Differentiating Nested Functions

The chain rule is arguably the most important differentiation rule. It allows us to differentiate composite functions, where one function is "nested" inside another, like $h (x) = sin (x^{2})$ .

This is not a product; it's the function $x^{2}$ plugged into the function $sin (\cdot)$ .

The intuition behind the chain rule is about a domino effect: a small change in $x$ causes a change in the inner function, which in turn causes a change in the outer function. The chain rule tells us how to multiply these rates of change.

Visualizing the chain rule and product rule | Chapter 4, Essence of calculus

Let's return to the 3Blue1Brown video for an intuitive visualization of the chain rule. It uses a set of number lines to show how a change in the input variable propagates through the nested functions.

Watch the segment from 08:35 to 14:26. This is the most important concept in this lesson, so pay close attention to how the changes dx, dh, and dg relate to each other.

The video leads us to the two common forms of the rule.

The Chain Rule

If $h (x) = f (g (x))$ , then:
$h^{'} (x) = f^{'} (g (x)) \cdot g^{'} (x)$
In words: "The derivative of the outer function (with the inner function left unchanged), multiplied by the derivative of the inner function."
Using Leibniz notation: If $y = f (u)$ and $u = g (x)$ , then:
$\frac{d y}{d x} = \frac{d y}{d u} \cdot \frac{d u}{d x}$
This form is very intuitive, as it looks like the $d u$ terms "cancel out".

Let's see this visually with a step-by-step example.

Chain Rule Example — This image breaks down the chain rule for the function $y = (x^2+1)^3$. It clearly identifies the inner function `u` and outer function `y`, computes their derivatives separately, and combines them according to the rule.

Now let's use this rule for our earlier example, $h (x) = sin (x^{2})$ .

Outer function: $f (u) = sin (u)$
Inner function: $g (x) = x^{2}$

Their derivatives are:

$f^{'} (u) = cos (u)$
$g^{'} (x) = 2 x$

Applying the rule $h^{'} (x) = f^{'} (g (x)) \cdot g^{'} (x)$ :
$h^{'} (x) = cos (x^{2}) \cdot (2 x) = 2 x cos (x^{2})$

This is a powerful rule that takes practice. The video below provides more worked examples.

Differentiation | Class 11 | JEE | PACE SERIES

The Physics Wallah video also gives a clear formulaic explanation of the chain rule and walks through some typical problems you might see on an exam.

Watch the segments from 00:31:37 to 00:34:51 and from 00:34:51 to 00:43:56. The first part explains the rule, and the second shows many examples combining all the rules we've learned.

4. Combining the Rules

Fluency in differentiation comes from applying these rules together to solve complex problems.

Example: Find the derivative of $y = \frac{x ^{2}}{e ^{5 x}}$ .

We start with the Quotient Rule because the overall structure is a division.
- High: $f (x) = x^{2} ⟹ f^{'} (x) = 2 x$
- Low: $g (x) = e^{5 x}$
To find $g^{'} (x)$ , we need the Chain Rule.
- Outer function: $h (u) = e^{u} ⟹ h^{'} (u) = e^{u}$
- Inner function: $k (x) = 5 x ⟹ k^{'} (x) = 5$
- So, $g^{'} (x) = h^{'} (k (x)) \cdot k^{'} (x) = e^{5 x} \cdot 5 = 5 e^{5 x}$ .
Now, we plug everything back into the Quotient Rule:
$y^{'} = \frac{( 2 x ) ( e ^{5 x} ) - ( x ^{2} ) ( 5 e ^{5 x} )}{( e ^{5 x} ) ^{2}}$
Simplify the expression:
$y^{'} = \frac{e ^{5 x} ( 2 x - 5 x ^{2} )}{e ^{10 x}} = \frac{2 x - 5 x ^{2}}{e ^{5 x}}$

Test your understanding!

Find the derivative of the function $f (x) = (3 x^{2} + 7)^{4}$ . Which rule is the main rule you need to apply?

Show answer

The main rule is the Chain Rule, because the structure is one function ( $3 x^{2} + 7$ ) inside another ( $(\cdot)^{4}$ ).

Outer function: $g (u) = u^{4} ⟹ g^{'} (u) = 4 u^{3}$
Inner function: $h (x) = 3 x^{2} + 7 ⟹ h^{'} (x) = 6 x$

Applying the chain rule:
$f^{'} (x) = g^{'} (h (x)) \cdot h^{'} (x)$
$f^{'} (x) = 4 (3 x^{2} + 7)^{3} \cdot (6 x)$
$f^{'} (x) = 24 x (3 x^{2} + 7)^{3}$

5. Application with Python

As you know, you can use programming to verify your calculus and handle very complex derivatives. The sympy library in Python is designed for symbolic mathematics. It can compute derivatives automatically, which is a great way to check your work.

Calculus for Data Science and Machine Learning

This article from Medium, 'Calculus for Data Science and Machine Learning', not only lists the derivative rules but also shows a practical Python example using the SymPy library.

In this resource, focus on the section '2. Differentiation'. Review the formulas for the Product, Quotient, and Chain rules. Then, carefully examine the Python code block. See how sp.Symbol('x') creates a mathematical variable and sp.diff(f, x) computes the derivative of the function f with respect to x.

Using tools like sympy reinforces your understanding and bridges the gap between theoretical math and practical coding, aligning with your goal of mastering data science.

Conclusion

Fantastic work! You have now learned the three most powerful rules of differentiation. With these, you can break down and solve almost any derivative problem for single-variable functions.

Key Takeaways:

Product Rule: Use for functions multiplied together. $(f g)^{'} = f^{'} g + f g^{'}$ .
Quotient Rule: Use for functions divided. $(f / g)^{'} = (f^{'} g - f g^{'}) / g^{2}$ .
Chain Rule: Use for functions nested inside each other. $(f (g (x)))^{'} = f^{'} (g (x)) \cdot g^{'} (x)$ .
Complex derivatives are solved by identifying the main structure (product, quotient, or chain) and applying rules step-by-step, potentially nesting them as needed.
Practice is essential for becoming fluent in applying these rules.

Preview of the Next Lesson:

So far, all our functions have had a single input, like $f (x)$ . However, most machine learning models involve functions with many inputs (e.g., a cost function that depends on multiple model weights). In our next lesson, we will extend our knowledge to compute partial derivatives and gradients of multivariable functions. This will be our first step into the world of multivariable calculus and the core of how algorithms like gradient descent actually work.