Derivatives of Common Functions

Introduction

In our last lesson, we built the core intuition behind calculus. You learned that a derivative represents an instantaneous rate of change and, geometrically, is the slope of the tangent line to a function's graph. We focused on the what and the why, understanding that the derivative is the value that the average rate of change approaches as our measurement interval shrinks to zero.

Now, we move from intuition to computation. While the limit definition is the theoretical foundation, using it every time you need a derivative is impractical. Fortunately, mathematicians have developed a set of rules for common functions. This lesson is dedicated to mastering these fundamental rules. You will learn to compute the derivatives of polynomial, exponential, and logarithmic functions. These are the essential building blocks you'll need to analyze and optimize the complex functions that power AI models.

1. Derivatives of Polynomials: The Power Rule

Polynomials, like $f (x) = 3 x^{2} + 5 x - 1$ , are one of the most common types of functions. Differentiating them relies on a simple yet powerful rule.

Let's start by learning this rule, known as the Power Rule.

Intro to Derivatives | Calculus for Machine Learning

In this segment from Kylie Ying's 'Calculus for Machine Learning' series, you'll be introduced to the Power Rule, the primary tool for differentiating polynomials.

Watch the clip from 20:27 to 23:03. Pay close attention to the formula presented for a function of the form f(x) = bx^a and how it's applied to individual terms.

As the video explained, the Power Rule is straightforward. For any function of the form $f (x) = x^{n}$ , its derivative is:

\frac{d}{d x} x^{n} = n x^{n - 1}

In simple terms: bring the exponent down as a multiplier, then subtract one from the original exponent.

Let's break this down with a few key cases:

A constant multiplier: If $f (x) = c \cdot x^{n}$ , the constant c just comes along for the ride: $f^{'} (x) = c \cdot n x^{n - 1}$ .
- Example: If $f (x) = 5 x^{3}$ , then $f^{'} (x) = 5 \cdot 3 x^{3 - 1} = 15 x^{2}$ .
A linear term: If $f (x) = a x$ , this is the same as $a x^{1}$ . Applying the rule gives $f^{'} (x) = a \cdot 1 x^{1 - 1} = a \cdot x^{0} = a \cdot 1 = a$ . This makes sense: the slope of a line like $y = a x$ is always $a$ .
A constant term: If $f (x) = c$ , this is the same as $c x^{0}$ . Applying the rule seems tricky, but it's simpler to think geometrically: a constant function is a horizontal line. Its slope is always zero. Thus, the derivative of any constant is 0.

To differentiate a full polynomial, we use the Sum Rule, which states that the derivative of a sum of terms is simply the sum of their individual derivatives.

For a concise, text-based summary of these rules, you can refer to the "Dive into Deep Learning" book.

22.3. Single Variable Calculus - Dive into Deep Learning

This resource provides a formal summary of the basic derivative rules we've just discussed.

Read the section titled 'Common Derivatives'. Note the rules for constants, linear functions, and the Power Rule. Then, look just below at the start of 'Derivative Rules' and note the Sum Rule.

Now, we can differentiate any polynomial. For example, let's find the derivative of $f (x) = 4 x^{3} + 2 x - 7$ .
We differentiate term by term:

The derivative of $4 x^{3}$ is $4 \cdot 3 x^{2} = 12 x^{2}$ .
The derivative of $2 x$ is $2$ .
The derivative of $- 7$ is $0$ .

Putting it all together: $f^{'} (x) = 12 x^{2} + 2$ .

2. Derivatives of Exponential Functions

Next, we turn to exponential functions, where the variable is in the exponent, like $f (x) = 2^{x}$ .

A crucial distinction: Do not confuse the Power Rule with the rule for exponentials.

Power Rule: Variable in the base, constant in the exponent (e.g., $x^{2}$ ).
Exponential Rule: Constant in the base, variable in the exponent (e.g., $2^{x}$ ).

The world of exponential functions has one very special member: $f (x) = e^{x}$ , where $e \approx 2.71828$ . This function has a unique and remarkable property.

Derivatives of Logarithmic and Exponential Functions

Let's watch a video from the 'Professor Dave Explains' channel to uncover the derivative of exponential functions, starting with the special case of e^x.

First, watch the segment from 4:51 to 07:34. This part reveals the derivative of e^x and provides a fascinating proof of its property. Then, watch from 03:40 to 04:51, which covers the general rule for any exponential function a^x.

As you saw, the function $f (x) = e^{x}$ is its own derivative!

\frac{d}{d x} e^{x} = e^{x}

This means that at any point on the graph of $y = e^{x}$ , the value of the function is exactly equal to the slope of its tangent line. This is a profound property that makes $e$ a fundamental constant in mathematics and science.

For the general exponential function $f (x) = a^{x}$ , the rule is slightly different:

\frac{d}{d x} a^{x} = a^{x} ln (a)

Here, $ln (a)$ is the natural logarithm of the base $a$ . Notice that if we set $a = e$ , this rule becomes $\frac{d}{d x} e^{x} = e^{x} ln (e)$ . Since $ln (e) = 1$ , the general rule simplifies to the special case.

3. Derivatives of Logarithmic Functions

Finally, let's look at the derivatives of logarithmic functions, which are the inverses of exponential functions. The two main forms are the natural logarithm, $ln (x)$ , and the logarithm with a general base, $lo g_{a} (x)$ .

Derivatives of Logarithmic and Exponential Functions

Let's return to the 'Professor Dave Explains' video to learn the rules for logarithmic functions.

Watch the segment from 00:44 to 03:40. Focus on the derivative rule for ln(x) and see how the change of base formula is used to derive the rule for log_a(x).

The rules for logarithms are as follows. For the natural logarithm:

\frac{d}{d x} ln (x) = \frac{1}{x}

And for a logarithm with base $a$ :

\frac{d}{d x} lo g_{a} (x) = \frac{1}{x ln ( a )}

Again, you can see the consistency: if we set $a = e$ , the general rule becomes $\frac{d}{d x} lo g_{e} (x) = \frac{1}{x l n ( e )}$ , which simplifies to $\frac{1}{x}$ since $ln (e) = 1$ .

Summary and Practice

You have now learned the fundamental rules for differentiating three important classes of functions. Let's summarize them in a single table for easy reference.

Function Type	Function Form $f (x)$	Derivative $f^{'} (x)$
Constant	$c$	$0$
Power Rule	$x^{n}$	$n x^{n - 1}$
Exponential	$e^{x}$	$e^{x}$
Exponential (General)	$a^{x}$	$a^{x} ln (a)$
Logarithm	$ln (x)$	$\frac{1}{x}$
Logarithm (General)	$lo g_{a} (x)$	$\frac{1}{x l n ( a )}$

Let's apply these rules to a function that combines several types.
Consider the function: $f (x) = 2 x^{5} - 3 e^{x} + 4 ln (x)$ .

Using the Sum Rule, we can differentiate it term-by-term:

Derivative of $2 x^{5}$ : Using the Power Rule, we get $2 \cdot 5 x^{5 - 1} = 10 x^{4}$ .
Derivative of $- 3 e^{x}$ : The derivative of $e^{x}$ is $e^{x}$ , so this becomes $- 3 e^{x}$ .
Derivative of $4 ln (x)$ : The derivative of $ln (x)$ is $\frac{1}{x}$ , so this becomes $4 \cdot \frac{1}{x} = \frac{4}{x}$ .

Combining these, the full derivative is:

f^{'} (x) = 10 x^{4} - 3 e^{x} + \frac{4}{x}

Conclusion

In this lesson, you have equipped yourself with the essential computational tools of differential calculus. You've moved beyond the conceptual definition of a derivative to the practical application of rules for finding them.

Key Takeaways:

The Power Rule ( $\frac{d}{d x} x^{n} = n x^{n - 1}$ ) is used to differentiate polynomials.
The derivative of the natural exponential function, $e^{x}$ , is uniquely itself. The general rule for $a^{x}$ is $a^{x} ln (a)$ .
The derivative of the natural logarithm, $ln (x)$ , is $\frac{1}{x}$ . The general rule for $lo g_{a} (x)$ is $\frac{1}{x l n ( a )}$ .
The Sum Rule allows us to differentiate complex functions by breaking them down into simpler terms.

Preview of the Next Lesson:
The rules you've learned today apply to functions that are added or subtracted. But what happens when functions are multiplied, divided, or even nested inside one another (like $f (x) = e^{x^{2}}$ )? In the next lesson, we will expand our toolkit by learning the Product, Quotient, and Chain Rules, which will allow you to differentiate almost any single-variable function you might encounter.