Thread by @TivadarDanka

The way you think about the exponential function is (probably) wrong.

Don't think so? I'll convince you. Did you realize that multiplying e by itself π times doesn't make sense?

Here is what's really behind the most important function of all time:

First things first: terminologies.

The expression aᵇ is read "a raised to the power of b."

(Or a to the b in short.)

The number a is called the base, and b is called the exponent.

Let's start with the basics: positive integer exponents. By definition, aⁿ is the repeated multiplication of a by itself n times.

Sounds simple enough.

But how can we define exponentials for, say, negative integer exponents? We'll get there soon.

For that, two special rules will be our guiding lights. First, exponentiation turns addition into multiplication.

We'll call this the "product of powers" rule.

Second, the repeated application of exponentiation is, again, exponentiation.

We'll call this the "power of powers" rule.

These two identities form the essence of the exponential function.

To extend the definition to arbitrary powers, we must ensure that these properties remain true.

So, what about, say, zero exponents? Here, the original interpretation (i.e., repeated multiplication) breaks down immediately.

How do you multiply a number by itself zero times?

To find the definition, we’ll use wishful thinking.

I am not kidding. Wishful thinking is a well-known and extremely powerful technique.

The gist is to assume that powers with zero exponents are well-defined, then use some algebra to find out what the definition might be.

In this case, the "product of powers" property gives the answer: any number raised to the power of zero should equal to 1.

What about negative integers? We cannot repeat multiplication zero times, let alone negative times.

Again, let's use wishful thinking.

If powers with negative integer exponents are indeed defined, the "product of powers" tells us what they must be.

What about rational exponents? You guessed right. Wishful thinking!

The "power of powers" rule yields that it is enough to look at exponents where the numerator is 1.

The same rule gives that rational exponents with numerator 1 must be defined in terms of roots.

Thus, we finally see how to make sense of rational exponents.

Now, what about arbitrary real numbers?

Buckle up. We are about to floor the gas pedal. So far, we have defined the exponential function for all rationals.

Let's use this to our advantage!

Real numbers are weird.

Fortunately, they have an exceptionally pleasant property: they can be approximated by rational numbers with arbitrary precision.

This is because no matter how close we get to a real number, we can find a rational number there.

When the approximating sequence is close to the actual exponent x, the powers are also close.

Closer and closer as n grows.

Thus, we can define exponentials for arbitrary real exponents by simply taking the limit of the approximations.

And we are done!

Join 20,000+ ML practitioners who get 2 actionable emails every week to help them understand the math behind ML, make smarter decisions, and avoid costly mistakes.

Subscribe here (it’s free): thepalindrome.org