Profile picture
Ian Goodfellow @goodfellow_ian
, 9 tweets, 2 min read Read on Twitter
A math trick I like a lot is the approach to taking derivatives using hyperreal numbers. Thread:
For this trick we introduce a new kind of number, called an infinitesimal hyperreal number. Imagine we have some number epsilon, such that epsilon > 0 but epsilon < x for all positive real numbers x. Imagine that we can use algebra to manipulate epsilon like any other variable.
Now we can compute derivatives just by using "plug-and-chug" algebra on the expression f'(x) = real_part(f(x+epsilon) - f(x)) / epsilon) where the "real_part" function just rounds off the infinitesimal part of some expression
For example, let's take the derivative of x^2: ((x+eps)^2 - x^2) / eps = (x^2 + 2 eps x + eps^2 - x^2) / eps = 2 + eps. We round off the infinitesimal eps and are left with 2.
This is a lot more "automatic" than setting up a limit as epsilon approaches zero and proving that the limit converges.
It might seem like cheating to just make up infinitesimal hyperreal numbers and say that we can manipulate them with the standard rules of algebra... but it turns out that such algebra rules are logically consistent if and only if the same rules for the real numbers are
Hyperreal numbers formalize some of the intuitions that Newton and Leibniz used without much formal justification in the early development of calculus. Hyperreal numbers stay much closer to the original intuition than earlier formalizations based on limits do.
There are also infinite hyperreals, larger than any real number, but I don't personally find them quite as useful for the math I need to do for machine learning research.
It is also possible to compute integrals using hyperreal numbers, but I don't personally find as much of an advantage to hyperreal numbers over limits for integration as I do in the case of derivation.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Ian Goodfellow
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!