A math trick I like a lot is the approach to taking derivatives using hyperreal numbers. Thread:
For this trick we introduce a new kind of number, called an infinitesimal hyperreal number. Imagine we have some number epsilon, such that epsilon > 0 but epsilon < x for all positive real numbers x. Imagine that we can use algebra to manipulate epsilon like any other variable.
Now we can compute derivatives just by using "plug-and-chug" algebra on the expression f'(x) = real_part(f(x+epsilon) - f(x)) / epsilon) where the "real_part" function just rounds off the infinitesimal part of some expression
For example, let's take the derivative of x^2: ((x+eps)^2 - x^2) / eps = (x^2 + 2 eps x + eps^2 - x^2) / eps = 2 + eps. We round off the infinitesimal eps and are left with 2.
This is a lot more "automatic" than setting up a limit as epsilon approaches zero and proving that the limit converges.
It might seem like cheating to just make up infinitesimal hyperreal numbers and say that we can manipulate them with the standard rules of algebra... but it turns out that such algebra rules are logically consistent if and only if the same rules for the real numbers are
Hyperreal numbers formalize some of the intuitions that Newton and Leibniz used without much formal justification in the early development of calculus. Hyperreal numbers stay much closer to the original intuition than earlier formalizations based on limits do.
There are also infinite hyperreals, larger than any real number, but I don't personally find them quite as useful for the math I need to do for machine learning research.
It is also possible to compute integrals using hyperreal numbers, but I don't personally find as much of an advantage to hyperreal numbers over limits for integration as I do in the case of derivation.
