,
8 tweets,
2 min read
Read on Twitter
There's a subtle ambiguity in the definition of a "derivative of a function" that has a habit of confusing people. The derivative is often introduced as "how quickly a function changes at a point", which would make it a scalar value, at least in one dimension.
But the term "derivative" is also used to denote a function that maps _each_ point to the local derivative value around that point.
In other words differentiation can be defined as a map from an entire function and a point to a single number (the derivative at that point) or as a map from an entire function to and entirely new function (the mapping from point to derivative).
Incidentally this is why mathematical notation like d(f(x_{0}) / dx is sloppy and confusing. You don't differentiation the single value f(x_{0}) but rather the entire function f and then evaluate it at a point. Hence the much better notation is df/dx(x_{0}).
One place this confusion arises? In algorithmic differentiation. Automatic differentiation computes the derivative at a point where as symbolic differentiation computes the entire derivative function. Those different outputs are why the implementations are so different.
This "define something as behavior in small neighborhood and then define a function that maps points to local behavior in a small neighborhood around that point" pattern is also all over differential geometry and took me forever to really understand while learning the math.
i.e. the difference between a vector and a vector field, a tensor and a tensor field, a tangent space and a tangent bundle, etc. When first learning you have to be really careful to explicitly lay out what is a point value and what is a function to avoid confusing the two.
This was Sunday morning jet lagged mathematical theater.
