Thread by @seanjtaylor

📈Long thread on how Prophet works📈

- Instead of sharing slides I'm transcribing my QCon.AI talk to tweets with lots of GIFs :)
- Thanks to @_bletham_ for all his help.

Prophet motivation:

- there are many forecasting tasks at companies
- they are not glamorous problems and most people aren't trained well to tackle them
- 80% of these applications can be handled by a relatively simple model that is easy to use

We approach time series forecasting as a *curve fitting problem.* This has some benefits:
- curves are easy to reason about and you can decompose them
- the parameters you fit have straightforward interpretations
- curve fitting is very fast so you can iterate quickly

Your first attempt at fitting a curve might be a polynomial. You can see that as you add terms you can *interpolate* better but the *extrapolation* is very poor. This is because the local trend it is estimating is usually very wrong.

Many of the weaknesses of polynomial models can be improved by using splines. Here is a natural spline fit for the same data. It uses a similar number of parameters to the polynomial but fits a much better local trend. Prophet borrows this idea.

Prophet is also a piecewise curve, and in its simplest form it is piecewise *linear*. That magic is in how we select the knots -- where the curve is allowed to change. Notice:
- knots are introduced where they are needed
- the extrapolation is stable as we vary knots

We call the knots "changepoints." We generate a grid of them and then use an L1-penalty to select a small number of non-zero ones. Here's what that process looks like as we change the penalty.

To generalize this model, we add other components to capture seasonality and holidays. Prophet is really a special kind of GAM, with i.i.d. noise. Notice that we don't focus on modeling the error! You could add some ARMA process to the errors but we have not.

For seasonality, we generate a partial Fourier sum (which can approximate an arbitrary periodic signal) for standard periods (weekly/daily/yearly).

Prophet forecasts have three sources of uncertainty:
- irreducible noise (🤷‍♂️)
- parameter uncertainty
- trend forecast uncertainty

Anything Prophet cannot fit with a curve is modeled as mean-zero i.i.d. random noise.
- This creates tube-shaped uncertainty in the forecast.
- Large uncertainty indicates the model has fit the historical data poorly.

Every parameter we have fit in the model has sampling variance.
- This includes all seasonalities, trends, and changepoints.
- As an option, can use Stan’s built in HMC implementation to sample draws from posterior.
(credit @twiecki for the animation)

The most interesting thing we handle in Prophet is *trend uncertainty.* We perform a "trend change simulation:"
- at each date in the forecast we allow the trend to change
- rate is based on how many changepoints were selected
- distribution is selected based on the magnitudes

Notice this interplay between interpolating well (selecting a large number of changepoints) which causes the model to fit well but a lot of (cone-shaped) trend uncertainty. Underfitting causes a lot of tube-shaped uncertainty.

Two tuning parameters that I'd like to highlight:
- seasonality prior scale (SPS)
- capacities

SPS controls how flexibly we fit the different seasonalities. You can also just turn seasonalities off if you have a prior they don't exist.

Capacities are useful for injecting domain knowledge about growth.
- Piecewise logistic growth curves have a capacity parameter that we do not fit from data.
- use obvious constraints as upper and lower bounds on forecasts.
- capacity can be a constant or as a time series.

Conclusions:
- Prophet is a simple model but covers many important use-cases at FB and elsewhere.
- Simple is good! Prophet works robustly and fails in understandable ways.
- Using curve-fitting w/ interpretable parameters allows users to input domain knowledge into forecasts.