🤖Conformal PID Control for Time-Series Prediction🕹
📝w/Candès & Tibshirani!



Is conformal prediction out of control🤪?

Yes! CP can be thought of as PID control, giving stronger guarantees+algorithms that predict distribution shifts as they happen! A🧵 https://t.co/3NvH41Gw2farxiv.org/abs/2307.16895
Before I explain how it works, check this result on COVID death forecasting.

The official CDC forecast missed the upswing. Running PID control on top improved the result — by a lot.

Why? Because the errors were predictable, so it just PREDICTED them! Hence, better coverage :)

📯Prediction-Powered Inference📯



With the rise of AlphaFold etc., people are using ML predictions to replace costly experimental data.

But predictions aren't perfect; can we still use them for rigorous downstream inferences?

The answer: yes. A 🧵 arxiv.org/abs/2301.09633
The key idea is to define a mathematical object called

Δ, the rectifier.

We use Δ to "rectify" the predictions in a way that recovers the ground truth estimand.

For means, Δ is a bias, but in general, it is not. We can use Δ to form a prediction-powered confidence set (green).

I’m thrilled to announce Conformal Risk Control: a way to bound quantities other than coverage with conformal prediction.

arxiv.org/abs/2208.02814

Check out the worked examples in CV and NLP!

The best part is: it’s exactly the same algorithm as split conformal prediction🤯🧵1/5 Conformal prediction bounds the miscoverage:
P(Y notin set)<=α.

Our paper shows that conformal prediction actually works to bound any risk,
E[L(set,Y)]<=α,
for some loss L that shrinks as the set grows.

L(set,Y)=I(Y notin set) reduces to conformal, but you can do more... 2/5