Main points: (1) Conformal Inference can be made applicable in many #stats problems (2) There are lots of misconceptions about Conformal Inference (3) Try it!
1/n
Conformal Inference was designed for generating prediction intervals with guaranteed coverage in standard #ML problems.
Nevertheless, it can be modified to be applicable in
✔️Causal inference
✔️Survival analysis
✔️Election night model
✔️Outlier detection
✔️Risk calibration
2/n
Misconceptions about conformal inference:
❌ Conformal intervals only have marginal coverage and tend to be wide
✔️ Conformal intervals w/ proper conformity scores achieve conditional coverage & efficiency (short length) if the model is correctly specified
3/n
Misconceptions about conformal inference:
❌ Conformal inference is slow
✔️ Split conformal inference incurs almost negligible computational overhead (nearly as fast as the algorithm it wraps around)
4/n
Misconceptions about conformal inference:
❌ Conformal inference can’t be used with Bayesian procedures
✔️ Conformal inference can not only wrap around Bayesian methods, but also achieve the validity (w/ a simple adjustment) in both Bayesian and Frequentist sense.
5/n
I’m so proud that my job talk convinced many folks to read more about Conformal Inference. There are tons of exciting methodological questions (distribution shifts, dependence, conditional coverage, …) and real-world applications.
It’s a very🔥area now! Come and join us!
6/6
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Call F_2 & F_2^0 the classes of dists satisfying 1-3 & 1-4.
Hansen proves if \hat{β} is unbiased under F_2 for all Σ, then GLS (OLS) is BUE under F_2 (F_2^0).
Conformal inference, often framed as a technique to generate prediction intervals, is also a tool for out-of-distribution detection. We studied marginal/conditional conformal p-values for multiple testing with marginal/conditional error control 1/n
We consider the setting where a dataset of “inliers” is available. Existing outlier detection algorithms often output a “score” for each testing point indicating how regular it is.
But how to choose a cutoff to get guaranteed statistical error (e.g., type-I error) control? 2/n
For a single data point X, it can be formulated as a hypothesis testing problem with H0: X~P, where P is the (unknown) distribution of inliers X_1, …, X_n. Intuitively, H0 should be rejected if score(X) is too small compared to {score(X_1), …, score(X_n)}. 3/n
Check out our new work on conformalized survival analysis w/@RenZhimei and Emmanuel Candès: arxiv.org/abs/2103.09763 Our method can wrap around any survival predictive algorithms and produce calibrated covariate-dependent lower predictive bounds (LPBs) on survival times. 1/n
Survival predictive analysis is complicated by *censoring*, which partially masks the outcome. For example, the actual survival time is unknown for units whose event (e.g., death) has yet to happen. A common type is called the “end-of-study” censoring, illustrated below. 2/n
Under two standard assumptions, our conformalize LPBs achieve
(a) marginal coverage in finite samples if P(C|X) is known
(b) approx. conditional coverage if P(T|X) is well estimated
(c) approx. marginal coverage if either P(C|X) or P(T|X) is well estimated (doubly robust)