Thread by @y2silence

@yudapearl @nagpalchirag Indeed, I have one very strange example where identifiability seems easier to see with PO. First, let me be clear that I am not making up the causal question. It is a real research problem that has baffled educational measurement researchers since the 1920s.

@yudapearl @nagpalchirag While taking multiple-choice exams, examinees may or may not change their initial answers. The decision is made by their ability, anxiety, and many other unknown factors, which also affect the final score. People have wondered whether answer changing is beneficial or harmful.

@yudapearl @nagpalchirag Viewing answer changing as a treatment and final score as an outcome, I may draw a DAG where such unknown factors (e.g., ability, anxiety, etc.) will create a backdoor path. The conclusion from the DAG is, we cannot identify the causal effect of answer changing.

@yudapearl @nagpalchirag However, let's assume that examinees are solving a true/false question. One may initially choose "true," which is in fact incorrect. His potential control outcome is then 0 (wrong) while his potential treatment outcome is 1 (right). So, the individual causal effect is 1-0=1.

@yudapearl @nagpalchirag This reasoning seems straightforward with PO, skipping the description about the answer changing selection process (by ability, anxiety, etc.), which is the key to drawing DAGs. In this case, just directly considering "what we wish to know" seems to work better.

@yudapearl @nagpalchirag However, this is a really weird example where the so-called "fundamental problem of causal inference" is easily solved due to the special nature of the treatment (answer changing in true/false questions). In general, I think DAGs work better but this is one crazy counterexample.

@yudapearl @nagpalchirag More detailed explanations about this weird research question can be found here: arxiv.org/ftp/arxiv/pape…
Any comments are welcome!