Great work showing that a good AI system doesn't always help doctors.
Echoes the decades of experience with radCAD: when the system is wrong, it biases the doctor and makes them *worse* (OR 0.33!) at diagnosis.
It is *never* as simple as AI+doctor is better than doctor alone.
Echoes the decades of experience with radCAD: when the system is wrong, it biases the doctor and makes them *worse* (OR 0.33!) at diagnosis.
It is *never* as simple as AI+doctor is better than doctor alone.
I personally suspect the biggest problem is automation bias, which is where the human over-relies on the model output.
Similar to self driving cars where jumping to complete automation appears to be safer than partial automation.
Similar to self driving cars where jumping to complete automation appears to be safer than partial automation.
But interestingly (and perhaps counter-intuitively) this could also mean that "blind" ensembling (where the human gets no AI input, and the human and AI opinions are combined algorithmically) might be better than showing the doctor what the AI thinks.
There is a school of thought that dominates a lot of biostatistics community that we should simply give doctors as much information as possible, to the point that dichotomizing AI decisions into disease vs no disease is wrong (and we should supply percentage chance or similar).
But I suspect we will find out through hci research that more information is objectively worse in most scenarios; that it promotes cognitive biases that harm human performance.
Interesting topic, interesting results. We need more cognitive psychologists working with AI use!
Interesting topic, interesting results. We need more cognitive psychologists working with AI use!
PS turns out I underestimated the harm here going by @pranavrajpurkar's tweet. The actual odds ratio when the system is wrong was 0.25!
This very clearly shows that humans do not have the ability to "stick to their guns" when an AI contradicts them. Scary!
This very clearly shows that humans do not have the ability to "stick to their guns" when an AI contradicts them. Scary!
