Today there was a lot of discussion regarding the novelty of Google’s new paper on using AI for breast cancer screening in comparison to our earlier work (in the tweet below).
Let me offer my take on this.
1/N
Let me offer my take on this.
1/N
First of all, I would like to congratulate the authors of that paper for its acceptance and the attention it got. Overall, I think it’s an excellent paper.
nature.com/articles/s4158…
2/N
nature.com/articles/s4158…
2/N
“Novelty” is difficult to quantify. There are already multiple papers that show similar results. Some of them even precede ours. You can find many examples in the “related work” section of our paper or in a review that we wrote around the same time.
3/N
3/N
Having said that, the strength of G’s paper is in careful analysis of the results. I appreciate that. We will use some of their ideas for evaluations in future papers. Our paper also has extensive evaluations, but the methods are its more important part.
4/N
4/N
I believe when multiple groups show similar results with similar methods, it’s a good thing. It co-validates our approaches and it’s showing that the toolbox that we use (in this case, deep neural networks) is robust and works in different scenarios.
5/N
5/N
The greatest shortcoming of G’s paper is the lack of publicly available code reproducing their results. It’s very hard for anyone to build upon their work without it, especially for smaller groups that could potentially also contribute to the progress of the field.
6/N
6/N
Our code and the weights of trained networks are publicly available for that reason. I hope that sharing the code will become a standard in the future. I believe that journals, especially the big ones, should require it.
github.com/nyukat/breast_…
7/N
github.com/nyukat/breast_…
7/N
Ours and Google’s work on breast cancer screening is just the beginning. We both have shown that there is a potential of using neural networks in this context, but neither of the papers are showing any prospective evaluation in a clinically realistic setting.
8/N
8/N
Finally, breast cancer screening is the first step. There are fascinating challenges ahead: explainability, robustness, utilizing other imaging modalities, longer-term diagnosis, prognosis, discovery of new biomarkers… There is work to be done by many groups for years.
9/N
9/N
Ultimately, we should remember what the final goal of our research is, which is to decrease the effect that breast cancer has on millions of women and their families. We have an obligation to work towards this goal as a team.
10/N
10/N
