I think people (including the authors) are over-interpreting arxiv.org/abs/1711.11561 "Measuring the tendency of CNNs to learn surface statistical regularities" 1/
The authors claim "deep CNNs tend to learn surface statistical regularities in the dataset RATHER THAN higher-level abstract concepts". [emphasis mine] 2/
But while their experiments show that the CNNs do learn surface statistical regularities, they do not prove that the CNNs fail to learn higher-level abstract concepts 3/
In fact, the networks are able to generalize fairly well even when the data have been perturbed by Fourier filters. Generalization accuracy is damaged by 8ppt on SVHN and 28ppt on CIFAR10. Very significant, but not catastrophic 4/
I suggest an alternate hypothesis: "Deep CNNs tend to learn surface statistical regularities in the dataset IN ADDITION TO higher-level abstract concepts". Of course, it remains to be seen how "high" and "abstract" the learned "concepts" are. 5/
A related hypothesis is that adversarial examples work by manipulating the surface statistics, and the CNNs have put enough weight on those that they are fooled. 6/
What experiments could refute these alternate hypotheses? Perhaps manipulations of surface statistics that reduce Deep CNN performance to the level of random guessing? end/
