As far as current machine learning is concerned, generalization originates from the ability to learn the latent manifold on which the training data lies, i.e. the ability to interpolate between training samples (local generalization, by definition)
As such, attempting to study generalization by fitting models to random data is a category error, since such data offers no possibility of generalization. To understand generalization, you must look at the relationship between models and natural data manifolds.
In the future, AI will be capable of extrapolation, not just interpolation, i.e. broad generalization, & further, general intelligence. This is an endeavor entirely orthogonal to interpolation / curve fitting, which will be achieved through an entirely different set of techniques
Of course, interpolation & pattern recognition are nevertheless a fundamental component of intelligence -- just not the most powerful. Being able to do it well -- by refining deep learning -- is immensely valuable
