Thread by @TerribleQuant

Found this bad boy in my bookmarks from back when I decided manifolds were the key to the market and bought like 5 textbooks on them.

quantivity.wordpress.com/2011/05/08/man…

For those wondering if they are, I'll give a few comments:

T-SNE is always something I want to apply, but can never quite figure out the right way to do it. There is certainly some benefits to be had from having a basic understanding of what this all means so you can get a better chance at visualizing your features in 2D...

Much like stochastic methods, as much as I would never make them the center of a model, there is always use as a feature or ensemble. Ensemble is truly the free lunch of alpha...

Ricci curvature, etc. There are plenty of papers that try to predict crashes with these sorts of things but it is very tricky to pick out one structure in particular and not find some false flags. Differential, topological, and algebraic structures are all key components...

Back to ensemble, for those who aren't aware an ensemble of all GARCH family models beats any single EGARCH, TGARCH, HDMI-FLATSCREEN-TV-GARCH model you can whip up..

I think it has a little bit of use for visualizing data but always put it alongside something like PCA so you can see all the elements of structure, it is particularly nice when you are creating synthetic data (if you haven't heard of this already, try MAD-GAN, it's sick)...

Dimensionality reduction is definitely an art. PCA smashes most autoencoders unless you have loads of data so use a mix. Keep in mind PCA is unsupervised and simply orders your data in the form of key components, you then pick how many of the components you want...

You'll find some great results just from applying PCA and leaving it the same dimensionality and then applying a supervised autoencoder. Same with manifolds. You wouldn't use an NN without Kalman, HHT, FFT, TA, etc as features so why use an AE without features...

Even if it is supervised it can still do with some help. I've seen MLP and LSTM supervised AEs so far, both work great. Think of PCA and Manifold learning for dimensionality reduction as your features.

Just remember if I take in 50 features, use PCA, I still get 50 components. All it's done is split it into the shared (linearly) information and the residuals. I.e. X and Y perform a linear regression. I take the linear regression as my new first variable, and the residual...

as my second variable. Still the same data, I've just transformed it into something with more meaning. It's just that people then take only the first variable and chuck away the residuals. A good AE will find assistance and guidance from that structure and then...

pick up on non-linear effects. Don't waste data for it to learn the easy linear shit, use established models -> introduce that rigid basis -> and then let it do it's magic.

There is more to this thread, I just replied to the tweet above this on (the one ending with "without features...") instead so twitter had fucked it all up