1\13 One of the biggest challenges in quant and alpha research is obtaining clean, error-free data. Models need be built, using assumptions to reduce “dimensions” of reality for tractability.
A THREAD ▼ ▼ ▼
Machine Learning, Sparse Datasets and Error-Free Simulations
A THREAD ▼ ▼ ▼
Machine Learning, Sparse Datasets and Error-Free Simulations
2\ Mathematical models, by definition are built to simulate and capture some phenomena, practical or abstract. Often, they are built on data, which themselves are derived from some unknown, statistical distribution.
3\ For example, an alpha model, is backtested upon data where prices/returns are drawn from some distribution. Widely in quantitative finance, they are assumed to be drawn from a log-normal distribution.
4\ However, there is extensive literature suggesting that empirical returns do not fit this assumption. In particular, asset prices display auto-correlative properties, while the random walk hypothesis suggests that returns are independently sampled.
5\ An explanation on the challenges of the assumptions of GBM by @VolQuant : a thread on Monte Carlo and Fractional Brownian motion.
https://twitter.com/VolQuant/status/1401038592424787973?s=20
6\ We take a look at a tangential topic.
Problem: some assets do not have much significant data to be modelled on, particularly in the crypto space. To fit flexible models using advanced features and non-linear approximations, we encounter the problem “Curse of Dimensionality”
Problem: some assets do not have much significant data to be modelled on, particularly in the crypto space. To fit flexible models using advanced features and non-linear approximations, we encounter the problem “Curse of Dimensionality”
7\ That is, we have too limited amount of data relative to the features, and in the low signal environments our model is bound to overfit and perform poorly.
8\ We may apply dimension reduction techniques such as component analysis and looking at its eigensystem, but this usually does little to solve our problem. Pruning features is also a solution, but not ideal.
9\
i) If our alpha modelling is dependent on having clean, continuous datasets, a common solution is just to “forward fill” the data, reflecting real world “trading halts”-
i) If our alpha modelling is dependent on having clean, continuous datasets, a common solution is just to “forward fill” the data, reflecting real world “trading halts”-
10\
-however, this does not really help any of our statistical learning algorithms. A decent solution is an application of a Brownian Bridge, for shorter periods.
ii) If the data were just too short (e.g. traded only for a year), most earlier dates missing, then we need more.
-however, this does not really help any of our statistical learning algorithms. A decent solution is an application of a Brownian Bridge, for shorter periods.
ii) If the data were just too short (e.g. traded only for a year), most earlier dates missing, then we need more.
11\
A solution is to apply Geometric Random Walks modelling the data we already have. However, as addressed, we know that when generated for longer durations, this is likely to depart from the true distribution. Classic Stochastic Models and Volatility Models also lack.
A solution is to apply Geometric Random Walks modelling the data we already have. However, as addressed, we know that when generated for longer durations, this is likely to depart from the true distribution. Classic Stochastic Models and Volatility Models also lack.
12\
A spectacular approach to generating synthetic data is to model the simulation of high frequency data, and using Style Transfer techniques to higher duration frequencies such as daily resolutions. A Denoising Autoencoder is trained with high frequency data.
A spectacular approach to generating synthetic data is to model the simulation of high frequency data, and using Style Transfer techniques to higher duration frequencies such as daily resolutions. A Denoising Autoencoder is trained with high frequency data.
13\
Results after Style Transfer found that the simulated paths are more similar in autocorrelation, and matches the true distributions most closely up to the 4^th moment.
Paper: arxiv.org/pdf/1906.03232…
Results after Style Transfer found that the simulated paths are more similar in autocorrelation, and matches the true distributions most closely up to the 4^th moment.
Paper: arxiv.org/pdf/1906.03232…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
