Thread by @chrisoutofspace

Our team has a new paper out lead by Martin Reiss on what is for sure one of the hardest problems in #spaceweather forecasting: predicting the magnetic field inside solar storms.
🧵 👇
#openaccess @AGUSpWx
@FWF_at @Know_Center @IWF_oeaw @UniGraz @ZAMG_AT
agupubs.onlinelibrary.wiley.com/doi/10.1029/20…

Have you ever wondered why forecasts of solar storms, well, really suck? Nature has added this weird twist that for developing a geomagnetic storm (+ aurora), the magnetic field of Earth needs to be temporarily "cut up" by a solar storm so the magnetosphere can be energized.

But this happens *only if* the magnetic field that is carried by the solar wind points into the opposite direction to Earth's field, which is northward 🌍⬆️ at its front. Only long-lasting southward ☀️⬇️ magnetic fields from the Sun are able to drive strong magnetic storms.

These smooth negative "Bz" fields are only present in solar storms (aka coronal mass ejections, CME). Now, nobody in the world is currently able to predict how this field behaves before a CME impacts the Earth - this is the "Bz problem".

There are many reasons for this: we don't exactly understand how those fields form on the Sun, or how they evolve up to Earth. But even if we did and if we would have a perfect physics simulation for those processes, it still wouldn't be enough.

We would also need to know how to set the CME initial direction and other parameters with very high precision from solar data alone - and this is a precision that we don't have in the foreseeable future.

So aurora watchers drive out in the field, or power grid and satellite operators are on alert when a CME impacts, but as there are no accurate forecasts further than about 30 min to 1 hour into the future, it's always a guessing game whats gonna happen.

Now in Reiss et al. 2021, we have started a new approach: we use the CME parameters that spacecraft like DSCOVR or ACE at the Sun-Earth L1 point measure, like arrival time, speed, and magnetic field and try to extrapolate from there what is going to happen.

A while ago this was an idea raised by @tamithaskov and myself, but its a science rule that *every idea* was there earlier if you just search long enough, and indeed Chen et al. from the 1990s already came up with it, so neither of us can claim it.🤷‍♂️
But: now we have more data.

Here, for a start, we used machine learning methods (gradient boosting and random forest regressors) in order to find the minimum value of Bz in a CME. This already gives a constraint on the maximum expected strength of the magnetic storm.

We train the algorithms with data from 348 CMEs observed in situ with the @NASASun STEREO and Wind spacecraft since 2007, available at helioforecast.space/icmecat

We find that we can predict the minimum Bz value in CMEs with a mean error of about 3 nT and correlation of 0.7, which is not bad at all. Of course, we would like to know when this minimum happens and predict the time series evolution of Bz, but this is something for follow ups.

But the *really cool thing* is that if we add more and more data from the CME magnetic core, like in reality when you wait for a few more hours and you start to see the smooth magnetic fields, the prediction gets better!
Error in field magnitude (left), Bz component (right)

☀️Summary 🌍
By feeding measurements of solar storms into machine learning algorithms and the hypothesis that we can extrapolate the magnetic field parameters from there, we find that our predictive tool can forecast a minimum value for Bz component reasonably well.

While our approach does not solve the Bz problem (yet), it shows some promise to follow that avenue in the future - so maybe one day we will be able to predict the aurora with half a day or even a day in advance with good accuracy.