Last Saturday, at 12:02pm, I found what looked an awful lot like Planet Nine (spoiler: it wasn't). It was in the data in the right places at the right brightness moving in the right directions. The data in that part of the sky were generally pretty good. My breathing slowed...
I fit the data to an orbit; the orbit as good and made sense for how we think P9 is affecting the outer solar system. My mind starting racing to "what do I do now?". My daughter was the only person home. She was going to be the second human to know (then I'd call @kbatygin )
I'd never seen anything so promising. 9 points in the sky that, over the course of 3 years, are all in the perfect spot to fit an orbit around the sun that is the right orbit. I mean COME ONE THIS MUST BE IT RIGHT? [spoiler: no]
I started looking carefully at the individual data points. The first one was a normal looking detection. The second one was.... fake data that I put in as a test. So were the next 8. It was 1 real data point that matched 8 fake points by chance.
The orbit was good because I had made it myself. And my normal vetting software didn't flag it as a fake because the first matched object was not fake (software changed). I started to breath again. I didn't disturb my daughter.
This is just your reminder that every real discovery in science is preceded by 99 LOOK WHAT I JUST FOUND OH NEVER MIND almost discoveries. Or more. That's why I never think there are Eureka! moments. There are hmmm, I wonder what I screwed up this time moments. And sometimes, ...
..sometimes the answer to what-did-I-screw-up is nothing. Those are sweet moments and oh so rare. I'm still waiting, in this particular case. But it is fun to be reminded of what the real day might be like.
/end
/end
OK OK a little bit more. Why was the test data still in the data set, you ask? Because I need to make sure my software finds things. Searching the sky for a singular Planet Nine is weird. If your search is finished and it comes up empty, did you do it right?
Did you have a bug somewhere? So you make test data and run through all of it and it works. But does it work at the scale of the real data? So you inject the test data into the real data and do the analysis all together. In my current data set I inject 100,000 fake P9s.
If I don't find them all I need to find out why. If I do and don't find P9 I can be confident that if P9 has the properties of my fakes I would have found it. The trick, though, is to make sure you don't confuse fakes with real ones. I won't again. Maybe.
/reallytheend
/reallytheend
