finally got this mf working end-to-end
behold the dawn of a new era
behold the dawn of a new era
I have a working dynamic photometric SLAM for some definition of working and photometric. it is currently v. janky for many reasons but the PoC just worked for the first time as a full system. In ~2w this thing will fly and be as precise as a swiss watch
some refinements, different dataset
lots more still to do
lots more still to do
strategically cropped so you don't see the really bad track of this one tiny Smart ForTwo that's almost the same color as the road surface
name my 5K LoC child
OK, here we go, final push on this version of this system
we are going to do a full joint optimization per-object to get FINE TUNED tracks and not janky-ass independent unconstrained tracks
the Jacobian, she is so big fam
we are going to do a full joint optimization per-object to get FINE TUNED tracks and not janky-ass independent unconstrained tracks
the Jacobian, she is so big fam
if I was going to do this for real I'd also constrain it on the camera-background pose and not just the object-object poses between frames but I would like to have A version done that I can iterate on
there is a dissertation's worth of work on this problem only, but I have so much I want to do
shit that absolutely does not work at all yet:
- tracking robustness is hilariously bad, if you go behind a street sign I have no idea where you went
- tracking robustness is hilariously bad, if you go behind a street sign I have no idea where you went
- conversely, detecting when I've lost tracking is also hilariously bad, it's literally 'moving that far doesn't seem reasonable, I'm evicting you from the active set'
- association of tracks between keyframes is just intersection over union of the reprojected segment into the new keyframe, which has so many problems oh my god
- everything is done in intensity space and not in colourspace; colours would really help mitigate shit like the fact that you can partially go into a shadow and I have no idea what happened to you
- I assume your object doesn't take up a volume larger than a sphere centered on your simple arithmetic centroid, and will just eject points that don't match that assumption
things that work really well:
expectation maximization using a CRF as an augmentation to the traditional direct photometric alignment works _really_ well, and can be done in realtime no problem with a GPU.
expectation maximization using a CRF as an augmentation to the traditional direct photometric alignment works _really_ well, and can be done in realtime no problem with a GPU.
- the code base is terrible and will need a full refactoring before I release it. Right now it straddles CPU / GPU paradigm and doesn't get the most out of either.
there is a thing I want to do that should be possible with memory layout hacking but idk how to do it yet
there is a thing I want to do that should be possible with memory layout hacking but idk how to do it yet
first quant results coming in on my simulated data.
~0.12m of absolute trajectory error in the camera motion, over ~20m with only a single (!) keyframe.
very promising.
~0.12m of absolute trajectory error in the camera motion, over ~20m with only a single (!) keyframe.
very promising.
my graphs, let me show you them
this is the camera moving from north to south in the image (-z = forward in this setup for reasons, don't worry just roll with it). the tiny red tabs are how far away I was from the true camera position for that frame.
the little black dots at the end are untracked frames
the little black dots at the end are untracked frames
just got the first object
0.13m over the same ~20m section. This is very good, it's almost within error bars for the camera motion itself.
0.13m over the same ~20m section. This is very good, it's almost within error bars for the camera motion itself.
