Thread by @atroyn

gonna go out tonight then hermit up for the long weekend to get some computer things done

bunch of half finished things laying around and for the first time in a while I won’t need to function like a human being so I can pull some all nighters

gonna get mad at the computer i bet

i want to make a significant push on the structure from motion for rigid bodies thing because im sick of it only being in my head

approach is, do the feature based thing with RANSAC’d bundles of motions like in the Oxford paper and use that as initialization, then do joint optimization with photometric costs

this is roughly one long weekend’s work

replicating the feature thing should be about an entire day on its own, i don’t want to have to do it because features are fake but i don’t currently have a better idea about how to do this that doesn’t require the invention of a lot of new machinery

also have some poasts ive been meaning to write and also to put my zotero online but this thing is first

stereo for depth initialization to start but i think I can do this in a mono setting

this is absolutely because i dislike the way my brain is right now and want to apply a shock to fix it

today we are making this.

it is gross and i hate it.

please understand how much i hate this

feature matching and its consequences have been a disaster for the robot race

wish i was making video games

grinding through tracklet associations god DAMN do i hate feature based methods

of course the paper lied about how they did the tracklet associations by leaving it ambiguous

you motherfuckers

this looks correct enough

time to execute on this plan:

https://twitter.com/atroyn/status/1168349966621741057?s=20

this took so much longer than it should have holy shit

at least tomorrow is just about writing down and minimizing labeling energies but holy shit

i had to write an n^2 implementation of this dumb bullshit because my brain recoils at having to do this efficiently let alone at all

it is all alpha expansion from here lads

except for the RANSAC part

lol the whole tracklets thing was totally unnecessary I am dumb

should have gone straight to the graph segmentation right away

going to stop implementing this paper's pipeline and go back to doing what i actually set out to do in the first place

for clarity, i am bypassing a problem in my original approach (detailed here: troynikov.io/dynamic-photom…) where even modest relative rotations put us too far away from the initialization point, by using feature matching instead of gauss-newton on photometric costs

my approach to bypassing it is to estimate rigid body transforms between frames from feature correspondences and segment these by recursive inlier/outlier segmentation with cuts on a two-label graph.

then we will use this as the initialization for photometric costs

we'll use rough depth from stereo (SPS-Stereo for those playing at home) to initialize depths, for a keyframe.

Subsequently we will jointly estimate the depth and motion in a completely direct framework

turns out i really do have to implement RANSAC PnP myself, it was a mistake to believe that either OpenCV or Matlab would have done the nice thing for me

in good news i think all of this put together means i can get rid of the gross semantic segmentation i had before

got inlier/outlier segmentation for motion from PnP working so that's enough for today i think

way behind where i wanted to be this weekend but progress in any case

naive segmentation just taking the RANSAC outliers

we could probably refine this but i'm going to choose not to and press on to the next phase, i want to get to a photometric cost as fast as possible