Running DSO on 7-Scenes?

[djkeyes]

Has anyone tried running DSO on Microsoft Research's 7-Scenes dataset?

I am trying to do so. I am using a camera calibration from a separate SfM system (colmap), and I am running in mode 1 (no photometric calibration) on the RGB images. I have encountered several problems though, so I am wondering if anyone else has run into similar issues.

For one, some parts of 7-Scenes contain fairly strong rotations with little translation. Clearly this is a failure case for DSO, but is it possible to recover after the rotation, from the pixels that remained in the frame the whole time? I have tried increasing the maximum number of keyframes and number of active points, so that more pixels with valid depth values remain in the frame, but I find that this does not help consistently.

I also find that sometimes the tracking fails (resulting in dso_dataset crashing) even on trajectories with little rotation, and I'm not sure why. Could the coarse tracker be failing to get a good coarse estimate?

Thanks

[MorganChabaud]

I did not try the Microsoft dataset.

I am using a camera calibration from a separate SfM system

This seems odd to me, is it a camera calibration for the mentioned Microsoft dataset? Because it has to, you cannot just take a random calibration file.

For the recovery after a strong rotation... Considering it is strong, you will not have much points overlapping and as no translation is present, DSO can't estimate depth for new candidate points. DSO will keep a number of points for the tracking (2000 by default). And even if you have 50 good points, the photometric error will be huge thus making these perfectly tracked 50 points useless in the end. When a new image is processed, the algorithm reprojects tracked points onto it (given several pose initial estimates) and try to minimize the photometric reprojection error in order to find the new pose and if the rotation is too strong, the local minimum won't lead to a good image alignment

The issue of rotation only movement is really related to monocular camera odometry (without any IMU or other sensors) and as said somewhere in the readme, DSO can't do magic. A rotation only don't provide depth information on points. And if you end up seeing a scene without depth information, DSO will stop and it will not try to recover (as it is only visual odometry). I wonder if an initialization (or alike) step in the middle of the algorithm would work... I guess it could but this has yet to be implemented.

The dso_dataset crashing issue may be related to #81, which states that DSO will crash if no idepth is available on an image level.

Hope it helps,

[NikolausDemmel]

The questions seems answered. Closing due to inactivity. Feel free to comment and we can reopen if needed.

[rui2016]

Hi @chabaudmorgan:

by "calibration from a separate SfM system" I assume @djkeyes meant that he estimated the intrinsic parameters of the RGB camera used in 7Scenes using colmap, as they are not provided by the 7Scenes dataset.