SOLD²: Self-supervised Occlusion-aware Line Description and Detection

Rémi Pautrat*1
Juan-Ting Lin*1
Viktor Larsson1
Martin R. Oswald1
Marc Pollefeys1 2

1 ETH Zurich
2 Microsoft Mixed Reality and AI Zurich lab
* equal contribution

CVPR 2021 (Oral)




Our Self-supervised Occlusion-aware Line Description and Detection (SOLD²) is a deep line detector and descriptor able to match line segments that are partially occluded. It is the first self-supervised line detector that can be trained on any unlabelled dataset. Both detector and descriptor reach state-of-the-art performance on visual localization metrics.


Compared to feature point detection and description, detecting and matching line segments offer additional challenges. Yet, line features represent a promising complement to points for multi-view tasks. Lines are indeed well-defined by the image gradient, frequently appear even in poorly textured areas and offer robust structural cues. We thus hereby introduce the first joint detection and description of line segments in a single deep network. Thanks to a self-supervised training, our method does not require any annotated line labels and can therefore generalize to any dataset. Our detector offers repeatable and accurate localization of line segments in images, departing from the wireframe parsing approach. Leveraging the recent progresses in descriptor learning, our proposed line descriptor is highly discriminative, while remaining robust to viewpoint changes and occlusions. We evaluate our approach against previous line detection and description methods on several multi-view datasets created with homographic warps as well as real-world viewpoint changes. Our full pipeline yields higher repeatability, localization accuracy and matching metrics, and thus represents a first step to bridge the gap with learned feature points methods. Code and trained weights are available at


Website template from Minyoung Huh and Richard Zhang.