Rémi Pautrat

I am a research scientist in the Mixed Reality & AI Lab of Microsoft in Zurich.

Prior joining Microsoft, I did my bachelor at Ecole polytechnique in Paris and my master in Computer Science at ETH Zurich. I then conducted my PhD thesis in the Computer Vision and Geometry (CVG) group at ETH Zurich under the supervision of Prof. Dr. Marc Pollefeys. I also collaborated with and interned at Meta Reality Labs, Leica Geosystems, INRIA Nancy - Grand Est, and MetraLabs.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn

profile photo
Research

I am interested in Computer Vision, Robotics and Deep Learning in general. My research focuses on local feature detection and description based on deep learning, and their applications to visual localization and sparse 3D reconstruction.

Handbook on Leveraging Lines for Two-View Relative Pose Estimation
Petr Hruby, Shaohui Liu, Rémi Pautrat, Marc Pollefeys, Dániel Béla Baráth,
International Conference on 3D Vision (3DV), 2024 (Spotlight)
arXiv

A complete classification of all solvers for relative pose estimation based on point and line correspondences.

GlueStick: Robust Image Matching by Sticking Points and Lines Together
Rémi Pautrat*, Iago Suárez*, Yifan Yu, Marc Pollefeys, Viktor Larsson,
International Conference on Computer Vision (ICCV), 2023
project page / code / arXiv

A joint point-line matcher with graph neural networks.

Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction
Rémi Pautrat, Shaohui Liu, Petr Hruby, Marc Pollefeys, Dániel Béla Baráth,
International Conference on Computer Vision (ICCV), 2023
project page / code / arXiv

Solvers to extract the 3 orthogonal vanishing points of an uncalibrated image (i.e. unknown focal length), given a prior on the gravity direction.

3D Line Mapping Revisited
Shaohui Liu, Yifan Yu, Rémi Pautrat, Marc Pollefeys, Viktor Larsson,
Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)
project page / code / arXiv

An open-sourced system that robustly and efficiently constructs 3D line maps from multi-view images.

DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients
Rémi Pautrat, Dániel Béla Baráth, Viktor Larsson, Martin R. Oswald, Marc Pollefeys,
Computer Vision and Pattern Recognition (CVPR), 2023
project page / video / code / arXiv

A generic line detector that combines the robustness of deep learning with the accuracy of handcrafted detectors.

SOLD²: Self-supervised Occlusion-aware Line Description and Detection
Rémi Pautrat*, Juan-Ting Lin*, Viktor Larsson, Martin R. Oswald, Marc Pollefeys,
Computer Vision and Pattern Recognition (CVPR), 2021 (Oral)
project page / video / code / arXiv

A deep line detector and descriptor able to match line segments partially occluded.

Online Invariance Selection for Local Feature Descriptors
Rémi Pautrat, Viktor Larsson, Martin R. Oswald, Marc Pollefeys,
European Conference on Computer Vision (ECCV), 2020 (Oral)
project page / teaser / oral / code / arXiv

A learned feature descriptor able to adapt its invariance to illumination and rotation at matching time.

Object Finding in Cluttered Scenes Using Interactive Perception
Tonci Novkovic*, Rémi Pautrat*, Fadri Furrer, Michel Breyer, Roland Siegwart, Juan Nieto,
International Conference on Robotics and Automation (ICRA), 2020
project page / teaser / oral / arXiv

We leverage reinforcement learning and computer vision to perform interactive perception: a robot manipulator has to find a hidden target object in a scene by interacting with its environment.

Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search
Rémi Pautrat, Konstantinos Chatzilygeroudis, Jean-Baptiste Mouret,
International Conference on Robotics and Automation (ICRA), 2018
video / arXiv

We propose a new acquisition function for Bayesian Optimization that combines the likelihood of prior information with the expected improvement. We apply it to the task of damage recovery in robotics and automatic adaptation to new environments.


Template gratefully borrowed from Jon Barron's personal website.