Rémi Pautrat

I am a senior research scientist in the Spatial AI Lab of Microsoft in Zurich.

Prior joining Microsoft, I did my bachelor at Ecole polytechnique in Paris and my master in Computer Science at ETH Zurich. I then conducted my PhD thesis in the Computer Vision and Geometry (CVG) group at ETH Zurich under the supervision of Prof. Dr. Marc Pollefeys. I also collaborated with and interned at Meta Reality Labs, Leica Geosystems, INRIA Nancy - Grand Est, and MetraLabs.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn

I am interested in Computer Vision, Robotics and Deep Learning in general. My current research aims to improve Vision Language Models (VLMs) pre-training and how to infuse spatial understanding into these models. I have also a strong track of records and experience in the topics of local feature detection and description based on deep learning, and their applications to visual localization and sparse 3D reconstruction.

CoPE-VideoLM leverages codec primitives to efficiently process videos for large video language models.

A fast joint point-line matcher with graph neural networks.

New solvers for relative pose estimation leveraging monocular depth priors and affine corrections.

An open-sourced system that runs Structure-from-Motion (SfM) with points, lines, and vanishing points features.

A learned approach to estimate an edge density and extract 3D edges from a scene.

A complete classification of all solvers for relative pose estimation based on point and line correspondences.

A joint point-line matcher with graph neural networks.

Solvers to extract the 3 orthogonal vanishing points of an uncalibrated image (i.e. unknown focal length), given a prior on the gravity direction.

An open-sourced system that robustly and efficiently constructs 3D line maps from multi-view images.

A generic line detector that combines the robustness of deep learning with the accuracy of handcrafted detectors.

A deep line detector and descriptor able to match line segments partially occluded.

A learned feature descriptor able to adapt its invariance to illumination and rotation at matching time.

We leverage reinforcement learning and computer vision to perform interactive perception: a robot manipulator has to find a hidden target object in a scene by interacting with its environment.

We propose a new acquisition function for Bayesian Optimization that combines the likelihood of prior information with the expected improvement. We apply it to the task of damage recovery in robotics and automatic adaptation to new environments.

Template gratefully borrowed from Jon Barron's personal website.