Tutorials

2024

1. Deep Stereo Matching in the Twenties Poggi Matteo, and Tosi Fabio (CVPR) [Abs] [HTML]

   COMING SOON...

2020

1. Facing depth estimation in-the-wild with deep networks Poggi Matteo, Tosi Fabio, Aleotti Filippo, Batsos Konstantinos, Mordohai Philippos, and Mattoccia Stefano (ECCV) [Abs] [HTML]

   Obtaining dense and accurate depth measurements from images is of paramount importance for many 3D computer vision applications. In the last years, stereo and monocular depth estimation have become the most popular techniques for this purpose, with deep neural networks improving consistently over state of the art. However, in contrast to hand-made algorithms, deep learning solutions are particularly data dependent. Indeed, better performance is achieved by training on large and diverse datasets, either real or sourced through computer graphics. Nevertheless, deep networks always suffer non-negligible drops in performance when moving to different domains. These drops in performance could be catastrophic, but also likely to occur when a deep network is deployed in-the-wild. Therefore, in this tutorial, we aim at highlighting the limitations of deep neural networks for stereo and monocular depth estimation and how far we are from their unconstrained deployment in the wild. Then, we will introduce very recent practices aimed at shrinking the gap in performance between academic datasets and the real world.

2. Learning and understanding single image depth estimation in the wild Poggi Matteo, Tosi Fabio, Aleotti Filippo, Mattoccia Stefano, Godard Clément, Watson Jamie, Firman Michael, and Brostow Gabriel J. (CVPR) [Abs] [HTML]

   Depth estimation from a single still image, often referred to as depth-from-mono, although considered for a long time barely feasible, with the advent of deep learning stood out as a viable and effective alternative to more complicated setups as witnessed by the compelling results achieved in very recent years. In this context, particularly appealing is the possibility of learning monocular depth estimation through geometry, replacing the need for ground truth depth annotation (hard to source and limiting practical deployment) with multiple images of the same scene acquired from different viewpoints, in self-supervised manners. This research topic has been extraordinarily active in the last three years, bringing a vast and ever-increasing number of papers and novel contributions published in top-level computer vision venues. For these reasons, in this tutorial, we aim at giving a comprehensive overview of advances in self-supervised depth-from-mono research, highlighting the rapid evolution of this topic and how fast its popularity is growing and, nonetheless, the open challenges. Through the tutorial, we will introduce the audience to the topic of monocular depth estimation, pointing out the main strategies to source self-supervision from images in order to replace traditional supervision from ground truth labels, then exploring additional forms of weak supervision obtained by means of hand-made algorithms or multi-task frameworks. We plan a hands-on session dedicated to deployment on smartphones, discussing tools and practices to run monocular depth estimation on the most popular devices. To conclude, we will give an overview of open challenges by discussing recent works about uncertainty estimation and the interpretability of what neural networks learn to estimate depth from single images.

2019

1. Learning-based depth estimation from stereo and monocular images: successes, limitations and future challenges Poggi Matteo, Tosi Fabio, Batsos Konstantinos, Mordohai Philippos, and Mattoccia Stefano (CVPR) [Abs] [HTML]

   Obtaining dense and accurate depth measurement is of paramount importance for many 3D computer vision applications. Stereo matching has undergone a paradigm shift in the last few years due to the introduction of learning-based methods that replaced heuristics and hand-crafted rules. While in early 2012 the KITTI dataset highlighted how stereo matching was still an open problem, the recent success of Convolutional Neural Networks has led to tremendous progress and has established these methods as the undisputed state of the art. Similar observations can be made on all recent benchmarks, such as the KITTI 2012 and 2015, the Middlebury 2014 and the ETH3D benchmark, the leaderboards of which are dominated by learning-based methods. The tutorial will cover conventional and deep learning methods that have replaced the components of the conventional stereo matching pipeline, end-to-end stereo systems and confidence estimation. The second part will focus on related problems, specifically single-view depth estimation and multi-view stereo, that have also benefited from the availability of ground truth datasets and learning algorithms. The tutorial will conclude with open problems including generalization as well as unsupervised and weakly supervised training.

2018

1. Learning-based depth estimation from stereo and monocular images: successes, limitations and future challenges Poggi Matteo, Tosi Fabio, Batsos Konstantinos, Mordohai Philippos, and Mattoccia Stefano (3DV) [Abs] [HTML]

   Obtaining dense and accurate depth measurement is of paramount importance for many 3D computer vision applications. Stereo matching has undergone a paradigm shift in the last few years due to the introduction of learning-based methods that replaced heuristics and hand-crafted rules. While in early 2012 the KITTI dataset highlighted how stereo matching was still an open problem, the recent success of Convolutional Neural Networks has led to tremendous progress and has established these methods as the undisputed state of the art. Similar observations can be made on all recent benchmarks, such as the KITTI 2012 and 2015, the Middlebury 2014 and the ETH3D benchmark, the leaderboards of which are dominated by learning-based methods. The tutorial will cover conventional and deep learning methods that have replaced the components of the conventional stereo matching pipeline, end-to-end stereo systems and confidence estimation. The second part will focus on related problems, specifically single-view depth estimation and multi-view stereo, that have also benefited from the availability of ground truth datasets and learning algorithms. The tutorial will conclude with open problems including generalization as well as unsupervised and weakly supervised training.