Model Selection for Surface Approximation and Scene Interpretation
- Facade components around a window (left) and around a door (right).
- © CV
We want to combine surface reconstruction from images of urban facades and the interpretation and compact description of the scene. We will start by deriving dense point clouds from unordered facade image sets by current image orientation methods, and a further dense reconstruction. At the core of our project proposal is an iterative process which combines both tasks, i.e., the surface reconstruction and the scene interpretation. Each task is very difficult due to noise in the reconstructed point clouds and false or missing points caused by repetitive facade patterns, reflecting object surfaces, e.g., at windows, or insufficient illuminations, e.g., at building entrances or in shadow areas. By combining both tasks, we are confident to improve the results, because we consider the result of one task from the previously determined step as prior information of the other. Additionally, model selection will be employed to derive optimal solutions for specific subtasks.
A geometrical surface representation will be derived that completely describes a 2D manifold in 3D space. It will consist of parts of various detected geometric primitives, e.g., planes and cylinders, whose spatial relationships are explicitly described by an adjacency graph. Semantic information is included as prior for a more reliable estimation particularly regarding the roughness of the surface. Reversible Jump Markov Chain Monte Carlo (rjMCMC) model selection is employed within a Bayesian framework to find a globally optimal surface reconstruction.
The facade analysis is initialized with detecting and delineating windows, because they are the most dominant facade part. Together with the derived approximation of the surface, candidates for doors are found and verified. Subdivisions of windows and doors are used to verify the semantic interpretation. Model selection is employed within these subtasks to find correct type and form of the facade parts. Aggregates around doors and windows are the basis to detect other facade parts, e.g., canopies or window shutters. All detected facade parts are modelled in a 2D facade grammar of 3D objects.