We test algorithms for 3D reconstruction from a single image specifically for building envelopes. This research shows the current limitations of these approaches when applied to classes outside of the initial distribution. We tested solutions with differentiable rendering, implicit functions, and other end–to–end geometric deep learning approaches. We recognize the importance of gener-ating a 3D reconstruction from a single image for many different industries, not only for Architecture, Engineering, and Construction (AEC) industry but also for robotics, autonomous driving, gaming, virtual and augmented reality, drone delivery, 3D authoring, improving 2D recognition and others.

With the growing interest in deep learning algorithms and computational design in the architectural field, the need for large, accessible and diverse architectural datasets increases. We decided to tackle this problem by constructing a field-specific synthetic data generation pipeline that generates an arbitrary amount of 3D data along with the associated 2D and 3D annotations.

We present RackLay, a deep neural network for real-time monocular multi-layer layout estimation, for warehouse racks. We present applications like affordance estimation and 3D reconstruction.