Velodyne LiDAR and monocular camera data fusion for depth map and 3D reconstruction

M. Rayyan Akhtar; Huabiao Qin; Guancheng Chen

doi:10.1117/12.2539863

14 August 2019 Velodyne LiDAR and monocular camera data fusion for depth map and 3D reconstruction

M. Rayyan Akhtar, Huabiao Qin, Guancheng Chen

Proceedings Volume 11179, Eleventh International Conference on Digital Image Processing (ICDIP 2019); 111790E (2019) https://doi.org/10.1117/12.2539863
Event: Eleventh International Conference on Digital Image Processing (ICDIP 2019), 2019, Guangzhou, China

Abstract

The autonomous vehicles are required to perceive the environment to take a correct driving decision. The sensors which have been commonly used by autonomous vehicles are the camera and the Light Detection and Ranging (LiDAR). In this work, we have integrated the LiDAR data with the image captured by the camera to assign the color information to the point cloud which resulted in a 3D model and to assign depth information to the image pixels which resulted in a depth map. The LiDAR data is sparse and the resolution of the image is much greater than that of the LiDAR data. In order to match the resolution of the LiDAR data and image data, we had utilized Gaussian Process Regression (GPR) to interpolate the depth map but it was not able to completely fill the empty locations in the depth map. In this paper, we have proposed a method to interpolate the 2D depth map data to completely fill the empty locations in the depth map. In this study, we have used Velodyne VLP-16 LiDAR and a monocular camera. Our method is based on the covariance matrix in which the depth value assigned to the empty locations in depth map is decided according to the value of covariance function in the covariance matrix. Our method surpassed the GPR in run time and interpolation result. This shows that our approach is fast enough in real-time for autonomous vehicles.

Citation Download Citation

M. Rayyan Akhtar, Huabiao Qin, and Guancheng Chen "Velodyne LiDAR and monocular camera data fusion for depth map and 3D reconstruction", Proc. SPIE 11179, Eleventh International Conference on Digital Image Processing (ICDIP 2019), 111790E (14 August 2019); https://doi.org/10.1117/12.2539863

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available