Loading...

Multi-Sensor Data Fusion with Deep Learning in Semantic Segmentation

Sadeghi, Aryan | 2022

508 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 55232 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Kasaei, Shohreh
  7. Abstract:
  8. In image processing applications, sensors (Camera, LiDAR and Stereo) are essential for scene perception and Deep learning methods outperform most of the image processing tasks like 3D and 2D object detection and semantic segmentation. Different sensors are used in image processing tasks. Sensor fusion is using multiple sensors data to get better performance. Each sensor captures different data (e.g, color, texture, and depth). Some of them are distorted in inclement weather, intense illuminance changes, and dark environments which multi-sensor data fusion is used to overcome sensor weaknesses. One of the most important fields that sensor fusion used is Auto Driving cars (AD). Different modalities of sensors data are a challenge of sensor fusion. There are few types of research that directly tackle the difference in RGB and depth modalities different fusion module and their good results are the superiority of previous works. Proposed fusion module used to solve sensor’s different modality challenges by multi-task model. This method is real-time and the computational load of multi-tasking does not affect the main branch. Run-time complexity and labeled data set are limitations of the proposed method. The proposed method is supervised thus labeled data are necessary. Cityscapes and Lost and Founds data sets are used for semantic segmentation with RGB-D fusion in this research. Experiments show the superiority of the proposed method to previous methods. Mean intersection over union improved by 1,4% and the run-time complexity in prediction mode is 15 frames per second.
  9. Keywords:
  10. Deep Learning ; Semantic Segmentation ; Stereo Camera Images ; Multi-Sensor Data Fusion Network ; Image Processing

 Digital Object List

 Bookmark

...see more