parking

source japanautomotivedaily

Summary of main ideas and points from some of the reading of research papers needed to implement parking system of autonomous cars using deep learining while fusing several different sensors on the car. This is project in colaboration with DAI-LABOR and be-intelli which goal is to build end to end system for autonomous parking or build deep learning system for sensor fusion.

One of the final goals before formulation the exact proposal of the project is to run demo og Autonomous Valet parking using ROS2 and Autoware.auto and LGSVL simulator.

Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges. Di Feng et al

Brief overview

The aim of this paper is to provide short summary of existing methods and corresponding challenges which needs to be addressed when developing autonomous driving system. It provides a short description of the most important concepts for begginers to start in the topic of autonomous driving. The main contribution is in my view their interactive online platform availible at https://boschresearch.github.io/multimodalperception/

Background

There is a high requirement for realible autonomous system as these vehicles are critical to fight with traffic congestion, and reduce current transportation carbon footprint. This is a very diffuclt task for several reasons. In order for a vehicle to operate driveless it needs to be able to perfectly percieve, predict and execute all its decision in noisy, highly unpredictible and real enviroment. Even the smallest errors can have catastrophical consequences and therefore minimizing any errors in multimodal autonomous systems is crucial.

The paper introduces main requirements for such a system as follow:

  1. accurate
  2. robust
  3. real-time

In next the paper is docusing on deep leraning methods for object detection and semantic segmentation, called deep multi-modal perception. The as well sumarize all topics related to developing autonomous systems such as typical automotive sensors, research purposes and some existing vehicles for testing these systems. The paper is also very useful tool of comprehensive overview of each mentioned topic and corresponding research to each of these topics.

There are 6 main sensing modalities in autonomous systems and each of them has its own advantages and shortcommings:

  1. Visual and Thermal Cameras
    • Pros: provides detailed texture information, low cost
    • Cons: sensitive to lightning and weather conditions, no depth information
  2. LiDARs
    • Pros: accurate depth information, robust to lightning and weather changes
    • Cons: no texture infrmation and sprase data points the further the object is
  3. Radars
    • Pros: robust to changing lightning and weather
    • Cons: low resolution
  4. Ultrasonics
    • Pros:
    • Cons: affected by humidity temperature and dirt,
  5. GNSS and HD Maps
    • Pros:
    • Cons:
  6. IMU and Odometers
    • Pros:
    • Cons:

The paper mention some of the peineering vehicles in the autonomous driving domain and it sensors with which it was able to achieve these impressive results. The first mentioned is “Boss” that won famos DARPA challenge in 2007 with camera and several Radars and LiDAR sensors. More leading companies in autonomous space are Google (Waymo), BMW, Daimler.

Next comes summary of main methods for perception based system. The paper speaks about deep object detection, which is a task of localization and rocognition of objects in the image. Usually these are recognized by it’s category probability and localizaed by bounding boxes. The best approach to solve object detection to date is deep learning algorithms, which set benchmarks on popular datasets such as PASCAL VOC and COCO. There are two approaches to deep object detection networks: 1. Two-stage detection - In first stage, regions of interest (ROI) are extracted from the image, then these candidates are verified, classified and refined based on classification scores and locations. Exaples of such a system: OverFeat, R-CNN, SPPnet, Fast-RCNN (VGG. ResNet, GoogLeNet)

  • pros: better accuracy
  • cons: higher inference time and more time needed for training

2. One-stage detection - maps feature maps directly to bounding boxes and classification via single-stage, unified CNN model. As example authors metioned MultiBox, YOLO, SSD

  • pros: faster, easier to optimize
  • cons: underperform in terms of accuracy to two-stage detectors

Another approach to obejct detection is deep semantic segmentation. Here the goal is to partition the scene into separate parts/object by labeling each pixel in the image with semantics or detecting objects and at the same time …..

Authors also give good overview of availible datasets used in ….