Lab 6: Sensor Fusion

By now, we have used several sensors to allow the car to understand its place in its environment. Several of these sensors can be used to calculate the car’s velocity, but on their own, no particular method is particularly accurate.

Sensor fusion is the process of combining several uncertain sources of truth to create a more reliable source of truth. In this lab, we will perform sensor fusion to calculate a more accurate velocity measurement for the car.


Main objective

Use sensor fusion to calculate an accurate measurement of velocity.

Learning objectives

  • Explore several ways of calculating velocity with the car’s hardware.
  • Identify the strengths and limitations of each approach.
  • Use accurate velocity measures to improve past features.


Warm Up: Calculating Velocity

In order to perform sensor, we will need several independent sources of velocity information. Suppose that the car is driving straight towards a wall with a red block in front of it. Replace each TODO (warm up) comment with an independent method of calculating velocity.

  1. Estimate velocity from the input to the Drive module.
  2. Estimate velocity from the color image captured by the Camera module.
  3. Estimate velocity using the depth image captured by the Camera module.
  4. Estimate velocity using the Physics module.
  5. Estimate velocity using the LIDAR module.

Main Challenge: Combining Velocity

Consider the strengths and limitations of each method of you identified in the previous section. Under what circumstances would each method be particularly unreliable?

Find a way to combine these measurements into a single velocity estimate. In general, a measurement should be given greater weight if it is more reliable. How can we estimate the reliability of a measurement?

Application 1: Speed Limit

Replace the TODO (application 1) comment with code to prevent the car from traveling over 2 meters per second when the A button is pressed.

Application: Safety Stop Revisited

In Lab 3: Depth Images, we implemented a safety stop feature to prevent the car from colliding with objects in front of it. At the time, our approach did not take into account the velocity of the car. However, the faster the car is traveling, the sooner it will need to start breaking to stop hitting a wall.

Replace the TODO (application 2) comment with an improved safety stop feature which takes into account the car’s velocity.