This is our fourth year design project. The objective of this project is to design a prototype stereo camera system that provides depth information for use in Advance Driver Assistance Systems(ADAS).

Assuming perfect matching between the stereo images, accuracy in stereo systems decreases quadratically with distance due to the fact pixels are discrete. In addition, distance can only be triangulated in the regions of the image where two cameras have overlapping field of view. Accuracy can be increased in two ways: increase the baseline distance between two stereo cameras or decrease the cameras’ field of view (FOV). Both of which decreases the overlapping region, thus increasing the minimum distance where detection is possible.

We attempted to solve this problem by using a duo-baseline setup with three cameras. One baseline is between the outermost cameras at a distance of 0.6 meters which provides the greatest accuracy, but it only works at longer distances. The other baseline is 0.2 meters between the two cameras on the left which is used for close range. We chose 1.3MP USB3 global shutter cameras from Point Grey, a NVIDIA Jetson TX1 for processing. In addition, we designed custom PCBs for supplying power and provide hardware synchronization for the cameras.

The camera system is calibrated for lens distortions as well and the position and orientations between cameras. Then, stereo images from two baselines are matched to generate a depth image. We selected Semi-Global Matching (SGM) on the Jetson TX1 GPU for stereo matching. SGM offers a good trade off between speed and accuracy. Finally, depth from the two baselines are fused according to their optimal distances.

We tested the stereo camera system on a car by mounting it behind the windshield and drove the car around the university. Note that the project focused on generating depth information, the car and pedestrian detections were added last minute addition to demo how the depth information could be used, and thus there are lots of false detection. The depth results are not very good around the edges due to calibration and cheap low quality lens ($7 each) we used.