The input to the Stereo Pipeline consists of images from a calibrated stereo pair. The first step in processing the input stereo pair is referred to as preprocessing. First, optional cropping and subsampling can be performed on a selected region of interest. Histogram equalization and edge enhancement can then be performed to improve downstream correlation performance. Finally, the input stereo pair is rectified to account for systematic epipolar and rotational offsets in the stereo pair.

The second step in the pipeline correlates the stereo pair, generating a dense disparity map. A three-pass correlation approach is used to improve both speed and accuracy. The first pass of the correlator bounds the disparity values in the input image. These bounds are used to limit the disparity search interval in the second pass, resulting in less computation and lower false match probability. The second correlation pass creates a dense disparity map using large window correlations. Disparity search intervals in the third and final pass are constrained to the neighborhood of the disparity calculated in the previous pass. A single correlator, with different parameters, is used for the all three passes. Depending on the pass, parameters are optimized either for speed, percentage of matches or accuracy of the matches. A texture-based sum-of-absolute-difference (SOAD) correlation algorithm is used, and the consistency of each match is validated by doing both a correlation and cross-correlation. Inconsistent matches are discarded. The correlation is followed by a filtering step to remove outlier matches. Gaps in the disparity map are then filled. Gaps, in general, are the result of real-world discontinuities in surface shape, such as the occluding boundaries of rocks in the terrain. In order to retain these boundaries in the map, gaps occurring at large discontinuities are filled with the minimum disparity value in the gap neighborhood. Gaps in regions with small disparity variance are more likely due to a smooth, texture free surface; these gaps are filled by averaging. A first-order function is used to offset the disparity estimates from the correlation phase to account for relative error in focal lengths between the left and right lens of the stereo pair.

The final stages derive the final texture-mapped terrain mesh from the dense disparity step. First, a local horizontal frame elevation dotcloud is computed from the dense, filtered disparity map. Range values in the camera frame are calculated based on pinhole optical model and CCD imager specifications. These range values are transformed into the local frame based on a defined reference transformation defining the camera's pose. This set of points is converted into a triangle mesh by pairing adjacent camera frame pixels together. The meshing algorithm preserves discontinuities in the surface by limiting the depth range of a given triangle. The original image is then textured onto the mesh.