Panorama Sitching
Introduction
Photographers obviously spend a lot of time thinking about and trying to capture the perfect perspective for their photographs. Sometimes, however, certain vantage points are unreachable due to obstructions, space constraints, or lens limitations (such as zoom). Creating panoramic composite images from multiple photos is one way to circumvent a few of these restrictions. Below is a method of panoramic stitching that I have implemented in MATLAB.
Methodology
Given a set of N images, pairs of which contain overlapping segments of a scene, we would like to merge and blend all of these images together to result in out continuous scene in one image.
For each pair of images, we first compute Harris corners of both images. Next, we calculate Normalized Cross Correlation (NCC) scores for each corner in the left image against all corners in the right image. Given these scores, we find the pairs with the highest scores and use these matchings for the homography to warp the second image into the coordinates of the first. RANSAC is used to remove outliers from these pairs that would result in inconsistent mappings.
Once the second image is warped into the space of the first image, the steps above are repeated for each consecutive image, constantly warping images onto the previous solution. The limitation of this design is that the order of which the files are fed into the program is significant. Each consecutive image must overlap some portion of either the image prior to it, or the known solution thus far.
An archive of the code and sample images can be downloaded below. To test, execute mosaic.m.
Results
The following set of images were taken handheld and contain slight rotation transformations as well as translation. This method produces great results when stitching two images together. However, results degrade with the more images added to the panorama.
Test #1
 |
 |
 |
 |
Test #2
 |
 |
 |
Limitations
As you can tell by the examples below, this implementation can get bad results when computing many consecutive homographies. These poor results occur when a pair of the image matches contain a significantly low number of mathing Harris corners (the 5th mathing image below). Once this occurs, the program does not recover.
Corner Matching
 |
 |
 |
 |
 |
 |
Conclusion
I started this project with the goal of implementing Brown and Lowe’s simplified two band blending technique which blends high frequency structures and low frequencies separately. However, due to time (and brain power) limitations, I resorted to the linear blending technique that is currently implemented. However, while the blending implementation was greatly simplified, much of the time commitment was refocused on my own implementation of Harris Corner detection and matching as well as the projection transformations between image coordinates.