# Video Stitching and Processing

Computational photography project 5.

(click to enlarge)

Figure 1: Seamless panorama stitching. The algorithm finds the minimum cost path between the pairs of intersection points over a cost map $(proj_1-proj_2)^2 + foreground_1^3 + foreground_2^3$ in order to parse the foreground objects. More on the bottom of the page (link).

## Stitch two key frames

Figure 2: This one also uses seamless blending. The rest of this page uses the plain maximum pixel value blending.

## Map the video the reference plane

For this part, I implemented automatic failure detection. My approach was to first compute the correct homography for a set of frames that are known to be stable, then try all nearest $k$ frames upon failure. Failure is detected when the SSD of the coordinates of the transformed frame significantly differs from the average of previous three values. After a certain number of tries, it will start increasing the threshold.

## Create background panorama

I computed the mean pixel values ignoring pixels outside the frame. Computing the median took too long on my machine, but I think that would have yielded better results. The aberration-like (?) artifacts near the top left corner have to do with trying to remove the edges between frames (this is just max pixel blending).

## Create background movie

Figure 3: The background panorama mapped back onto the original frame coordinates.

# Bells and whistles

## Smooth blending

Figure 4: Minimum cost paths

Figure 5: Implemented using dynamic programming. To compare with the naive blending method, scroll up or click here.

Figure 6: The starting and ending points

## Generate a wider video

Figure 7: 50% wider width.

## Process two more videos - video #1

Original video can be found here: https://www.youtube.com/watch?v=7kUPvIxnR-8. (CC BY)

### Map the video the reference plane

Having the automatic re-try mechanism was useful for this one because the camera moves a lot in the last two seconds which made it difficult to find a smooth mapping in the first try.

## Process two more videos - video #2

### Stitch two key frames

U of I Main Quad last year.

### Map the video the reference plane

The camera rotates 180 degrees.