Video Stitching and Processing

Computational photography project 5.

(click to enlarge)

Figure 1: Seamless panorama stitching. The algorithm finds the minimum cost path between the pairs of intersection points over a cost map in order to parse the foreground objects. More on the bottom of the page (link).

Stitch two key frames

Figure 2: This one also uses seamless blending. The rest of this page uses the plain maximum pixel value blending.

Panorama using five key frames

Map the video the reference plane

For this part, I implemented automatic failure detection. My approach was to first compute the correct homography for a set of frames that are known to be stable, then try all nearest $k$ frames upon failure. Failure is detected when the SSD of the coordinates of the transformed frame significantly differs from the average of previous three values. After a certain number of tries, it will start increasing the threshold.

Create background panorama

I computed the mean pixel values ignoring pixels outside the frame. Computing the median took too long on my machine, but I think that would have yielded better results. The aberration-like (?) artifacts near the top left corner have to do with trying to remove the edges between frames (this is just max pixel blending).

Create background movie

Figure 3: The background panorama mapped back onto the original frame coordinates.

Create foreground movie

Insert an unexpected object in the video

Bells and whistles

Smooth blending

Figure 4: Minimum cost paths

Figure 5: Implemented using dynamic programming. To compare with the naive blending method, scroll up or click here.

Figure 6: The starting and ending points

Generate a wider video

Figure 7: 50% wider width.

Process two more videos - video #1

Original video can be found here: https://www.youtube.com/watch?v=7kUPvIxnR-8. (CC BY)

Stitch two key frames

Panorama using five key frames

Map the video the reference plane

Having the automatic re-try mechanism was useful for this one because the camera moves a lot in the last two seconds which made it difficult to find a smooth mapping in the first try.

Create background panorama

Create background movie

Create foreground movie

Process two more videos - video #2

Stitch two key frames

U of I Main Quad last year.

Panorama using five key frames

Map the video the reference plane

The camera rotates 180 degrees.

Create background panorama

Create background movie

Create foreground movie