How to create large panoramic images from onboard camera video.
Introduction:We will begin with a little background information to preface this tutorial. Think of this as a little history lesson. We think a lot of you are like minded and would enjoy reading a bit of history.
We have always had and interest in photography as a separate hobby and career path (for some of our members), and the gorgeous images which our human made remote probes send back have always been a source of inspiration to us. We have been emulating this by putting cameras on our water rockets to capture images of our own area, and every flight reveals something new and exciting. The feeling of discovery must be very similar, but on a much grander scale for the people in the space programs.
When we first started flying cameras, it was long before Google Earth and all of the other satellite images we find so commonplace today existed, so we were discovering our little corner of this planet in very much the same way a group of soldiers and scientists in the New Mexico desert saw the first pictures of Earth as seen from space when they launched the first camera on a captured V-2 missile.
Our first grainy images taken with some very primitive cameras were quite similar in many regards, but the technology exists today to help improve images in ways they could only dream of in the 1940's. They say necessity is the mother of invention, and NASA developed their photo stitching methods out of necessity.
How they managed to get such breathtaking images from a camera that would be considered a 'toy" intrigued us, and we discovered that the trick involved is incredibly simple: you take a lot of low resolution images of small overlapping areas and then use a computer program to blend them together into a single large image with much more detail. You can picture (no pun intended) this by thinking of making a gigapixel image sensor by putting a lot of low resolution sensors side by side in a grid. The idea is the same.
We were fascinated by this technique, and wanted to improve our grainy little pictures using this same method. Initially, there were no commercially available software packages to automate this process, so we had to painstakingly align and blend individual images by hand using an expensive program called "Photoshop". This produced similar results but at great cost in time and effort.
A few years later some commercially available software to merge images the same way NASA does hit the market, but it was very expensive. We continued to hand blend images to make our panoramas. Eventually, we got a taste of true automated stitching with some "crippleware" trial versions of commercial programs, and from then on we were addicted.
By 2007, public awareness of the technique was being raised by the Google Street View applications, which use the same methods to construct their "first person" views. In addition to that, Adobe Systems soon added a feature called "Photomerge" to their Photoshop Software. We began making our own panoramas using the Photomerge software, and it worked amazingly well. This feature could merge images, correct exposures, warp them to proper alignment, and output a beautiful panorama in minutes. We loved it, and discussed our experiences with it on the MSN group we sponsored (before Microsoft closed down the unprofitable group feature of MSN). Hopefully, we have inspired others to emulate us in the same way we were inspired by NASA. and those groundbreaking pioneers at White Sands Missile Range in the 1940's! ;-)
However, we have come to realize that not everyone can justify purchasing a program like Photoshop for this purpose, so we looked around and discovered that in the past few years since Photomerge has been available, numerous new and free alternatives have come out that work just as well and are much easier to use for the average person. This tutorial explains how to create a panoramic view using some free image stitching software which you may already have on your computer and were not even aware of!
So, we are ready to begin the tutorial. Let's get started!
- Image stitching software
- Video files from onboard camera
Step 1: Getting Free Software:The software we will be using today is part of a package called "Windows Live Essentials" which you can download as a package for free from Microsoft. Your computer may actually have this software pre-installed, or you may have installed it sometime in the past to get one of the other features in the package. You should check and see if you have it already before you bother to download it again.
Don't despair Mac and Linux folks... you can scroll down to the "Conclusion" section at the end for some good news!
The essentials package comes with many applications but the two that interest us for this tutorial are "Windows Live Movie Maker", and "Windows Live Photo Gallery". If you don't already have these installed, you can download the entire package at the following URL:
When you go to install the package, you will probably want to only install "Windows Live Movie Maker" and "Windows Live Photo Gallery". Un-check the programs you do not want or need, to keep the clutter out of your PC.
Step 2: Load On-board Video from rocket flight:Open the Windows Live Movie maker software and load in your in-flight video by clicking "Add Videos and Photos" and select the video you want to make a panorama from. If your video camera is the type (MD80, 808 car keys, or gumpack) that always records a date/time on the video then you have a couple of choices.
You can check out this tutorial series on timestamp removal:
You can also use one of the more advanced video processing programs (see "Additional Resources" below) and trim the bottom of the movie off to eliminate the date.
Step 3: Locating suitable video frames for a Panorama:Move the video slider control to the section of your video you wish to include in your panorama. The key to getting a good final panoramic image is to find a portion of the video where multiple frames in the video make up a large overlapping area or swath across a wide view of the subject (ground, clouds, horizon, etc.).
Tip: Our sample video was recorded using a downward looking rocket camera which tilted with the rocket at apogee, and captured a view of the ground below sweeping up to the horizon, this camera view usually produces a tall and narrow panorama. A camera mounted looking sideways from the rocket will usually rotate and sweep across a wide area of the horizon and produce a short but wide panorama output. The "camera viewing sideways" mount usually is easier to get overlapping images of a wider area from, so if you are trying this technique for the first time you may wish to start out with a video recorded with this style of camera mount.
Step 4: Taking snapshots:When you have found the section of your video where the frames will form your panorama, you simply click on the "Snapshot" icon to save each overlapping frame as a still image. You should create a new folder and save all the images there so they are easy to locate when you want to build the panorama from them. Take as many snapshots as you need to cover the entire area of interest. The images do not have to be oriented the same way (like a camera looking down from a spinning rocket might record), and they do not have to be saved or named in any particular way. The software only needs a modest amount of overlap to automatically rotate and match up every snapshot to get good results. Tip: Use the frame step and back buttons below the video slider to quickly find the best video frames more easily.
When you have finished collecting still frames, exit Windows Live Movie Maker.
Step 5: Preparing your snapshots:Run Windows Live Photo Gallery and click the "Include Folder" menu item. Navigate to the folder where you saved your stills and accept the selection. All of the still images are now visible in the Photo gallery.
Step 6: Make the Panorama:Select the images you want to make a panorama from and click the "Panorama" button. The software will read in your files, rotate, align, match or blend the color or lighting, and will generate a new image that is the combination of all the frames.
If you click on the "Actual Size" button, the image will expand to show that the actual image is now much larger than the small images that came from the comparatively low resolution video we started out with. You can zoom in and out and scroll around the panorama just like the NASA panoramas!
Step 7: Cropping the Panorama:You can now click on the "crop" button which will allow you to trim the black areas from the panorama if you prefer to have a more rectangular image that does not reveal it is a mosaic. You may wish to do this if you want to frame the image.
To use the crop tool you simply use your mouse to drag the edges of the rectangular box on the screen until you find a shape that contains the part of the image you want to keep. The rest of the image will be cut off and you will save a nice clean rectangular image.
We like the way the black borders resemble the NASA photos. It gives it a unique look that has a nice high tech flavor to it. Here is the raw panorama without cropping.
Finish:There are many other programs available to stitch panoramic images together, we have not tested every one but have compiled a list of some of the more popular tools so that other options are available. People who own Linux or Mac machines could find some of these resources valuable.
For a truly platform independent method of creating panoramas, we strongly encourage people to use a package called "Gimp" which stands for "GNU Image Manipulation Program", Gimp is essentially an "Open Source" (free) program patterned after Adobe Systems Photoshop. Gimp accepts "plug-ins" to add capabilities, and there are several which can be used to make panoramas! The details of installing and running Gimp are beyond the scope of this tutorial, but perhaps some enterprising blogger out there will take this idea and run with it! ;-)
Additionally, no panorama tutorial would be complete without mentioning some of the latest advances in the field: as of this writing there are a number of smart-phone based "apps" which can be used to stitch images from their internal cameras together and there is a virtual avalanche of inexpensive digital cameras coming out recently which have panorama capability built in. You could very easily save your snapshots to a media card and use your camera or phone to make a panorama. Technology is so cool!
Thank you for taking the time to read this tutorial. If you find this tutorial useful and wish to use our techniques in your own panoramas, feel free to do so. Additional Resources:
Video Editing / Snapshot Resources:
AVIDemux - http://avidemux.sourceforge.net/
VirtualDub - http://www.virtualdub.org/
VLC Media Player - http://www.videolan.org/
Stitching Resources: AutoStitch - http://cvlab.epfl.ch/~brown/autostitch/autostitch.html
CleVR - http://www.clevr.com/
Gimp - http://www.gimp.org/
Hugin - http://hugin.sourceforge.net/
Panorama Tools - http://panotools.sourceforge.net/
Photoshop - http://www.adobe.com/products/photoshop/
PTgui - http://www.ptgui.com/
Stitchpanorama (Gimp Plugin) - http://stitchpanorama.sourceforge.net/