Capturing high-quality video with G3D
My collaborators and I put together a sketch submission for SIGGRAPH '05. Along the way we came up with a process that lets us capture high-quality video with G3D, the C++ OpenGL abstraction we're using for CaveWriting. This technique is a vast improvement over the video capture techniques I've used for interactive 3D graphics in the past, because it captures the actual bits on screen and yields a high-frame-rate video with smooth animation.
(Until recently, we did video capture of interactive graphics either with a digital-to-analog scan converter, or by filming the walls of the cave with the head tracker attached to the camera. These techniques yielded low-resolution images with a fair amount of distortion; they did, however, capture real-time interactions. Based on my preliminary research, standard video capture applications will have difficulty with capturing high-frame-rate graphics which rely on extensive hardware acceleration; the technique described below slows down the frame rate, captures each frame, then re-creates interactive frame rates in post.)
The process has several steps:
- Set your rendering hardware to do exactly what you want. In particular, do you want to turn on hardware antialiasing? You'll want to go through the whole process with and without hardware antialiasing, to see what gives you better results after your whole encoding & compression process.
- After drawing each frame, capture a screengrab using G3D::RenderDevice::screenshot("grabs/"); This will create a series of dated and numbered screenshots in jpeg format in the grabs/ directory. Interact very slowly! Your motions will be speeded up by a large factor (10-300 or so) depending on many things. You'll have to do iterate through this process a few times to get the timings right.
- Assemble the screenshots into a video. On the Mac, using QuickTime Pro, do File -> Open Image Sequence, open the directory with all your images in it, import at your desired output frame rate (probably 30 fps), then save as that same frame rate. You can also do this with Graphic Converter, which (unlike QuickTime) doesn't require registering to use this feature, On Windows, Morgan suggsets using VirtualDub. For Linux, Peter Sibley suggets mencode/mplayer
- Tune timings; if your animation is based on real time passing, extend the duration of each tween; if your animation is based on simulation time, slow down the passing of simulation time.
- Archive, if desired, and clean up the captured images. Frame numbers just keep incrementing with every subsequent call to RenderDevice::screenshot(), so if you don't clean out the grabs directory, you'll be using an awful lot of disk space, and it will be tricky to identify the first frame of each run.
- Repeat steps 2-5 until you have the timing you want. In our experience, we captured about a thousand frames at 800x600 at 1-2 fps; the same program running without frame-grabbing ran at >100 fps.
- Edit and title. We edited using iMovie 4, which was acceptable but not great.
- Add audio.
Export. Video is huge; for SIGGRAPH we had to get our sketch down from 250 mb without audio to under 40 mb with audio. Video compression is an art, and not one that I know well. If you can recommend good video compression settings for captured CG, let me know. We used MPEG-4 compression with medium quality.
A few suggestions:
- Leave a lot of time for working with video. Video can bring even blazingly fast computers (like our dual CPU 2 gig ram G5) to a crawl.
- If you're using linux, figure out how to capture to a local temporary directory (/tmp or /ltmp) rather than using the networked file system. You might have to ask for permission in advance to use local storage.