Practical, real-time studio matting using dual imagers

in Eurographics Symposium on Rendering 2006

Morgan McGuire, Williams College
Wojciech Matusik, MERL
William Yerazunis, MERL

Preprint (PDF)
Presentation (PPT.ZIP)
Video (MOV) 		Abstract
Data
Images
BibTex

Abstract

This paper presents a practical system for capturing high-resolution video mattes using cameras that contain two imagers on one optical axis. The dual imagers capture registered frames that differ only by defocus or polarization at pixels corresponding to special background `gray screens.' This system eliminates color spill and other drawbacks of blue-screen matting while preserving many of its desirable properties (e.g., unassisted, real-time, natural illumination) over more recent methods, and achieving higher precision output for Bayer-filter digital cameras. Because two imagers capture more information than one, we are able to automatically process scenes that would require manual retouching with blue-screen matting.

The dual-imager system successfully pulls mattes for scenes containing thin hair, liquids, glass, and reflective objects; mirror reflections produce incorrect results. We show result comparisons for these scenes against blue-screen matting and describe materials and patterns for building a capture system.

Data

Raw data and results captured with the dual imager system, including video that does not appear in the proceedings. Free for non-commercial use as long as attribution is made.

Data includes the same scenes prepared for defocus matting, polarization matting, blue screen matting/green screen matting, difference matting, luminance matting, and natural image matting, along with selected results. Not all sequences contain all variations. See data-readme.txt for a key to the filenames.

Fluid.zip (0.76 GB) Red liquid poured from a measuring cup. Lots of translucency and specularity.
StillLife.zip (0.07 GB) Still images of fur, glass, metal, color charts, and natural objects.
People.zip (2.65 GB) Costumed actors in motion. Dancing, sword fighting, talking, weatherman.
Cats.zip (1.00 GB) House cats grooming, sitting, and fighting. Green-screen and polarization only.
Defocus.zip (0.09 GB) Extremely high-resolution still images from the defocus camera.
data-readme.txt License and explanation of filenames.

All zipfiles contain uncompressed AVI files, lossless PNG files, and PPM files. These can all be opened using Matlab or the AVIFile and GImage classes in G3D.

Images

Selected images from this project (see the paper for more):

Polarizer Camera

Pony Polarizer Result: Imager 0 View, Imager 1 View, Recovered Matte

Weather Forecaster Polarizer Result: Imager 0 View, Imager 1 View, Recovered Matte

Defocus Results

BibTex 


@inproceedings{mcguire06dualimagers,
crossref = {EGSR06-proc},
author = {Morgan Mc{G}uire and Wojciech Matusik and William Yerazunis},
title = {{Practical, Real-time Studio Matting using Dual Imagers}},
pages = {235--244},
URL = {http://www.eg.org/EG/DL/WS/EGWR/EGSR06/235-244.pdf},
DOI = {10.2312/EGWR/EGSR06/235-244}
}