Eyes for Relighting
In order to relight real objects and scenes, it is important to
know their lighting conditions. Obtaining the lighting of a scene
from a single image remains a difficult and open problem. However,
this problem can be solved when we have a face (and hence an eye)
in the image, which is often the case with images and videos. We
show how an image of an eye taken in a completely unstructured setting
can be used to extract a dense illumination distribution. In concrete,
we show that an environment map of the scene with a large field
of view can be computed from the eye which, in turn, represents
the illumination distribution of the scene with respect to the eye.
In other words, the eye can be used as a natural light probe which
gives us not just the directions of a few point sources but the
complete distribution of the frontal illumination incident on the
eye and hence the face. We demonstrate the use of an eye in a number
of relighting scenarios including virtual object insertion, casual
face relighting, and face replacement in already taken images. These
results show that the eye not only serves as a useful tool for relighting
but also make relighting possible in situations where other techniques
are hard to use. (with S.K.
Nayar)
Primary Reference
(TBA)
|