Dynamic Facial Projection Mapping
-Basic Test-

image

A person’s face expresses an innate physical identity. It verifies one is human, classifies individuals, and reveals inner traits. These are ideas we have long believed in, but are they actually immutable truths?

Contemporary technology (including smartphone apps) allows for the real-time editing of human facial representations. Few people use their real faces in the virtual world; faces have become outer shells that are as liquid and digitally exchangeable as any virtual avatar.

These changes have occurred rapidly, and are likely to continue. Although currently, technological intervention in one’s appearance remains at a cyber level - we have not yet been able to edit our own faces at will in the real world - we expect that in the near future, this will not only become possible, but will become accepted and established as the new norm.

Our innovation, dynamic projection mapping, will support this progression. We project graphics that overlay the face perfectly to change viewers’ perception of a face to one that is different from what they would see in real-life. We have previously projected graphics onto dancers’ faces to verify the effects. However, this method was inappropriate for application to daily life faces, with a challenging set of requirements including the necessity for markers on faces, multiple cameras, and fixed facial expressions from the subjects.

We have constructed a new method for dynamic facial projection mapping that solves these problems. In this system, a single high-speed camera recognizes facial expressions without the use of markers, generates images according to their position and deformation, and projects them using DynaFlash, a high-speed projector. The system tracks regular movements such as head-movement, talking, and laughing, making it possible to completely morph the facial surface, allowing faces to be swapped easily and freely.

Image
Image
Image

Your face is no longer limited to what you were born with, but now faces possess a physical, as well as a digital, aspect, opening up a wide range of applications and development possibilities. We demonstrated dynamic projection mapping in collaboration with a speaker in a presentation, by morphing the speaker’s face mid-speech. The title of the talk, "The Death of Reality," correlated with the theme underlying the technology.

It is worth noting, however, that the technology is still in its infancy: the demonstration used a monochrome projector, and the projected video only provided limited texture mapping effects. Although there are many remaining challenges, the dynamic projection technology that we have created has only began to be put to use. "Face" will continue to evolve, and the possibilities are vast.

Image
Image
Image
Image

Research member

Yoshihiro Watanabe