DIS 2026 · Research walkthrough

Tuning the Face

Modulating facial expressions for realistic self-avatars in virtual reality.

Human–Computer Interaction Virtual Reality Avatar Expression Jiamu Tang · Yang Lu · collaborators

Full conference slides ↗ Demo video ↗ Code & prototypes ↗

The central question

Should a realistic avatar faithfully mirror every facial movement, or should it help people express the social signal they intend to send?

01 · Motivation

Faithful tracking is not always intended expression

Slide explaining why faithful facial mirroring can differ from intended expression — **Reading note**A one-to-one copy can expose nervousness or other involuntary cues even when a person intends to appear composed. Because people cannot accurately monitor their own facial movement, small mismatches become especially noticeable on a photorealistic self-avatar.

02 · Concept

A real-time modulation lens

Slide illustrating facial expression exaggeration and suppression — **Reading note**The system re-renders tracked expressions instead of mirroring them directly. It can exaggerate an existing cue so it reads more clearly, or suppress it so a user can remain composed. The goal is not to invent an emotion, but to tune the intensity of what the avatar communicates.

03 · Study design

The same face, viewed from two perspectives

Study design comparing self-perception and observer perception — **Reading note**Eighteen participants embodied their own avatar and rated accuracy and social appropriateness. Eighteen separate observers rated recognition, naturalness, and intensity. Across six emotions and nineteen modulation conditions, the study also logged raw facial input to test whether the avatar changed users’ own behavior.

04 · Finding one

Suppression is tolerated; exaggeration is emotion-specific

Charts of perceived accuracy and social appropriateness across emotions — **Reading note**There is no universal multiplier. Happiness and surprise have a narrow acceptable range, while anger and disgust tolerate more exaggeration. Fear and sadness remain difficult to perform convincingly. Expression tuning therefore needs per-emotion boundaries rather than a single global setting.

05 · Finding two

The avatar changes the user’s real face

Reading noteWhen the avatar suppresses an expression, users amplify their own facial movement. When it exaggerates, users relax. Embodiment and a live self-view create a feedback loop: people compensate for what the avatar shows, even though the modulation happens digitally.

06 · Design guidance

From perception curves to “safe” ranges

Per-emotion effective multiplier ranges derived from study results — **Reading note**The project combines user appropriateness, observer naturalness, and recognition into a composite score, then retains the band within 80% of each emotion’s peak. These ranges are a design framework—not a fixed rule—and can shift with context and priorities.

07 · Validation

More positive expression without a naturalness cost

Results from the VR graduation speech validation study — **Reading note**In a counterbalanced VR speech study, twelve participants spoke once with modulation and once without it. Ten judged the modulated speech as more positive, eight preferred their overall performance, and seven saw no difference in naturalness. This is an initial validation in one social context, not a population-wide estimate.

08 · Implications

Expression control also raises questions of trust

Discussion slide covering behavior, realism, and ethics — **Reading note**A tunable face can support self-presentation and expression training, but it can also create dependence, social pressure, or a mask-like effect. The project treats disclosure—whether others should know that a face is being modulated—as an open design and governance question.