blenderFaceUV: Toward Marker‐Based Precision and Flexibility in Facial Tracking With “Invisible” Markers

Authors:
Open Access
Publisher website
Google Scholar
Abstract
BlenderFace is a highly precise marker‐based approach for measuring facial activity. Unlike most camera‐based automated facial expression recognition (AFER) systems, which rely on fixed natural landmarks, blenderFace flexibly tracks virtually any point of interest using painted markers. However, visible markers, like EMG electrodes, may alter appearance, increase self‐awareness, and affect spontaneous responding in face‐to‐face interactions. To address these limitations, we introduce blenderFaceUV, which uses sunscreen‐based markers that are visible only under ultraviolet (UV) light (365 nm). In Study 1, we assessed UV‐marker tracking accuracy using parallel visible‐light and UV recordings from two participants. In Study 2, we examined the scalability of blenderFaceUV in N = 18 lay participants prompted to produce a broad range of facial movements using common emotion terms. We also compared identical facial movements measured with blenderFaceUV and OpenFace, a widely used AFER system based on a point distribution model (PDM). Study 1 showed that UV markers could be tracked with similarly high accuracy as visible markers (reprojection/solve error < 0.3 pixels). Across 10 markers and two participants, with more than 3800 measurement occasions per participant, the mean correlation between visible and UV marker movements across the x‐ and y‐coordinates was r = 0.957, while centered RMSE values indicated average deviations of approximately 1 mm. Study 2 demonstrated the scalability of the method and revealed PDM‐related artifacts in OpenFace measurements, particularly during eye blinks and large mouth movements. PDM‐interpolated landmarks, such as upper‐lip markers, also deviated substantially from marker‐based blenderFaceUV measurements. Practical challenges remain, including the complexity of applying invisible markers and the relatively high cost of UV cameras (≈$2000–$6000). We discuss technical requirements, limitations, and a recommended workflow for implementing blenderFaceUV.
Funding Information
  • Deutsche Forschungsgemeinschaft (SCHM 1092/16‐1)

This publication has 35 references indexed in Scilit: