MCMingliang Chen

Mingliang Chen

*This is an algorithmically generated author record based on publicly available information.

Mingliang Chen has a total of 30 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A Comparative Study of Principled rPPG-Based Pulse Rate Tracking Algorithms for Fitness Activities

    by Qiang Zhu, Chau-Wai Wong, Zachary McBride Lazri, Mingliang Chen, Chang-Hong Fu, Min Wu

    Performance improvements obtained by recent principled approaches for pulse rate (PR) estimation have typically been achieved by adding or modifying certain modules within a reconfigurable system. Yet, evaluations are usually performed only at the system level. To better understand each module's contribution and facilitate future research in explainable learning and artificial intelligence for physiological monitoring, this paper conducts a comparative study of video-based, principled PR tracking algorithms, with a particular focus on challenging fitness scenarios. A review of the progress achieved over the last decade and a half in this field is utilized to construct the major processing modules of a reconfigurable remote pulse rate sensing system. Experiments are conducted on two challenging datasets---an internal collection of 25 videos of two Asian males exercising on stationary-bike, elliptical, and treadmill machines and 34 videos from a public ECG fitness database of 14 men and 3 women exercising on elliptical and stationary-bike machines. The signal-to-noise ratio (SNR), Pearson's correlation coefficient, error count ratio, error rate, and root mean squared error are used for performance evaluation. The top-performing configuration produces respective values of -0.8 dB, 0.86, 9%, 1.7%, and 3.3 beats per minute (bpm) for the internal dataset and 1.3 dB, 0.77, 28.6%, 6.0%, and 8.1 bpm for the ECG Fitness dataset, achieving significant improvements over alternative configurations. Our results indicate a synergistic effect between pulse color mapping and adaptive motion filtering, as well as the importance of a robust frequency tracking algorithm for PR estimation in low SNR settings.

  • A Comparative Study of Principled rPPG-Based Pulse Rate Tracking Algorithms for Fitness Activities

    by Qiang Zhu, Chau-Wai Wong, Zachary McBride Lazri, Mingliang Chen, Chang-Hong Fu, Min Wu

    Performance improvements obtained by recent principled approaches for pulse rate (PR) estimation have typically been achieved by adding or modifying certain modules within a reconfigurable system. Yet, evaluations are usually performed only at the system level. To better understand each module's contribution and facilitate future research in explainable learning and artificial intelligence for physiological monitoring, this paper conducts a comparative study of video-based, principled PR tracking algorithms, with a particular focus on challenging fitness scenarios. A review of the progress achieved over the last decade and a half in this field is utilized to construct the major processing modules of a reconfigurable remote pulse rate sensing system. Experiments are conducted on two challenging datasets—an internal collection of 25 videos of two Asian males exercising on stationary-bike, elliptical, and treadmill machines and 34 videos from a public ECG fitness database of 14 men and 3 women exercising on elliptical and stationary-bike machines. The signal-to-noise ratio (SNR), Pearson's correlation coefficient, error count ratio, error rate, and root mean squared error are used for performance evaluation. The top-performing configuration produces respective values of $-$ 0.8 dB, 0.86, 9%, 1.7%, and 3.3 beats per minute (bpm) for the internal dataset and 1.3 dB, 0.77, 28.6%, 6.0%, and 8.1 bpm for the ECG Fitness dataset, achieving significant improvements over alternative configurations. Our results indicate a synergistic effect between pulse color mapping and adaptive motion filtering, as well as the importance of a robust frequency tracking algorithm for PR estimation in low SNR settings.


  • Open Access

    Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learning

    by Aditya Srinivas Timmaraju, Mehdi Mashayekhi, Mingliang Chen, Qi Zeng, Quintin Fettes, Wesley Cheung, Yihan Xiao, Manojkumar Rangasamy Kannadasan, Pushkar Tripathi, Sean Gahagan, et al.

    Variances in ad impression outcomes across demographic groups are increasingly considered to be potentially indicative of algorithmic bias in personalized ads systems. While there are many definitions of fairness that could be applicable in the context of personalized systems, we present a framework which we call the Variance Reduction System (VRS) for achieving more equitable outcomes in Meta's ads systems. VRS seeks to achieve a distribution of impressions with respect to selected protected class (PC) attributes that more closely aligns the demographics of an ad's eligible audience (a function of advertiser targeting criteria) with the audience who sees that ad, in a privacy-preserving manner. We first define metrics to quantify fairness gaps in terms of ad impression variances with respect to PC attributes including gender and estimated race. We then present the VRS for re-ranking ads in an impression variance-aware manner. We evaluate VRS via extensive simulations over different parameter choices and study the effect of the VRS on the chosen fairness metric. We finally present online A/B testing results from applying VRS to Meta's ads systems, concluding with a discussion of future work. We have deployed the VRS to all users in the US for housing ads, resulting in significant improvement in our fairness metric. VRS is the first large-scale deployed framework for pursuing fairness for multiple PC attributes in online advertising.


  • Open Access

    Identity-Privacy Protection for Facial-rPPG Based Smart Health Research

    by Mingliang Chen, Ashira Jayaweera, Chau-Wai Wong, Min Wu

    Camera-based remote photoplethysmography (rPPG) technology has shown a promising future in contact-free cardiac and other smart health applications. The rPPG technology typically requires facial videos as a source input, which may lead to identity-privacy concerns. Facial videos are sensitive and contain subjects' identifiable appearance features. Coupled with the health information potentially revealed by rPPG techniques, the compounding sensitivity has been a major obstacle to encouraging the sharing of facial rPPG video datasets in the research community to foster the advancement of the field. This paper investigates a suite of anonymization transforms that remove the identifiable appearance features in facial videos and retain the physiological signals for rPPG analysis. After the transformation, the facial videos are de-identified and may be shared in public with little risk of identity-privacy leakage. The proposed algorithm offers tunable options to balance the physiological fidelity and the identity-protecting strength to meet different levels of privacy requirements. A human subject study has been carried out to understand -- both qualitatively and quantitatively -- the perceived strength and efficacy of privacy protection by these anonymization techniques in de-identifying the facial videos and maintaining the physiological signals.


  • Open Access

    Detecting Essential Landmarks Directly in Thermal Images for Remote Body Temperature and Respiratory Rate Measurement With a Two-Phase System

    by Zachary Mcbride Lazri, Qiang Zhu, Mingliang Chen, Min Wu, Quanzeng Wang

    Infrared thermographs (IRTs, also called thermal cameras) have been used to remotely measure elevated body temperature (BT) and respiratory rate (RR) during infectious disease outbreaks, such as COVID-19. To facilitate the fast measurement of BT and RR using IRTs in densely populated venues, it is desirable to have IRT algorithms that can automatically identify the best facial locations in thermal images to extract these vital signs. The IEC 80601-2-59:2017 standard suggests that the regions medially adjacent to the inner canthi of the eyes are robust BT measurement sites. The nostril regions, on the other hand, are often used for RR estimation. However, it is more difficult to automatically identify inner canthi and nostrils in thermal images than in visible-light images, which are rich with exploitable features. In this paper, a unique system that can detect inner canthi and outer nostril edges directly in thermal images in two phases is introduced. In Phase I, original thermal images were processed in four different ways to enhance facial features to facilitate inner canthus and nostril detection. In Phase II, landmarks of the inner canthi and outer nostril edges were detected in two steps: (1) face detection using the Single Shot Multibox Detector (SSD) and (2) facial landmark detection to locate the inner canthi and outer nostril edges. The face detection, facial landmark detection, and overall system accuracies were evaluated using the intersection over union, normalized Euclidean distance, and success detection rate metrics on a set of 36 thermal images collected from 12 subjects using three different IRTs. Additional validation was performed on a subset of 40 random thermal images from the publicly available Tufts Face Database. The results revealed that the processed images—referred to as ICLIP images—yielded the highest landmark localization accuracy from the four types of processed thermal images, verifying that the system can automatically and accurately estimate the inner canthus and nostril locations in thermal images. The proposed system can be applied in IRT algorithms to provide reliable temperature measurements and RR estimates during infectious disease outbreaks.


  • Open Access

    PulseEdit: Editing Physiological Signals in Facial Videos for Privacy Protection

    by Mingliang Chen, Xin Liao, Min Wu

    Recent studies have shown that physiological signals such as heart beat and breathing can be remotely captured from human faces using a regular color camera under ambient light. This technology, referred to as remote photoplethysmography (rPPG), can be used to collect the physiological status of users who are in front of a camera, which may raise privacy concerns. To avoid the privacy abuse of the rPPG technology, this paper develops PulseEdit, a novel and efficient algorithm that can edit the physiological signals in facial videos without affecting visual appearance and thus protect the user’s physiological signal from disclosure. PulseEdit can either remove the trace of the physiological signal in a video or transform the video to contain a target physiological signal chosen by a user. Experimental results show that PulseEdit can effectively edit physiological signals in facial videos and prevent heart rate measurement based on rPPG. It is possible to utilize PulseEdit in adversarial scenarios against rPPG-based visual security algorithms. We present analyses on the performance of PulseEdit against rPPG-based liveness detection and rPPG-based deepfake detection, and demonstrate its ability to circumvent these visual security algorithms and its important role in supporting the design of attack-resilient systems.

  • PulseEdit: Editing Physiological Signals in Facial Videos for Privacy Protection

    by Mingliang Chen, Xin Liao, Min Wu

    Recent studies have shown that physiological signals such as heart beat and breathing can be remotely captured from human faces using a regular color camera under ambient light. This technology, referred to as remote photoplethysmography (rPPG), can be used to collect the physiological status of users who are in front of a camera, which may raise privacy concerns. To avoid the privacy abuse of the rPPG technology, this paper develops PulseEdit, a novel and efficient algorithm that can edit the physiological signals in facial videos without affecting visual appearance and thus protect the user’s physiological signal from disclosure. PulseEdit can either remove the trace of the physiological signal in a video or transform the video to contain a target physiological signal chosen by a user. Experimental results show that PulseEdit can effectively edit physiological signals in facial videos and prevent heart rate measurement based on rPPG. It is possible to utilize PulseEdit in adversarial scenarios against rPPG-based visual security algorithms. We present analyses on the performance of PulseEdit against rPPG-based liveness detection and rPPG-based deepfake detection, and demonstrate its ability to circumvent these visual security algorithms and its important role in supporting the design of attack-resilient systems.


  • Adaptive Multi-Trace Carving for Robust Frequency Tracking in Forensic Applications

    by Qiang Zhu, Mingliang Chen, Chau-Wai Wong, Min Wu

    In the field of information forensics, many emerging problems involve a critical step that estimates and tracks weak frequency components in noisy signals. It is often challenging for the prior art of frequency tracking to i) achieve a high accuracy under noisy conditions, ii) detect and track multiple frequency components efficiently, or iii) strike a good trade-off of the processing delay versus the resilience and the accuracy of tracking. To address these issues, we propose Adaptive Multi-Trace Carving (AMTC), a unified approach for detecting and tracking one or more subtle frequency components under very low signal-to-noise ratio (SNR) conditions and in near real time. AMTC takes as input a time-frequency representation of the system's preprocessing results (such as the spectrogram), and identifies frequency components through iterative dynamic programming and adaptive trace compensation. The proposed algorithm considers relatively high energy traces sustaining over a certain duration as an indicator of the presence of frequency/oscillation components of interest and track their time-varying trend. Extensive experiments using both synthetic data and real-world forensic data of power signatures and physiological monitoring reveal that the proposed method outperforms representative prior art under low SNR conditions, and can be implemented in near real-time settings. The proposed AMTC algorithm can empower the development of new information forensic technologies that harness very small signals.


  • Transparency Tools for Fairness in AI (Luskin)

    by Mingliang Chen, Aria Shahverdi, Sarah Anderson, Se Yong Park, Justin Zhang, Dana Dachman-Soled, Kristin Lauter, Min Wu


  • Open Access

    Modulation Model of the Photoplethysmography Signal for Vital Sign Extraction

    by Mingliang Chen, Qiang Zhu, Min Wu, Quanzeng Wang

    This paper introduces an amplitude and frequency modulation (AM-FM) model to characterize the photoplethysmography (PPG) signal. The model indicates that the PPG signal spectrum contains one dominant frequency component -- the heart rate (HR), which is guarded by two weaker frequency components on both sides; the distance from the dominant component to the guard components represents the respiratory rate (RR). Based on this model, an efficient algorithm is proposed to estimate both HR and RR by searching for the dominant frequency component and two guard components. The proposed method is performed in the frequency domain to estimate RR, which is more robust to additive noise than the prior art based on temporal features. Experiments were conducted on two types of PPG signals collected with a contact sensor (an oximeter) and a contactless visible imaging sensor (a color camera), respectively. The PPG signal from the contactless sensor is much noisier than the signal from the contact sensor. The experimental results demonstrate the effectiveness of the proposed algorithm, including under relatively noisy scenarios.