Jason Zhi-Shen Zhang

Background: Swallowing is a complex biomechanical process, and its impairment (dysphagia) poses major health risks for older adults. Current diagnostic methods such as videofluoroscopic swallowing (VFSS) and fiberoptic endoscopic evaluation of swallowing (FEES) are effective but invasive, resource-intensive, and unsuitable for continuous monitoring. This study proposes a novel end-to-end RGB–D framework for automated swallowing event localization in continuous video streams. Methods: The framework enhances the AdaTAD backbone through three key innovations: (i) finding the optimal strategy to integrate depth information to capture subtle neck movements, (ii) examining the best adapter design for efficient temporal feature adaptation, and (iii) introducing a Kolmogorov–Arnold Network (KAN) decoder that leverages Chebyshev polynomials for non-linear temporal modeling. Evaluation on a proprietary swallowing dataset comprising 641 clips and 3153 annotated events demonstrated the effectiveness of the proposed framework. We analysed and compared the modification strategy across designs of adapters, decoders, input channel combinations, regression methods, and patch embedding techniques. Results: The optimized configuration (VideoMAE + GRNConvNeXtAdapter + KAN + RGD + boundary regression + sinusoidal embedding) achieved an average mAP of 83.25%, significantly surpassing the baseline I3D + RGB + MLP model (61.55%). Ablation studies further confirmed that each architectural component contributed incrementally to the overall improvement. Conclusions: These results establish the feasibility of accurate, non-invasive, and automated swallowing event localization using depth-augmented video. The proposed framework paves the way for practical dysphagia screening and long-term monitoring in clinical and home-care environments.

Swallowing is a complex biomechanical process, and its impairment (dysphagia) poses major health risks for older adults. Current diagnostic methods such as videofluoroscopic swallowing (VFSS) and fiberoptic endoscopic evaluation of swallowing (FEES) are effec-tive but invasive, resource-intensive, and unsuitable for continuous monitoring. This study proposes a novel end-to-end RGB–D framework for automated swallowing event localization in continuous video streams. The framework enhances the AdaTAD back-bone through three key innovations: (i) an RGD input strategy that substitutes the noisy blue channel with depth information to capture subtle neck movements, (ii) a GRNCon-vNeXtAdapter that incorporates Global Response Normalization for efficient temporal feature adaptation, and (iii) a Kolmogorov–Arnold Network (KAN) decoder that leverages Chebyshev polynomials for non-linear temporal modeling. (iv) boundary regression head and sinusoidal patch embedding to further optimize model precision. Evaluation on a proprietary swallowing dataset comprising 641 clips and 3,153 annotated events demon-strated the effectiveness of the proposed framework. We analysed and compared modifi-cation strategy across design of adapter, decoder, input channel combination, regression method, and patch embedding technique. The optimized configuration (VideoMAE + GRNConvNeXtAdapter + KAN + RGD + boundary regression + sinusoidal embedding) achieved an average mAP of 83.25%, significantly surpassing the baseline I3D + RGB + MLP model (61.55%). Ablation studies further confirmed that each architectural compo-nent contributed incrementally to the overall improvement. These results establish the feasibility of accurate, non-invasive, and automated swallowing event localization using depth-augmented video. The proposed framework paves the way for practical dysphagia screening and long-term monitoring in clinical and home-care environments.

Thyroid cancer, a prevalent malignancy with rising incidence—especially among females—requires early detection to reduce overdiagnosis risks and enable less invasive treatments. Ultrasound (US) imaging is pivotal but limited by observer variability. We propose a 3D imaging system combining a US machine, robotic arm, depth camera, and customized software to address this. The system integrates automated robotic assistance for precision and repeatability, alongside 3D reconstruction of ultrasound elastography (USE) and brightness mode (USB) for flexible multi-view observation and quantitative stiffness analysis. Our system incorporates SegUnXt+, a deep learning model demonstrating competitive thyroid gland segmentation (IoU 82.8%, DC 90.6%) and thyroid nodule segmentation (IoU 71.9%, DC 83.3%), outperforming other models. The system enhances diagnostic accuracy by minimizing observer dependency, enabling precise 3D thyroid visualization, and supporting early detection through automated, quantitative elastographic and morphologic analysis.Clinical Relevance— Mass screening and support diagnosis of thyroid cancer to improve screening accuracy and accessibility