Learning end-to-end respiratory rate prediction of dairy cows from red, green, and blue videos
Open Access
- 1 November 2024
- journal article
- research article
- Published by American Dairy Science Association in Journal of Dairy Science
- Vol. 107 (11) , 9862-9874
Abstract
Respiratory rate (RR) is an important indicator of the health and welfare status of dairy cows. In recent years, progress has been made in monitoring the RR of dairy cows using video data and learning methods. However, existing approaches often involve multiple processing modules, such as region of interest (ROI) detection and tracking, which can introduce errors that propagate through successive steps. The objective of this study was to develop an end-to-end computer vision method to predict RR of dairy cows continuously and automatically. The method leverages the capabilities of a state-of-the-art Transformer model, VideoMAE, which divides video frames into patches as input tokens, enabling the automated selection and featurization of relevant regions, such as a cow's abdomen, for predicting RR. The original encoder of VideoMAE was retained, and a classification head was added on top of it. Further, the weights of the first 11 layers of the pre-trained model were kept, while the weights of the final layer and classifier were fine-tuned using video data collected in a tie-stall barn from 6 dairy cows. Respiratory rates measured using a respiratory belt for individual cows were serving as the ground truth (GT). The evaluation of the developed model was conducted using multiple metrics, including mean absolute error (MAE) of 2.58 breaths per minute (bpm), root mean squared error (RMSE) of 3.52 bpm, root mean squared prediction error (RMSPE; as a proportion of observed mean) of 15.03%, and Pearson correlation (r) of 0.86. Compared with a conventional method involving multiple processing modules, the end-to-end approach performed better in terms of MAE, RMSE and RMSPE. These results suggest the potential to implement the developed computer vision method for an end-to-end solution, for monitoring RR of dairy cows automatically in a tie-stall setting. Future research on integrating this method with other behavioral detection and animal identification algorithms for animal monitoring in a free-stall dairy barn can be beneficial for a broader application.Keywords
This publication has 19 references indexed in Scilit:
- A comparative study of machine learning models for respiration rate prediction in dairy cows: Exploring algorithms, feature engineering, and model interpretationBiosystems Engineering, 2024
- Non-contact respiration rate measurement of multiple cows in a free-stall barn using computer vision methodsComputers and Electronics in Agriculture, 2024
- How should the respiration rate be counted in cattle?Veterinary Research Communications, 2022
- Transformers in Vision: A SurveyACM Computing Surveys, 2022
- Detection of the respiratory rate of standing cows by combining the Deeplab V3+ semantic segmentation model with the phase-based video magnification algorithmBiosystems Engineering, 2020
- EMFIT QS heart rate and respiration rate validationBiomedical Physics & Engineering Express, 2019
- Contactless Monitoring of Breathing Patterns and Respiratory Rate at the Pit of the Neck: A Single Camera ApproachJournal of Sensors, 2018
- Continuous Respiration Rate Measurement of Heat-Stressed Dairy Cows and Relation to Environment, Body Temperature, and Lying TimeTransactions of the ASABE, 2018
- A computer vision-based system for the automatic detection of lying behaviour of dairy cows in free-stall barnsBiosystems Engineering, 2013
- Control, Management, and Prevention of Bovine Respiratory Disease in Dairy Calves and CowsVeterinary Clinics of North America: Food Animal Practice, 2010