Smartphone-based food recognition system using multiple deep CNN models
Open Access
- 12 August 2021
- journal article
- research article
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 80 (21-23) , 33011-33032
Abstract
People with blindness or low vision utilize mobile assistive tools for various applications such as object recognition, text recognition, etc. Most of the available applications are focused on recognizing generic objects. And they have not addressed the recognition of food dishes and fruit varieties. In this paper, we propose a smartphone-based system for recognizing the food dishes as well as fruits for children with visual impairments. The Smartphone application utilizes a trained deep CNN model for recognizing the food item from the real-time images. Furthermore, we develop a new deep convolutional neural network (CNN) model for food recognition using the fusion of two CNN architectures. The new deep CNN model is developed using the ensemble learning approach. The deep CNN food recognition model is trained on a customized food recognition dataset.The customized food recognition dataset consists of 29 varieties of food dishes and fruits. Moreover, we analyze the performance of multiple state of art deep CNN models for food recognition using the transfer learning approach. The ensemble model performed better than state of art CNN models and achieved a food recognition accuracy of 95.55 % in the customized food dataset. In addition to that, the proposed deep CNN model is evaluated in two publicly available food datasets to display its efficacy for food recognition tasks.Keywords
Funding Information
- Qatar University (QUCG-CED-20/21-2)
- Qatar University
This publication has 30 references indexed in Scilit:
- Going deeper with convolutions2015
- Food image recognition using deep convolutional network with pre-training and fine-tuning2015
- Food Detection and Recognition Using Convolutional Neural Network2014
- FoodCam-2562014
- A Food Recognition System for Diabetic Patients Based on an Optimized Bag-of-Features ModelIEEE Journal of Biomedical and Health Informatics, 2014
- Cloud-based SVM for food categorizationMultimedia Tools and Applications, 2014
- DietCam: Automatic dietary assessment with mobile camera phonesPervasive and Mobile Computing, 2012
- Finding objects for blind people based on SURF features2011
- ShopTalk: Independent Blind Shopping Through Verbal Route Directions and Barcode ScansThe Open Rehabilitation Journal, 2009
- Trinetra: Assistive Technologies for Grocery Shopping for the Blind2006 10th IEEE International Symposium on Wearable Computers, 2006