Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food
Preprint
Abstract
Understanding the nutritional content of food from visual data is a challenging computer vision problem, with the potential to have a positive and widespread impact on public health. Studies in this area are limited to existing datasets in the field that lack sufficient diversity or labels required for training models with nutritional understanding capability. We introduce Nutrition5k, a novel dataset of 5k diverse, real world food dishes with corresponding video streams, depth images, component weights, and high accuracy nutritional content annotation. We demonstrate the potential of this dataset by training a computer vision algorithm capable of predicting the caloric and macronutrient values of a complex, real world dish at an accuracy that outperforms professional nutritionists. Further we present a baseline for incorporating depth sensor data to improve nutrition predictions. We will publicly release Nutrition5k in the hope that it will accelerate innovation in the space of nutritional understanding.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information2021
- Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- You Are What You Eat: Exploring Rich Recipe Information for Cross-Region Food AnalysisIEEE Transactions on Multimedia, 2017
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era2017
- Learning Cross-Modal Embeddings for Cooking Recipes and Food Images2017
- Food Recognition: A New Dataset, Experiments, and ResultsIEEE Journal of Biomedical and Health Informatics, 2016
- Im2Calories: Towards an Automated Mobile Vision Food Diary2015