The biggest barrier to large-scale nutrition research today is the inadequacy of current dietary assessment methods, which are extremely burdensome for human participants and contain substantial biases and errors. AI technologies such as deep learning can be used to identify food from images and text with the goal of using photo diaries for real-time dietary assessment. In year 1 and 2 the researchers developed a benchmark database of food photos and photo diaries. Preliminary evaluations of publicly available algorithms suggest they are insufficient for ingredient prediction. Therefore, the challenge is to develop or adapt deep learning algorithms to predict ingredients from food photo diaries used in dietary assessment, which also include single ingredient foods (e.g. an orange) and beverages.

AI has the potential to enable consumers to learn the health effects of their food far beyond what is available on a Nutrition Facts Label. The long-term goal of this project is to develop a “food photo to ingredients” module and a “ingredients to health outcome” module, each of which will be needed in most computational systems in nutrition. In particular, the researchers are focusing on new data types (glycome of food) and new dietary data representations (hierarchical) to test the ability of these innovations to improve our ability to predict health outcomes of interest.