Visual feature modeling and refinement with application in dietary assessment

There has been rapid emergence of technologies for improving our lives and health. However, the technologies for real-time monitoring of diet are still in their infancy. In this thesis we describe an imaging based tool to assess diet. Meal images taken before and after eating allow for the automatic estimation of consumed foods using image processing and analysis methods. In this thesis we have investigated features for efficient visual characterization of food items, including color, texture and local descriptors. Emphasis is given to textural features by describing three unique texture descriptors for both texture classification and retrieval that can be used to characterize food items. We describe a classification system for food identification that can be extended to other object classification tasks. Multiple feature spaces are independently classified and corresponding decisions are fused together according to a set of rules to achieve a final decision. Potential misclassifications are corrected by using contextual information, such as object interaction, and information from the confusion matrix on the validation dataset. We evaluated our models based on food datasets from controlled and natural eating events and on publicly available object recognition benchmark datasets. A database architecture is described for capturing and indexing information from our food imaging system. This database system has been complemented with a web interface that allows researchers to monitor patients in real-time, and interact with the dietary data in unique ways. This system provide tools for nutritionists and the health research community that can be used for further data mining to extract diet pattern of individuals and/or social groups.