Demo: Smartwatch based shopping gesture recognition

In the current retail segment, the retail store owners are keen to understand the browsing behavior and purchase pattern of the shoppers inside the physical stores. Profiling the behavior of the shopper is key to success for any marketing strategies that can optimize or personalize shopping-related services in real-time. We envision that exploiting the knowledge of real-time behavior of shopper’s in-store activities enables novel applications such as: (a) targeted advertising or recommendations: based on longer term shopper profiles, (b) proactive retail help to assist the shoppers who are confused in choosing between two items, (c) smart reminders that can remind the shoppers to pickup an item in the shopping list that they might have missed. Our work is motivated by the fact that a significant fraction of in-store shopping activities involve gestural interactions with objects of interest (such as picking up an item and putting the item in the shopping cart in a grocery store or retrieving and trying out a dress in a clothing store). In our recent works, we showed the design and initial prototype of frameworks for reliably inferring shopper’s in-store interactions and behavior by just observing their hand and foot movement inside a store. The hand gestures and locomotive pattern of the shopper inside a store is identified by appropriately mining the sensor data from shopper's personal smartphone and wearable devices (smartwatch).