Speaker "Isha Chaturvedi" Details Back



Automated Detection of Street Product Displays using object detection techniques.


Tobacco marketing, restricted almost exclusively to the point-of-sale in recent years, has proven to be effective in getting more people to consume and fewer to quit cigarettes and smokeless tobacco products. The lack of empirical documentation linking product exposure to behavior, however, is a key obstacle to the adoption of additional restrictions on point-of-sale tobacco advertising. The goal of this project is to map point-of-sale tobacco marketing practices across New York City using automated detection of tobacco signage in street-level imaging data. Convolutional neural networks, which are particularly effective at detecting objects in images, were trained to identify and classify outdoor advertisements of cigarettes and smokeless tobacco. The automated tobacco signage detection model employed in this project involves Faster R-CNN, a state-of-the-art convolutional neural network related to image recognition. By efficiently discriminating between backgrounds and targets within an image, this model enables a detection algorithm to focus on areas that are more likely to contain tobacco signages. Previous analyses of visual data in public health research involving manual image coding are prohibitively costly and time-consuming. The importance and motivation of the project stems from the immediate and comprehensive effect of tobacco advertisements on its sales and consequently on public health. Detected advertisements derived from our model output provide a proof-of-concept for measuring exposure of at-risk communities to tobacco displays.

Who is this presentation for?
It is for people who have some background or interest in CNN's and Object Detection models, or have interest in knowing about AI for good projects. This project entirely belongs to NYU as it was done during the time of my masters as my master final year project.

Prerequisite knowledge:
Basic idea of Convolutional Neural Network, or Object Detection, or Image Classification. A general interest in the area is also good.

What you'll learn?
The data pipeline of extraction of Google Street View Images to Labeling, to Image Processing (Data Augmentation) to Object Detection Architecture (Faster RCNN, YOLO etc.)


I am a principal data scientist at Capital One. Prior to that, I worked at Ericsson as a data scientist. I completed my master's from New York University from an Urban Data Science program in 2018. I moved to the Bay Area in 2018 and before that, I worked in different NYU research labs (NYU Urban Observatory, NYU Audio Lab, etc.). Before moving to New York, I lived in Hong Kong for 5 years, where I did my bachelors from Hong Kong University of Science & Tech (HKUST) in Environmental Technology and Computer Science and later worked in HKUST- Deutsche Telecom Systems and Media lab (an Augmented Reality and Computer Vision focused lab) as a Research Assistant.