Speaker "Isha Chaturvedi" Details Back



Automated Detection of Street Product Displays using object detection techniques.


Tobacco marketing, restricted almost exclusively to the point-of-sale in recent years, has proven to be effective in getting more people to consume and fewer to quit cigarettes and smokeless tobacco products. The lack of empirical documentation linking product exposure to behavior, however, is a key obstacle to the adoption of additional restrictions on point-of-sale tobacco advertising. The goal of this project is to map point-of-sale tobacco marketing practices across New York City using automated detection of tobacco signage in street-level imaging data. Convolutional neural networks, which are particularly effective at detecting objects in images, were trained to identify and classify outdoor advertisements of cigarettes and smokeless tobacco. The automated tobacco signage detection model employed in this project involves Faster R-CNN, a state-of-the-art convolutional neural network related to image recognition. By efficiently discriminating between backgrounds and targets within an image, this model enables a detection algorithm to focus on areas that are more likely to contain tobacco signages. Previous analyses of visual data in public health research involving manual image coding are prohibitively costly and time-consuming. The importance and motivation of the project stems from the immediate and comprehensive effect of tobacco advertisements on its sales and consequently on public health. Detected advertisements derived from our model output provide a proof-of-concept for measuring exposure of at-risk communities to tobacco displays.

Who is this presentation for?
It is for people who have some background or interest in CNN's and Object Detection models, or have interest in knowing about AI for good projects. This project entirely belongs to NYU as it was done during the time of my masters as my master final year project.

Prerequisite knowledge:
Basic idea of Convolutional Neural Network, or Object Detection, or Image Classification. A general interest in the area is also good.

What you'll learn?
The data pipeline of extraction of Google Street View Images to Labeling, to Image Processing (Data Augmentation) to Object Detection Architecture (Faster RCNN, YOLO etc.)


I recently joined Ericsson as a Data Scientist. I graduated from New York University as a masters student from an Urban Data Science program last year. I moved to the Bay Area last year and prior to that, I worked in a startup called ARGO labs in New York as a Data Scientist where I was working on using computer vision algorithms to build traffic counter for NYC. Before moving to New York, I lived in Hong Kong for 5 years, where I did my bachelors from Hong Kong University of Science & Tech (HKUST) in Environmental Technology and Computer Science and later worked in HKUST- Deutsche Telecom Systems and Media lab (an Augmented Reality and Computer Vision focused lab) as a Research Assistant. I recently got my paper publication related to improving Navigation System in Smart Glasses at ACM Intelligent User Interfaces (ACM IUI) 2019. I also got my Masters final year group project work on Automated Detection of Street Product Labels accepted as a poster presentation at Applied Machine Learning Days (AMLD) 2019 at EPFL, Switzerland, which I later also presented at ODSC East 2019.