How to protect AI systems against image-scaling attacks Posted on : Aug 13 - 2020

We usually don’t expect the image of a teacup to turn into a cat when we zoom out. But in the world of artificial intelligence research, strange things can happen. Researchers at Germany’s Technische Universität Braunschweig have shown that carefully modifying the pixel values of digital photos can turn them into a completely different image when they are downscaled.

What’s concerning is the implications these modifications can have for AI algorithms.

Malicious actors can use this image-scaling technique as a launchpad for adversarial attacks against machine learning models, the artificial intelligence algorithms used in computer vision tasks such as facial recognition and object detection. Adversarial machine learning is a class of data manipulation techniques that cause changes in the behavior of AI algorithms while going unnoticed to humans.

In a paper presented at this year’s Usenix Security Symposium, the TU Braunschweig researchers provide an in-depth review of staging and preventing adversarial image-scaling attacks against machine learning systems. Their findings are a reminder that we have yet to discover many of the hidden facets — and threats — of the AI algorithms that are becoming increasingly prominent in our daily lives.

Adversarial image-scaling

When trained on many examples, machine learning models create mathematical representations of the similarities between different classes. For instance, if you train a machine-learning algorithm to tell the difference between cats and dogs, it will try to create a statistical model that can tell whether the pixels in a new image are more like those found in the dog or cat images. (The details vary between different types of machine learning algorithms, but the basic idea is the same.)

The problem is, the way these AI algorithms learn to tell the difference between different objects is different from how human vision works. Most adversarial attacks exploit this difference to create small modifications that remain imperceptible to the human eye while changing the output of the machine learning system. For instance, in the following image, adding a carefully crafted layer of noise will cause a well-known deep learning algorithm to mistake the panda for a gibbon. To the human eye, both the right and left images appear to be the same panda.

But while classic adversarial attacks exploit peculiarities in the inner workings of the AI algorithm, image-scaling attacks focus on the preprocessing stage of the machine learning pipeline (we’ll get to this in a bit). This is why the researchers have titled their paper “Adversarial preprocessing.”

“While a large body of research has studied attacks against learning algorithms, vulnerabilities in the preprocessing for machine learning have received little attention so far,” the researchers write in their paper. View More