COMPRESSION ARTIFACTS AS BACKDOORS

In a backdoor attack on a machine learning model, an adversary produces a model that performs well on normal inputs but outputs targeted misclassifications on inputs containing a small trigger pattern. Model compression is a widely-used approach for reducing the size of deep learning models without much accuracy loss, enabling resource-hungry models to be compressed for use on resource-constrained devices. In this paper, we study the risk that model compression could provide an opportunity for adversaries to inject stealthy backdoors. We design stealthy backdoor attacks such that the full-sized model released by adversaries appears to be free from backdoors (even when tested using state-of-the-art techniques), but when the model is compressed it exhibits highly effective backdoors. We show this can be done for two common model compression techniques — model pruning and model quantization. Our findings demonstrate how an adversary may be able to hide a backdoor as a compression artifact, and show the importance of performing security tests on the models that will actually be deployed not their precompressed version.

Security concerns are an everyday worry. Mobile phones now use AI facial recognition which is convenient but most likely not as robust as fingerprint scanners.

Given all of the scandals over hacking it’s very possible that malware has been implanted that waits for a trigger, Once triggered bank machines can be jackpotted or worse. The training of an AI with a subversive element an be hard to identify. Since we assume the model tester cannot control or observe the model training process, we only consider defenses that take a trained model as input and predict whether that model contains a backdoor.

foxconns-iphone-12-production-on-schedule

https://arxiv.org/abs/2104.15129