ICML 2024

Better Safe than Sorry

Pre-training CLIP against targeted data poisoning and backdoor attacks

The first defense that keeps CLIP safe during pre-training without giving up accuracy, and without needing any clean data.

Wenhan Yang,  Jingdong Gao,  Baharan Mirzasoleiman

University of California, Los Angeles

Image–caption similarity heatmaps for bird, dog and cat images and captions. Under CLIP, the caption bird 3 matches the beagle image it was poisoned with. Under SafeCLIP, images and captions cluster by category and bird 3 stays far from the beagle image.
Image–caption similarity on a poisoned batch, redrawn from Figure 1 of the paper. The caption “bird 3” is paired with a dog image. CLIP learns that pairing; SafeCLIP keeps the dog image with the dog captions.
93.75%0%

Targeted data poisoning success on CC3M

100%0%

BadNet backdoor success on CC3M

+0.9

Average zero-shot accuracy versus undefended CLIP across 10 datasets. The defense costs nothing.

Overview

CLIP learns from image–caption pairs scraped from the web, and that makes it easy to hijack. Poisoning roughly one in a million training pairs is enough to make it misclassify a chosen image, four orders of magnitude less than a supervised model needs.

The attack works because CLIP's contrastive loss pulls every image toward its caption, including the poisoned ones. SafeCLIP avoids that by never trusting a pair until the model has evidence it belongs together. It first learns images and captions separately, which leaves poisoned pairs far apart. It then trains with the CLIP loss only on the pairs whose image and caption already agree, and keeps learning from the rest one modality at a time. As the model improves, more pairs earn their way into the trusted set.

No clean data is needed. Earlier fixes repair CLIP after the fact: they fine-tune a poisoned model on a trusted clean dataset, or filter data with a clean pre-trained model. SafeCLIP trains from scratch on the poisoned web data itself.

Across CC3M, Visual Genome and MSCOCO, this drops targeted poisoning success from 93.75% to 0% and backdoor success from up to 100% to 0%, while matching undefended CLIP on zero-shot and linear-probe accuracy.

Method

SafeCLIP runs in three stages. The first two are a short warmup that separates poisoned pairs from clean ones; the third is ordinary pre-training with a trust boundary.

Stage 1 · 5 epochs

Learn each modality alone

images captions

The poisoned image clusters with its true category; its caption clusters with the attacker's.

Stage 2 · 1 epoch at lr 5e-6

Split by agreement

risky safe

A two-component Gaussian mixture over image–caption similarity; pairs past threshold t are trusted.

Stage 3 · rest of training

Train with a trust boundary

Safe set CLIP loss on augmented pairs Risky set Image-only and text-only losses

The safe set grows by 1% of the data every epoch as the model improves.

The three stages. Poisoned pairs (red) have low image–caption similarity after warmup, so they land in the risky set, where the model never learns to associate them.
  1. Learn images and captions separately

    5 epochs, image encoder and text encoder trained independently

    Each encoder is warmed up with unimodal contrastive learning. A poisoned image looks like its true category, not like the category named in its caption, so it clusters with its real neighbors and stays far from the adversarial caption. Positives are drawn from a nearest-neighbor pool rather than from augmentations of the same example, which keeps a small group of near-identical poisoned images from forming a tight cluster of their own.

  2. Loosely align, then split

    1 CLIP epoch at learning rate 5×10⁻⁶, about 1% of the base rate

    One gentle pass of the CLIP loss gives clean pairs a modest boost in similarity without letting poisoned pairs catch up. SafeCLIP then fits a two-component Gaussian mixture to the image–caption cosine similarities and puts a pair in the safe set when its probability of belonging to the high-similarity component exceeds a threshold t (0.9 by default).

  3. Pre-train with a growing safe set

    Remaining epochs; the split is refreshed every epoch

    The CLIP loss is applied only to augmented pairs in the safe set, and the risky set keeps contributing through unimodal contrastive learning, so no data is thrown away.

    LSafeCLIP = Lunimodal(Drisky) + LCLIP(Dsafe)

    After each epoch, similarities are recomputed and the safe set grows by 1% of the data. Only the top 30% of pairs are re-scored, so the update costs about a tenth of a CLIP epoch.

Two Gaussian components over image–caption cosine similarity. Pairs to the right of the threshold line form the safe set.

How strict should the split be?

Pick a threshold t. Higher t admits fewer pairs to the safe set and fewer poisons with them. Numbers are from CC1M with 0.05% of pairs poisoned.

11.4%of training pairs start in the safe set
0.000375%of the safe set is poisoned, down from 0.05%

Curves are drawn after Figure 2 of the paper; the numbers come from its threshold ablation (Table 6). t = 0.9 is the default used in all main results.

Results

Attack success rate (lower is better) for a targeted data poisoning attack (TDPA) and four backdoor attacks. Models are ResNet-50 CLIP pre-trained from scratch on each dataset with 0.05% of pairs poisoned.

MethodTDPABadNetLabel-consistentBlendedWaNet

Holds up under stronger attacks

Attack success rate under harder settings than the main table.

Vision transformer encoderThe same defense with ViT-B/32 in place of ResNet-50.
TDPA0%BadNet0%
Multi-trigger backdoorsHybrid and parallel triggers planted at once, on MSCOCO.
HTBA0%All2One0%All2All0%
Ten times more poisonBadNet at a 0.5% poison rate, and targeted poisoning at 0.1%, on MSCOCO.
BadNet 0.5%0%TDPA 0.1%0%
Attacks aimed at the warmupBackdoors targeting unimodal contrastive learning, at up to 0.15% of CC1M. After warmup, 94.5% of backdoored images are already in the risky set.
Linear probe0%

Accuracy is preserved

Average change in top-1 accuracy versus undefended CLIP over 10 downstream datasets, pre-trained on CC3M.

MethodZero-shotLinear probeImageNet 0-shot
CLIP——19.4
RoCLIP−9.7−8.35.8
SafeCLIP+0.9+0.518.2

And it is cheap

Extra training time on top of undefended training, compared with defenses designed for supervised models.

Citation

@inproceedings{yang2024better,
  title     = {Better Safe than Sorry: Pre-training {CLIP} against
               Targeted Data Poisoning and Backdoor Attacks},
  author    = {Yang, Wenhan and Gao, Jingdong and Mirzasoleiman, Baharan},
  booktitle = {Proceedings of the 41st International Conference
               on Machine Learning},
  series    = {Proceedings of Machine Learning Research},
  volume    = {235},
  publisher = {PMLR},
  year      = {2024}
}