ADS-C: Antidistillation Sampling for Classification

Threat model: soft-label model extraction

When a deployed classifier serves probability vectors over an API, each query leaks structural dark knowledge about how the teacher relates classes. An adversary can query the API and train a surrogate student via knowledge distillation—extracting proprietary model capability at a fraction of the training cost.

Margin-aware antidistillation sampling (ADS-C)

Antidistillation sampling was originally introduced for autoregressive language models. Directly applying uncalibrated exponential tilting to image classifiers fails because well-trained classifiers exhibit sharp confidence regimes: small perturbations remain inert, while larger perturbations flip the teacher’s top-1 prediction faster than they degrade the student.

ADS-C resolves this by bounding each perturbation using a closed-form per-input confidence margin budget. For every query $x$, the defense computes the exact margin between the teacher’s top-1 logit and runner-up classes, poisoning the non-top probability structure while mathematically guaranteeing that no competing class can overtake the served top-1 prediction.

Empirical results & companion review

Across three standard vision benchmarks and multiple student architectures, ADS-C preserves 100% of served top-1 predictions while reducing distilled-student accuracy by:

• 29.6 percentage points on CIFAR-10
• 17.4 percentage points on CIFAR-100
• 13.3 percentage points on Tiny-ImageNet

In our companion review manuscript (“Reversing the Incentive to Distill: A Review of Test-Time Antidistillation Defenses for Classification”, under review, 2026), we built a unified matched-cost benchmark and a method-agnostic top-1-preservation wrapper across prior test-time defenses—showing that directed perturbations under top-1 preservation drive distilled students 34.5 pp (CIFAR-10) and 47.7 pp (CIFAR-100) below the hard-label baseline.

Open-source PyTorch harness

The complete defense and its multi-architecture evaluation harness are released as open-source PyTorch code in the-kas-lab/ADS-C, alongside the preprint arXiv:2607.15467 and my earlier RIT Master’s Thesis.

@misc{ullah2026adsc,
  title = {ADS-C: Antidistillation Sampling for Classification},
  author = {Khawaja Abaid Ullah and Mohammad Javad Khojasteh},
  year = {2026},
  eprint = {2607.15467},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2607.15467}
}