The Challenge of Adversarial Attacks in Generative AI
Text-to-image models are increasingly vulnerable to adversarial prompts—carefully crafted inputs designed to bypass safety filters or force the generation of harmful, biased, or copyrighted content. Traditional defenses often rely on rigid keyword filtering or post-generation moderation, which are easily circumvented by sophisticated prompt engineering. The DiSCO (Distribution-guided contrastive prompt optimization) framework shifts the defense strategy from reactive filtering to proactive prompt optimization.
How DiSCO Works: Distribution-Guided Optimization
DiSCO operates by treating the prompt as a variable that can be optimized to align with safe, high-quality distributions. The core mechanism involves:
- Contrastive Optimization: The framework utilizes a contrastive learning objective that pushes the latent representation of a user's prompt away from known adversarial clusters and toward a distribution of 'safe' and 'benign' prompt spaces.
- Distribution Guidance: Instead of simply blocking a prompt, DiSCO guides the model to interpret the user's intent through a safer lens. By mapping the input prompt into a latent space informed by a distribution of safe training data, the system effectively 'sanitizes' the intent before it reaches the diffusion model's generation pipeline.
- Preserving Fidelity: A critical trade-off in adversarial defense is the degradation of image quality or prompt adherence. DiSCO addresses this by ensuring the optimization process maintains the semantic integrity of the user's original request, ensuring that the output remains relevant to the user's intent while stripping away the adversarial 'noise' that triggers harmful generation.
Practical Implications for Model Security
By integrating DiSCO into the inference pipeline, developers can create a more robust layer of defense that does not rely on static blacklists. This approach is particularly effective against 'jailbreak' attempts that use complex, multi-step prompt structures to confuse standard safety classifiers. Because DiSCO operates at the prompt-embedding level, it provides a scalable way to harden models against evolving adversarial tactics without requiring full retraining of the underlying diffusion model.