Researchers at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) have introduced a breakthrough method called DP-Fusion, designed to solve a critical vulnerability in artificial intelligence: the accidental leakage of sensitive information during live interactions.
Presented at the prestigious ICLR 2026 conference in Rio de Janeiro, DP-Fusion addresses the growing security risks faced by industries that rely on Large Language Models (LLMs) to process highly regulated data.
The Problem: The “Inference Gap” in AI Privacy
While much of the scientific community has focused on protecting data during the training phase of AI, a significant security gap remains during inference —the stage where a user actually interacts with a live model.
As Generative AI moves from experimental tools to essential infrastructure, it is being deployed in high-stakes environments, including:
– Healthcare: Analyzing patient medical records.
– Finance: Processing private client transactions and data.
– Government: Handling classified or sensitive documentation.
In these settings, a model might inadvertently “leak” private details through its generated responses. Current methods to prevent this, such as scrubbing or paraphrasing text, lack formal mathematical proof of security, leaving organizations vulnerable to sophisticated data extraction.
How DP-Fusion Works: A Mathematical Shield
DP-Fusion distinguishes itself by offering mathematically provable privacy guarantees at the “token level.” Rather than simply trying to hide words, it uses a sophisticated four-step process to ensure privacy without sacrificing the intelligence of the AI:
- Identification: Sensitive tokens (bits of information) in the input are labeled.
- Baseline Establishment: The model runs without those sensitive tokens to create a “neutral” baseline.
- Contextual Processing: The model runs again with the sensitive tokens included.
- Distribution Blending: The system blends the two outputs so that the final response remains mathematically bounded by the baseline.
This ensures that the sensitive information is effectively hidden within the output, while the AI’s ability to provide coherent, useful answers remains intact.
Superior Performance and Versatility
One of the most significant achievements of DP-Fusion is its ability to solve the “privacy-utility trade-off.” Usually, increasing privacy makes an AI less coherent (a concept known as perplexity ).
DP-Fusion achieves six times lower perplexity than competing privacy methods, meaning it produces significantly more useful and readable text while maintaining stronger security.
Furthermore, the technology offers two distinct advantages:
– Customizable Control: Operators can use a single parameter to balance protection. Setting it to zero provides maximum privacy (hiding tokens entirely), while higher values allow for more “utility” or accuracy depending on the use case.
– Dual-Purpose Defense: By treating data from untrusted external sources as “sensitive,” DP-Fusion also acts as a defense against prompt injection and jailbreak attacks, protecting the model from adversarial manipulation.
The Economic and Operational Context
The urgency of this research is underscored by the sheer scale of the AI market. According to McKinsey, AI inference is set to account for over 40% of total data center demand, growing at an annual rate of 35%.
With the global AI inference market projected to reach between $250 billion and $350 billion by 2030, the “attack surface”—the number of moments where a live model touches real-world data—is expanding exponentially. As inference represents up to 90% of the total lifetime cost of an AI system, securing this phase is not just a matter of privacy, but of long-term operational viability for the global digital economy.
Conclusion
By providing a mathematically rigorous way to protect data during live AI interactions, DP-Fusion paves the way for the safe adoption of Generative AI in highly regulated sectors like medicine and finance. This research marks a pivotal shift from “probabilistic” privacy to “provable” security in the era of large-scale AI deployment.
