Gartner: the majority of privacy incidents are expected to arise from AI inferences soon.

      For many years, safeguarding privacy has primarily involved preventing personal data from leaking. However, Gartner believes this notion is on the verge of change. By 2029, the research firm forecasts that most privacy incidents will not stem from leaked personal data but rather from insights that AI derives about individuals. The concern is shifting from the information a company possesses to the conclusions a model can make from seemingly innocuous tidbits.

      According to Gartner analyst Bart Willemsen, this represents a transition “from data exposure to insight exposure.” AI models can now reconstruct highly personal information, such as health issues or behavior patterns, from data that appears to be anonymous, aggregated, or harmless, without needing to penetrate a database.

      There's an irony in this situation. Companies are retaining less personal data due to regulations and cost-cutting measures intended to minimize risk. However, if an AI can still deduce sensitive information, merely holding less data does little to protect the individual it describes.

      These inference attacks are particularly challenging to detect. Unlike typical breaches that leave evidence—such as stolen files, exposed records, or security alerts—an inferred conclusion leaves no trace. “Inference attacks are especially perilous because they often bypass standard detection methods,” Willemsen noted. “Individuals can be exposed through AI-generated deductions rather than through leaked records, posing privacy risks that threaten data integrity and are difficult to detect, explain, and mitigate.”

      Moreover, these threats often escape the scope of most privacy legislation, which primarily addresses the personal data that companies collect, maintain, and disseminate. A model's guess doesn't fall into any of these categories, creating a growing blind spot as everyday technologies quietly record and analyze more aspects of our lives.

      Gartner advises security leaders to regulate what AI infers rather than only monitoring what it stores. The firm anticipates that spending on data "integrity" protections will equal investments in data confidentiality by 2028, as organizations react to inaccurate, biased, or unauthorized AI-generated profiles.

      The proposed solutions are pragmatic. Incorporate privacy assessments into the development of AI systems, use privacy-enhancing technologies like differential privacy and synthetic data, reduce the amount of data retained, and ensure a human reviews sensitive inferences before AI takes action.

      This final suggestion is crucial because exposure can occur subtly and accidentally, not just through malicious intent. TNW has reported instances where private AI conversations were indexed by search engines without any breach occurring.

      While these insights are predictions rather than certainties, the usual disclaimers apply. This is a forecast from Gartner, which sells research and conference access. The claim that "most privacy incidents" will happen by a certain year is straightforward to state but difficult to verify.

      Nevertheless, the concern is valid. Researchers have long demonstrated that models can re-identify individuals and deduce private characteristics from publicly available information. Regulators are only beginning to catch up, with the EU's AI Act still in its initial stages. Gartner's main assertion is that the industry has been focused on the wrong vulnerabilities for years.