Gartner: the majority of privacy breaches will soon arise from AI inferences.

      For many years, ensuring privacy has primarily focused on preventing personal data from being exposed. However, Gartner suggests that this perspective is about to change. By 2029, the majority of privacy breaches will likely result not from direct data leaks but from inferences made by AI about individuals, according to recent predictions from the research firm. The concern now extends beyond the data a company possesses; it's about the insights a model can derive from seemingly innocuous fragments of information.

      This marks a transition "from data exposure to insight exposure," as described by Gartner analyst Bart Willemsen. Models can reconstruct deeply personal details, such as health issues or behavioral tendencies, using data that appears anonymous, aggregated, or benign, without ever accessing a database.

      There’s an ironic twist here: Companies are collecting less personal data due to regulatory pressure and cost considerations, which is intended to mitigate risk. However, if AI can still deduce sensitive information, holding less data doesn't significantly safeguard the individual it pertains to.

      Inference attacks are particularly challenging to detect. Traditional breaches leave evidence: stolen documents, exposed records, or alerts. In contrast, inferred information produces none of these markers. Willemsen highlighted the dangers of inference attacks, noting that they can bypass standard detection methods, exposing individuals through AI-generated insights instead of through data leaks, which can pose privacy risks that threaten data integrity and are difficult to identify, clarify, and address.

      Moreover, these threats often fall outside most privacy regulations, which primarily oversee personal data that organizations collect, maintain, and distribute. An AI's inference doesn’t fall under this category, creating an expanding blind spot as common tools continually record and analyze more of our daily activities.

      Gartner advises security leaders to focus on regulating AI's conclusions, rather than merely its stored data. The firm anticipates that, by 2028, investments in data "integrity" measures will match those dedicated to data confidentiality, as organizations react to the issues of inaccurate, biased, or unauthorized AI-generated profiles.

      Gartner's recommended solutions are pragmatic: incorporate privacy checks during the development of AI systems; utilize privacy-enhancing technologies like differential privacy and synthetic data; limit data collection; and ensure human oversight before an AI acts on sensitive inferences. This last aspect is significant since exposure can occur quietly and inadvertently, not just through malicious intent. For instance, TNW reported instances where private AI conversations were indexed by search engines without any breach taking place.

      As with any predictions, the usual disclaimers apply. This is a projection from Gartner, which profits from its research and the conferences it hosts. Claims about "most privacy incidents" by a certain year are easy to make but hard to substantiate. Nonetheless, the underlying concern is legitimate; researchers have long demonstrated that models can re-identify individuals and infer private characteristics from openly available data. Regulatory bodies are only beginning to address these issues, with the EU’s AI Act still in the implementation stages. Gartner's main assertion is that the industry has been focused on guarding the wrong issues for years.