Apple is interested in PrismML's on-device AI technology for the iPhone.

      Apple is engaged in preliminary discussions with PrismML, a startup that develops technology to condense large AI models for use directly on mobile devices. This on-device AI approach could allow Apple to reduce its reliance on cloud processing for Siri's functionalities.

      Babak Hassibi, CEO of PrismML, shared with CNBC that both Apple and other firms are assessing its technology. He mentioned that the talks are still in their initial stages, and it's uncertain where they might lead, although he noted, “things are progressing nicely.” Apple did not provide any comments on this matter. The Information had first reported on Apple’s interest in PrismML last week.

      PrismML is a Khosla Ventures-supported spinoff from the California Institute of Technology, which holds the underlying patents and grants exclusive licenses to PrismML. The startup successfully raised a $16.25 million seed round in March.

      On Tuesday, PrismML introduced Bonsai 27B, which is a compressed version of Alibaba’s open-source Qwen model, rather than a new one created from scratch. The startup managed to compress it from approximately 54GB to as little as 3.9GB.

      PrismML offers two variants under a free license: a ternary model designed for laptops and a smaller 1-bit model, around 3.9GB, tailored for the memory capacity of an iPhone 17 Pro. According to PrismML, it is the first model of this size capable of running on a mobile device.

      The innovation lies in how the model retains its internal values. PrismML has reduced each value from 16 bits to just one or three possible options, leading to a reduction in memory usage by 10 to 15 times, a speed increase in responses by six to eight times, and a decrease in energy consumption by three to six times.

      However, this compression comes with trade-offs. Hassibi indicated that the smaller models experience a slight dip in performance. Initially, factual recall is impacted, followed by reasoning, math, and coding abilities. PrismML claims that its ternary model retains about 95% performance, while the 1-bit model maintains around 90%.

      The timing of this release is notable. PrismML launched the model just one day after Apple initiated the public beta for iOS 27, which includes a long-awaited revamp of Siri. Apple aims to enhance Siri’s competitiveness against assistants from OpenAI and Anthropic.

      Increased on-device processing could benefit Apple. The company currently routes complex requests to cloud services, and localizing more operations would reduce latency, lower cloud expenses, and align with its privacy messaging. Additionally, certain features might function offline.

      There's also a financial aspect to consider. Morgan Stanley projects that Apple’s memory costs might rise significantly in the 2027 fiscal year, anticipating that the company could increase iPhone prices to maintain profit margins. Smaller models could enable Apple to integrate effective AI into constrained hardware without incurring higher memory costs.

      Analysts have suggested caution regarding the claims. Tarun Pathak from Counterpoint Research pointed out that the true evaluation would involve analyzing millions of queries across a wide range of devices. Phil Solis from IDC emphasized that power consumption remains a significant concern, as an often-used model could potentially deplete battery life.

      The release also contributes to ongoing discussions about whether efficiency improvements will reduce the demand for memory and data center chips. Gil Luria, an analyst at D.A. Davidson, remarked that while shrinking models might change the landscape, they wouldn't eliminate the need for processors; instead, some processing duties would shift from data centers to smartphones, reflecting a general move towards edge AI.

      Hassibi mentioned that Google’s open-source Gemma model is next in line for development, followed by larger frontier models. He emphasized the importance of local intelligence that can operate rapidly.