Meta's Brain2Qwerty interprets typed sentences directly from brain activity.

      Meta claims it can convert brain activity into written sentences without the need for surgery. This advancement is significant, but there is a drawback: the system learns from typing, which the users it aims to assist cannot perform.

      On Monday, Meta introduced the second version of Brain2Qwerty, a non-invasive technology that interprets brain signals generated during typing and reconstructs the words. A participant wears a magnetoencephalography (MEG) scanner, a helmet-like device that detects the minute magnetic fields produced by the brain. An AI system then processes this information.

      The results are noteworthy. Brain2Qwerty v2 achieved an average word accuracy of 61%, with the best participant reaching 78%, according to Meta. Previous non-invasive systems had accuracy rates in the single digits, while last year’s version 1 peaked at around 48%.

      To develop this system, Meta trained it using approximately 22,000 sentences typed by nine volunteers, each of whom used the scanner for about 10 hours. The research was conducted at a center in San Sebastián, Spain, as reported by Gizmodo.

      The methodology relies on technology similar to that of ChatGPT. First, the signal from the scanner is converted into characters, which are then formed into words by a subsequent model. A large language model, specially adapted to use the brain data, predicts the intended sentence based on context, akin to how a smartphone anticipates the next word.

      Meta asserts this is the first instance of a large language model successfully decoding noisy brain activity into full sentences. The company even utilized AI agents to refine its decoding process, though human engineers made the final decisions.

      The choice of scanner proved more crucial than anticipated. Meta compared both MEG and the more common, cost-effective EEG. MEG demonstrated significantly better performance, achieving a character error rate of 29% compared to 65% for EEG.

      Meta has made the code and dataset open-source, reflecting a broader initiative towards public AI-for-science efforts. The project is portrayed as a potential aid for the millions who lose their ability to speak due to brain injuries or illnesses.

      However, several substantial limitations remain. The system is far from being commercialized. The MEG scanner is large, expensive, and intended for hospital use rather than personal settings. It also cannot provide real-time results; the models require a complete typing session before generating any output, resulting in no immediate feedback.

      There is an additional challenge. Brain2Qwerty relies on the brain signals of individuals who are typing, but its target audience—those severely affected by paralysis or disease—cannot type at all. Meta acknowledges this limitation. Individuals with some mobility may gain assistance, but those completely locked-in are unlikely to benefit. Addressing this issue would necessitate a shift towards interpreting imagined movements rather than actual keystrokes.

      The existing design must also know the exact timing of key presses, and Meta describes the path toward seamless, trigger-free decoding as "uncertain."

      Currently, invasive methods outperform non-invasive ones in terms of results. Implanted systems have achieved much higher accuracy, with some recent surgical applications reaching 92% accuracy at the sentence level. One surgical interface allowed a man with ALS to maintain a full-time job by decoding his attempted speech with remarkable precision. Companies like Neuralink and its competitors are competing to commercialize similar implants.

      Meta's proposition is that it can bridge the gap without invasive techniques, as accuracy continues to improve with more data fed into the models. While this may hold true, and the open approach could facilitate broader testing, a brain-reading system that occupies a room, waits for completion, and requires typing is still far from being an essential tool. Meta’s larger AI initiatives often arrive with significant fanfare, but this development showcases a genuine advancement in the lab, while also reminding us of the considerable distance remaining before it can be applied in clinical settings.