Developers at Harvard have created an AI application designed to assist visually impaired individuals in navigating their surroundings.

      Mobilio utilizes a smartphone's camera, GPS, and motion sensors to assist users in navigating around obstacles.

      AI accessibility tools have gradually emerged as some of the most practical applications of the technology. We have observed smartphones aiding visually impaired users in locating nearby objects and smart glasses allowing them to connect with individuals who can verbally describe their surroundings. Researchers at Harvard are advancing this concept with a navigation-focused app.

      Named Mobilio, the application employs a smartphone’s camera, GPS, motion sensors, and, when available, LiDAR to guide blind and low-vision users in following routes, maintaining walkable pathways, and avoiding obstacles. The team at Harvard’s John A. Paulson School of Engineering and Applied Sciences developed this system after conducting a survey with over 100 individuals who are blind or have low vision to understand their navigation technology needs.

      Initial testing results look promising. In a study with 14 participants, Mobilio used alongside a white cane allowed users to complete an outdoor route approximately 13% faster than using Google Maps with a cane. For indoor navigation, contacts with obstacles were reduced by about 41%.

      How Mobilio transforms a smartphone into a walking guide

      Conventional navigation apps might indicate when to turn, but that information alone is not very helpful for safely following a sidewalk or avoiding an obstacle right in front.

      Mobilio continuously assesses the environment using the phone’s camera while GPS and motion sensors monitor movement. Directional beeps guide users on where to navigate and the system can modify those signals based on user responses.

      Researchers also trained its computer-vision model from a pedestrian's viewpoint, as many current street datasets are collected from vehicles and do not effectively represent sidewalks, crosswalks, and other pedestrian areas.

      AI is truly finding a valuable role in accessibility

      This isn't the first instance of AI addressing these issues. Penn State’s NaviSense uses a smartphone to help visually impaired users find specific objects through sound and vibration, while Meta’s Ray-Ban smart glasses work with Be My Eyes to enable remote volunteers to assist blind users in navigating their surroundings.

      Mobilio emphasizes the journey itself, utilizing the phone as a constant navigation tool rather than merely identifying nearby objects. The technology remains experimental, and with only 14 participants in the initial study, Harvard intends to conduct broader testing in Boston, exploring how Mobilio might progress beyond the research phase.