Google Gemini simplifies web browsing for individuals with vision and hearing impairments.

      Android devices have included a built-in screen reader feature named TalkBack for many years. It aids individuals with vision impairments in understanding the content on their phone screens and allows for voice control. In 2024, Google incorporated its Gemini AI to provide users with more comprehensive descriptions of images.

      Google is enhancing this feature with an additional layer of interactive ease for users. Until now, Gemini could only describe images. Now, while looking at images, users can ask follow-up questions and engage in more detailed conversations about them.

      How does it assist users with vision challenges?

      "The next time a friend sends you a photo of their new guitar, you can receive a description and inquire about the make and color, or even what else is present in the image," explains Google. This builds on the accessibility improvements that integrated Gemini into the TalkBack system late last year.

      The TalkBack menu on Android now includes a specific Describe Screen feature that places Gemini in control. For instance, if users are browsing a clothing catalog, Gemini will not only describe what is visible on the screen but also respond to pertinent questions.

      Users might ask questions like, “Which dress would be best for a cold winter night outing?” or “What sauce pairs well with a sandwich?” Gemini will also be able to analyze the entire screen, providing detailed product information or indicating whether there are any discounts available.

      Enhancing captions and improving text zoom

      In the Chrome browser, Google is making slight improvements to the auto-generated captions for videos. For example, if a user is watching a football match, the captions will not simply echo the commentator’s words but will also reflect their emotions and expressions.

      For instance, instead of just saying "goal," users with hearing impairments will see a more emphatic "goooaaal" to convey emotional intensity. Google is branding these as Expressive Captions.

      In addition to spoken words, they will now also encompass significant sounds like whistles, cheers, or even the speaker clearing their throat. Expressive captions will be accessible on all devices running Android 15 or later in the US, UK, Canada, and Australia.

      Another notable update coming to the Chrome browser is adaptive text zoom, which enhances the existing Page Zoom functionality on Android phones. Now, when users enlarge text, it won't alter the layout of the rest of the webpage.

      “You can customize how much you want to zoom in and easily apply this preference to all the pages you visit or just specific ones,” states Google. Users will be able to adjust the zoom range using a slider located at the bottom of the page.