Why AI Vision Apps Can Struggle With Everyday Tasks
Imagine checking whether your dinner is fully cooked using only your smartphone. An AI assistant can describe what its camera sees, but if the answer does not explain what you should do next, you may need to ask the same question several more times.
That is one of the problems identified by researchers at the University of Maryland (UMD), who examined how blind people interact with AI-powered visual assistance. Their findings suggest that even when an app can describe an image, it may struggle to provide the specific information someone needs to complete a task.
The study, published in connection with the 2026 ACM Conference on Human Factors in Computing Systems, examined Be My AI, an artificial intelligence feature within the Be My Eyes accessibility app. Researchers investigated whether blind users could control the information they received and how they responded when the AI failed to meet their needs.
The findings highlight an important distinction in mobile technology: recognizing what appears in a photograph is not the same as understanding why a person took that photograph.
Be My AI: How the Visual Assistance Feature Works
Be My Eyes has helped blind and low-vision people access visual information since 2015 by connecting them with sighted volunteers who can provide assistance through a smartphone camera. In 2023, the service introduced Be My AI, which uses artificial intelligence to describe photographs and answer questions without requiring a volunteer to respond.
The feature can help with everyday activities such as identifying objects, examining food, locating belongings and understanding a scene. Instead of waiting for a human volunteer, users can receive an AI-generated description and continue the conversation by asking follow-up questions.
However, the University of Maryland research found that the experience can become frustrating when the system provides extensive background information instead of addressing the user’s immediate concern.
For example, someone looking for a set of keys may not need a complete description of a room. They need information about where the keys might be located. An answer that describes furniture, walls and unrelated objects can make it harder to identify the detail that matters.
This is particularly important for assistive technology because spoken responses must be listened to rather than scanned quickly on a screen. A lengthy answer can turn a simple question into a time-consuming interaction.
What the University of Maryland Study Discovered
The research involved 11 participants who identified as blind. They first practiced using Be My AI in a laboratory, then used the app during their everyday activities for 10 days before discussing their experiences with the research team.
Researchers analyzed 418 conversational interactions to understand how participants asked questions, interpreted responses and tried to obtain more useful information.
One notable finding concerned the difference between the length of users’ questions and the AI’s answers. Participants typically asked questions containing 10 to 20 words, while the system’s responses were more than 10 times as long.
Some conversations also extended to 21 back-and-forth exchanges. For people listening to every response through audio, that amount of interaction can create unnecessary effort when the original task requires only a short, direct answer.
The researchers also found that more than half of the participants considered the app’s tendency to begin with a broad description of the photograph a serious problem. That initial description could include information users had not requested, forcing them to continue questioning the system to obtain the relevant detail.
Hernisa Kacorri, an associate professor in the University of Maryland’s College of Information and a study co-author, explained that improved image descriptions do not automatically translate into useful assistance when an AI system fails to recognize the user’s intention.
The study’s findings do not mean that Be My AI is entirely ineffective. Rather, they identify a gap between describing visual information and delivering answers tailored to a person’s immediate goal.
How Blind Users Adapt to AI’s Limitations
Participants did not simply abandon the technology when it failed to provide suitable answers. They developed different ways to guide the AI toward information that was more relevant to their needs.
One approach involved explaining the purpose behind a question. Rather than asking for a general description, users could tell the app what they were trying to accomplish, helping establish a clearer context for the conversation.
Another strategy was to divide a complicated task into smaller questions. A participant examining bread dough, for example, asked about its size, surface, air bubbles and flour separately instead of expecting the AI to assess every detail in a single response.
This approach could make the information easier to evaluate, but it also required additional time and effort. Users were effectively developing their own techniques to work around limitations in the app’s design.
Researchers noted that these adaptations demonstrate how people with disabilities develop practical solutions when technology does not fully meet their needs. However, requiring users to master complicated questioning strategies creates another barrier to accessibility.
A well-designed assistive app should reduce unnecessary effort rather than expect users to become experts at prompting artificial intelligence. Understanding the user’s objective should be part of the technology’s responsibility, not an additional task placed entirely on the person using it.
Frequently Asked Questions About AI Visual Assistance
Is Be My AI a replacement for sighted volunteers?
No. Be My AI provides automated descriptions and answers about photographs, while Be My Eyes also connects users with sighted volunteers for human assistance. The study highlights limitations in the AI feature rather than suggesting that all forms of visual assistance are ineffective.
Can blind users rely on AI to make important decisions?
AI visual assistance can provide useful information, but the research shows that answers may not always address the user’s goal adequately. For cooking, medication identification or other situations where mistakes could cause harm, users should not treat an AI description as a guarantee of safety or correctness.
How can users get more relevant answers from Be My AI?
Users can explain what they are trying to accomplish, ask specific questions and divide complicated tasks into smaller steps. These techniques may improve the relevance of responses, although they do not eliminate the limitations identified in the research.
What AI App Developers Should Improve Next
The University of Maryland study points toward a broader challenge for developers of accessibility-focused mobile applications. Future improvements should prioritize understanding user intent, answering direct questions first and avoiding lengthy descriptions when a concise response would be more helpful.
Developers could also explore ways to let users choose how much detail they receive, making it easier to request a brief answer or a more comprehensive explanation. Such controls would give people greater influence over the information delivered through speech.
For anyone using Be My AI today, a practical next step is to state the goal clearly, ask focused follow-up questions and verify critical information when accuracy matters. Users should also check the app’s current official listing and update notes for any newly introduced accessibility controls or improvements.
The research paper, titled “Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Users,” offers a useful direction for future assistive AI development. Its central lesson is that better image recognition alone is not enough: technology must also understand what people are trying to achieve and provide information in a form they can use.
As AI-powered accessibility tools continue to evolve, the next major improvement may not be a longer or more detailed description. It may be a shorter, more relevant answer that helps someone complete an everyday task with greater independence.
