Autoethnographic study explores utility of AI tools for accessibility

Generative artificial intelligence tools such as ChatGPT, an AI-powered language tool, and Midjourney, an AI-powered image generator, can potentially help people with various disabilities. These tools can summarize content, compose messages, or describe images. Yet the extent of this possibility is an open question, since, in addition to revealing regular mistakes and failures in basic reasoning, these tools can perpetuate enabling biases.

This year, seven researchers from the University of Washington conducted a three-month autoethnographic study; drawing on their own experiences as people with and without disabilities -; To test the utility of AI tools for accessibility. While the researchers found cases where the tools were helpful, they also found significant problems with the AI ​​tools in most use cases, whether they were creating images, composing Slack messages, summarizing writing, or trying to improve the accessibility of documents.

The team presented its findings in October. 22 at the ASSETS 2023 conference in New York.

“When technology changes rapidly, people with disabilities risk being left behind,” said senior author Jennifer Mankoff, a UW professor in the Paul G. Allen School of Computer Science and Engineering. “I’m a really strong believer in the value of first-person accounts to help us understand things. Because we had a lot of people in our group who could experience AI as disabled people and see what worked and what didn’t, we thought we had It was a unique opportunity to tell a story and learn about it.”

The group presents its research in seven vignettes, often combining experiences into a single account to preserve anonymity. For example, in the first account, “Mia,” who has occasional brain fog, deploys ChatPDF.com, which summarizes PDFs, to help with the task. While the tool was occasionally accurate, it often gave “totally wrong answers.” In one case, the tool was both inaccurate and enabling, changing the argument of a paper to make it seem like researchers should be talking to caregivers rather than people with chronic illnesses. “Mia” was able to catch it, since the researcher knew the paper well, but Mankoff said that such subtle errors are some of the “most insidious” problems in using AI, as they can easily be overlooked.

Yet in the same vignette, “Mia” used the chatbots to create and format a paper reference they were working on while experiencing brain fog. AI models still make mistakes, but the technology has proven effective in this regard.

Mankoff, who has spoken publicly about being diagnosed with Lyme disease, contributed to this account. “Using AI for this task still required work, but it reduced the cognitive load. By switching from a ‘generation’ task to a ‘validation’ task, I was able to avoid some of the accessibility issues I was facing,” says Mankoff.

The results of other tests selected by the researchers were equally mixed:

• One author, who is autistic, found that AI helped her write Slack messages at work without spending too much time on words. Peers found the messages “robotic,” yet the tool still made the author feel more confident in these interactions.
• Three authors have attempted to use AI tools to increase the accessibility of content such as tables in research papers or slideshows for classes. AI programs were able to state accessibility rules but could not apply them consistently when creating content.
• Image generating AI tools helped an author with aphantasia (inability to visualize) interpret images from books. Yet when they used the AI ​​tool to create an illustration of “differently disabled people looking happy but not at a party,” the program could conjure up only filled images of people at a party that included ableist anomalies, such as an amputated arm and an amputated prosthetic leg. rest

I was amazed at how dramatically the results and outcomes varied depending on the task. In some cases, such as creating an image that makes people with disabilities look happy, even through specific prompting -; Can you do it this way? -; The results did not achieve what the author intended.”

Kate Glasgow, Lead author, UW doctoral student at the Allen School

The researchers noted that more work is needed to address the issues revealed in the study. A particularly critical issue is developing new ways to validate the products of AI tools for people with disabilities, because in many cases when AI is used for accessibility, either the source documents or the AI-generated results are not accessible. ChatPDF “Mia” and while “Jay”, who is legally blind, used an AI tool to generate code for a data visualization. He couldn’t verify the result himself, but a colleague said it “made no sense.” The frequency of AI-induced errors, Mankoff said, “makes accessibility validation research particularly important.”

Mankoff plans to research ways to document the types of affordances and affordances present in AI-generated content, as well as investigate issues in other areas such as AI-written code.

“Whenever software engineering practices change, there’s a risk that apps and websites become less accessible without good defaults,” Glazko said. “For example, if AI-generated code is accessible by default, it can help developers learn about and improve the accessibility of their apps and websites.”

Co-authors of the paper are Momona Yamagami, who completed the research as a UW postdoctoral scholar at the Allen School and is now at Rice University; Ashaka Desai, Kelly Avery Mack and Venkatesh Potluri, all UW doctoral students at the Allen School; and Zhuhai Xu, who completed this work as a UW doctoral student in the School of Information and is now at the Massachusetts Institute of Technology. This research was funded by Meta, the Center for Research and Education on Accessible Technology and Experience (CREATE), Google, a NIDILRR ARRT grant, and the National Science Foundation.