Facebook Taps Artificial Intelligence for Users With Disabilities

Jessica Guynn
USA TODAY, March 23, 2016

MENLO PARK, Calif. – Matt King, a software engineer who has been blind since college, came to Facebook last summer with a mission: to make websites and mobile apps friendlier for people like him with disabilities.

King, 50, uses screen-reader software that turns Web pages and documents into synthesized speech. The challenge he confronts every day: As many as half of websites are nearly impossible for him to browse.

“There are not many products out there where you can say – actually it’s hard to name any – that the experience of using them as a person with a disability is as good as it would be if you didn’t have a disability,” says King who is part of a team at Facebook that focuses on accessibility, such as providing closed captions for videos and keyboard shortcuts for people who can’t use a computer mouse.

Accessibility is a major problem that looms larger as the world’s population grows and ages and as more of everyday life – applying for college or jobs, making a major purchase, getting health information – happens online.

Websites are too seldom built with people with disabilities in mind. But increasingly, tech giants from Microsoft to Yahoo are focusing on making technology more accessible to everyone.

A major push is underway to add accessibility curriculum to computer science programs and to educate software developers on how to build sites and apps that don’t shut out people with disabilities, whether they use screen readers, mouth-controlled joysticks, closed captioning or eye-tracking technology.

“There is certainly more of an interest in just the last five years from these big companies in Silicon Valley,” said Geoff Freed, director of technology projects and Web media standards for the WGBH National Center for Accessible Media.

It’s also a hot topic at the 31st Annual International Technology and Persons with Disabilities Conference being held in San Diego this week.

Facebook is re-engineering its website and mobile apps, and it’s brainstorming a new generation of futuristic products that harness the power of artificial intelligence to improve the experience of Facebook for people with disabilities.

The first is an automated captioning tool launching in April that will help the visually impaired “see” a photo on Facebook by describing what’s in it.

The ever-quickening torrent of photographs and videos flooding Facebook presents a big challenge for the visually impaired. King says he gleans clues from the captions and comments, but “you really feel excluded when you can’t see the picture.”

Even small bits of information can be helpful, King says. When a friend uploads a new profile picture without a caption, the tool tells him there is a person smiling in the photo. When a friend uploads a photo from her phone, it says: “Image may contain: two people, one toddler, smiling, outdoors.”

“These are our very first baby steps,” he says. In time, Facebook hopes to provide a much fuller automated description of photographs and then videos. “It’s really the idea that we are including everybody in the conversation,” he says.


Tech companies aren’t focusing on disability access simply out of altruism. Dependent on growth, they can’t afford to overlook large swaths of the population. In the United States, one out of every five adults has a disability, according to the Centers for Disease Control and Prevention. Some 15% of the world’s population, an estimated 1 billion people, have disabilities.

Another factor: legal risk. Courts are divided on whether websites and mobile apps are legally required to provide equal access to people with disabilities under the 1990 Americans with Disabilities Act, which was enacted before the Internet became as ubiquitous as it is today. But advocates are increasingly filing lawsuits, claiming companies have a legal obligation to make their websites as accessible as a retail store, movie theater or restaurant.

The Department of Justice delayed a plan to issue accessibility regulations until 2018, but in November said: “The inability to access websites put individuals at a great disadvantage in today’s society, which is driven by a dynamic electronic marketplace and unprecedented access to information.”

Online barriers can translate into higher prices if the lowest price is available on an inaccessible website. People with disabilities can’t apply for jobs if the application is only available on a website that isn’t accessible.

“If you can’t get equal access, it will negatively impact your economic status, your privacy, your social life, your independence, even your safety,” said Jonathan Lazar, a computer science professor at Towson University in Maryland.


DeAnn Elliott, a Boston disability advocate, was diagnosed at 28 with retinitis pigmentosa, a genetic disorder that causes gradual retinal degeneration, and declared legally blind at 41.

The Web has opened up many opportunities for people with vision loss, she says. Recent technological advances, such as Apple’s VoiceOver, a gesture-based screen reader that reads a description of everything happening on an iOS device, have turned smartphones into indispensable aids.

Yet too much of the Internet remains tantalizingly beyond her reach. Among the most common obstacles: “captchas,” a security feature that requires users to retype numbers or letters. Audio captchas are often unintelligible, Elliott says.

“It’s terribly frustrating,” Elliott said. “We pay for the same service from our Internet providers as our neighbors but for a fraction of the functionality.”

That lack of functionality is an unnecessary barrier to online access, advocates say. Many websites, such as those run by government, libraries and museums, are required to be accessible.

Making technology accessible benefits everyone, says Daniel Goldstein, counsel for the National Federation of the Blind. Think curb cuts and wheelchair ramps for strollers, captions for television broadcasts in noisy bars or the software used to create e-books.

“What’s needed are things like: ‘It is the policy of our company to build accessibility in from the beginning of the design process,’ ” Goldstein said.

Facebook is seeking to embed that kind of awareness in its corporate culture.


Steps from his desk in Facebook’s Menlo Park, Calif., headquarters is an “empathy lab,” a row of devices that browse the social network using keyboard shortcuts, braille or the sound of a human voice. The devices are strategically placed along a busy walkway to remind engineers to build – or, in Facebook speak, hack – accessibility into all products.

That is King’s life work. He was born with retinitis pigmentosa and considered legally blind, though as a child he was able to ride a bike and hold down a route delivering newspapers. While studying electrical engineering and music at the University of Notre Dame, King lost his sight completely, but never his drive. He began tinkering with screen readers to improve the technology. At IBM, he championed equal access for people with disabilities. He’s also a three-time Paralympian, one of the world’s top tandem cyclists and a classical pipe organist.

King makes good use of assistive technology. While most software engineers have large monitors on their desks, King has a sound mixing board like those used by a recording engineer or a professional DJ. The mixer lets him control the volume of the screen reader and other audio coming from three laptops – two PCs and a MacBook – and two phones – one iPhone, one Android – using the mic on his stereo headset so he can listen to a phone call in one ear and the screen reader in the other.

King was recruited from IBM by Jeff Wieland, who started Facebook’s accessibility team five years ago. King, who navigates the sprawling campus with a white cane, says he was taken with Facebook’s mission to connect every person on the planet.

“I don’t think there is any other company in the world where accessibility is that core to the mission, where it’s impossible to accomplish the mission without making accessibility great,” he says.

Using Facebook was a frustrating experience for people with vision loss when King signed up for it in 2009. The social network was riddled with buttons and graphics that were not labeled so he had no idea what they were for. “It took me several hours to do what would have taken someone else 15 or 20 minutes,” he recalls.

Today, King can skim his news feed as quickly as a sighted person. King predicts artificial intelligence will power even greater advances for people with disabilities.

“This is a problem,” he says, “that as machines get smarter, that machines can solve.”

Reproduced from http://www.usatoday.com/story/tech/news/2016/03/23/facebook-accessibility-people-with-disabilities/82026554/