Show and Tell is an Alexa feature designed to help blind and low vision customers identify grocery items.

"Alexa, what am I holding?"

After losing her sight suddenly at age 41, Stacie Grijalva faced a new reality, one that inspired a new mission. A mechanical engineer by background, Grijalva was committed to exploring how assistive technology could make life a little easier for the blind and visually impaired, and sharing that information with others. “My job is to help people with visual impairments see how technology can affect people’s lives and make them feel better about what they do on a day-to-day basis.”

Alexa-enabled devices are already a big help around the house for Grijalva, from changing the channel on her Fire TV to using her voice to turn off lights and appliances. Now, an Echo Show feature makes it easier to identify household pantry items.

With Show and Tell, blind and low vision customers can hold up an item to the Echo Show camera and ask, “Alexa, what am I holding,” and Alexa helps identify the item through advanced computer vision and machine learning technologies for object recognition.

“It’s a tremendous help and a huge time saver because the Echo Show just sits on my counter, and I don’t have to go and find another tool or person to help me identify something. I can do it on my own by just asking Alexa,” Grijalva said.

Amazon believes in starting from the customer and working backward, a philosophy also applied to invent and using our technology resources for good. That means paying attention to what all of our customers are telling us.

“The whole idea for Show and Tell came about from feedback from the blind and low vision customers,” said Sarah Caplener, head of Amazon’s Alexa for Everyone team. “We heard that product identification can be a challenge and something customers wanted Alexa’s help with. Whether a customer is sorting through a bag of groceries or trying to determine what item was left out on the counter, we want to make those moments simpler by helping identify these items and giving customers the information they need in that moment.”

From early research and exploration through product development and testing, Caplener’s team collaborated with the Vista Center for the Blind and Visually Impaired in Santa Cruz, California, where Grijalva is the assistive technology manager. For Show and Tell, Grijalva enlisted other blind and low vision customers for user studies, providing feedback to the Alexa for Everyone team. “By working together, we create an amazing product for everyone,” said Grijalva.

Vista Center community member Brett Fowler lost his vision at age 10. The stay-at-home dad loves to cook, but he has trouble identifying things like spices. Fowler says the Show and Tell feature on the Echo Show is a game-changer. “All of these devices that are acting as your eyes, it’s revolutionary. For me, the less stress I have to put on somebody else is less stress on me. And it makes me feel good.”

The Alexa for Everyone team also collaborated closely with blind Amazonians, including principal accessibility researcher, Josh Miele. His decades of experience designing and evaluating the effectiveness of products for people with visual disabilities provided valuable insight into product and research design. “I get to help create products like Show and Tell, make our Fire tablets and TVs accessible and delightful, and constantly help to imagine new ways for our products and services to improve the lives of our customers, including those with disabilities,” said Miele. Globally, an estimated 1.3 billion people live with some form of vision impairment, according to the World Health Organization, and 15% of the world's population experience some form of disability.

---