AI and Accessibility: How Be My Eyes Partners with OpenAI to Assist the Blind
Making Vision Possible Through Human-AI Collaboration
Growing up, we always looked forward to the Christmas season homecoming. The vibrant colours, sounds, events, activities, and smells made those moments unforgettable. I recall during one of the Christmases, when my mum and I went shopping to stuff up the house in preparation for the rest of my relatives’ homecoming. While navigating through the crowded market with my mother, I noticed a man sitting by the side, his eyes closed, holding a sign that read, “Blind, please help.” My mother, ever compassionate, approached him and offered assistance. She gently guided his hand to the items he needed, explaining the surroundings and the people around us.
That moment left a lasting impression on me. It wasn’t just about offering help, it was about understanding the world from someone else’s perspective and finding ways to bridge the gap. This experience stayed with me, shaping my understanding of empathy and the importance of accessibility.
Fast forward to today, and I see that same spirit of empathy and innovation in action. Be My Eyes, a Danish startup, has partnered with OpenAI to create a groundbreaking solution that combines human assistance with artificial intelligence to help blind and low-vision individuals navigate the world more independently.
The Be My Eyes and OpenAI Collaboration
Be My Eyes connects blind and low-vision users with sighted volunteers through live video calls, assisting with tasks like reading labels, navigating new environments, or identifying objects. In 2023, the company introduced “Be My AI,” an AI-powered virtual assistant built on OpenAI’s GPT-4 vision model. This integration allows users to receive immediate, context-aware assistance without waiting for a volunteer.
The AI can describe images, interpret text, and even provide real-time guidance, making everyday tasks more accessible. For instance, a user might point their phone at a product label, and the AI will read it aloud, including details like expiration dates or nutritional information.
Real-World Impact: Case Studies
Microsoft’s Disability Answer Desk: Microsoft integrated Be My AI into its Disability Answer Desk, enabling blind and low-vision customers to receive support via live video and AI assistance. The results were remarkable: This integration not only improved efficiency but also enhanced the accessibility of Microsoft’s support services for all users. The AI drastically reduced the time needed to resolve issues. Be My AI solves customer issues in one-third the time on average compared to live agents (4 minutes on average for AI vs. 12 minutes for live agents). The AI handles a higher percentage of customer issues without human intervention. Be My AI achieved a 90%+ successful resolution rate. Only 10% of consumers using AI interactions chose to escalate to a human call center agent.
Knorr Thailand’s Accessible QR Codes: In Thailand, Knorr partnered with Be My Eyes to implement Accessible QR codes on their product packaging. Blind and low-vision consumers can scan these codes using the Be My Eyes app, which provides audio descriptions of product details in Thai. This initiative empowers users to make informed choices about nutrition and cooking independently. The collaboration launched the first-ever use of Zappar’s enhanced Accessible QR (AQR) codes in Thailand on Knorr’s instant porridge flavors. Consumers scan the codes using the Be My Eyes app to receive audio product details (ingredients, nutritional values, and cooking instructions) in Thai via Be My AI.
UX Design Principles for Accessibility
The success of Be My Eyes and its AI integration offers valuable lessons for UX designers:
Simplicity and Clarity: The app’s interface is straightforward, with minimal steps to request assistance. This simplicity ensures that users can navigate the app without confusion, even in stressful situations.
Immediate Feedback: Whether it’s a volunteer response or AI-generated description, users receive prompt feedback, reducing uncertainty and building trust in the system.
Context-Aware Assistance: The AI’s ability to understand and describe the user’s environment adds a layer of context that enhances the quality of assistance, making it more relevant and helpful.
Ethical Considerations: Be My Eyes prioritizes user privacy and consent, ensuring that all interactions are secure and respectful. This commitment to ethics builds confidence among users and volunteers alike.
The Future of AI and Accessibility
The collaboration between Be My Eyes and OpenAI is just the beginning. As AI technology continues to evolve, the potential for creating more inclusive and accessible experiences grows. Future developments might include:
Enhanced Object Recognition: AI systems could identify a wider range of objects and provide more detailed descriptions.
Multilingual Support: Expanding language capabilities to serve a more diverse user base.
Predictive Assistance: AI anticipating user needs based on context and history, offering proactive help.
These advancements could further empower blind and low-vision individuals, enabling them to navigate the world with greater independence and confidence.
Conclusion
The partnership between Be My Eyes and OpenAI exemplifies how technology, when designed with empathy and inclusivity in mind, can transform lives. By combining human insight with artificial intelligence, they have created a solution that not only assists but also empowers individuals with visual impairments.
As we continue to innovate, let us remember the lessons learned from the markets of Lagos and the spirit of empathy that drives meaningful change. By prioritizing accessibility and understanding, we can build a world that is truly inclusive for all.
“Writing here is my way of connecting, teaching, and learning with you. If you would like to help sustain this space, please consider supporting my work. It truly means so much to me.”
References
#BlessingNuggets #AccessibilityUX #AIforGood #InclusiveDesign #BeMyEyes #OpenAI