Speak'n'hear assistive mobile app

Speak'n'Hear

Speak’n’Hear is a mobile app designed to facilitate communication for individuals with hearing and speech differences. It serves as a tool for transcribing audio, synthesizing speech, and recognizing speech in text format. The app aims to improve social interaction and reduce communication challenges for its users.

  • UI/UX Design
  • Flutter
  • ASP.NET
  • OpenAI Whisper
  • Espnet, txtai

Challenge

From a tech standpoint, Speak’n’Hear faces several challenges. First, the app must seamlessly integrate advanced speech recognition and text-to-speech technologies, ensuring they work harmoniously without compromising performance. Achieving real-time, accurate translations across various languages and dialects is crucial, particularly given the diverse user base and unique communication needs.

Developing a user interface that is intuitive and caters to the varied abilities and preferences of hard-of-hearing and non-speaking users presents a complex design challenge.

Balancing sophistication in technology with ease of use and accessibility remains a key focus for the development team.

  • This project is all about creating a bridge between technology and people. We’re focused on making things simple, clear, and easy to use for everyone. Our aim is to design in a way that breaks down barriers and resonates with people from all walks of life.

    Mykyta  Hrach,
    UX/UI Design
  • Speak'n'hear assistive mobile app
    HealthTech
    Industry
  • Speak'n'hear assistive mobile app
    10 people
    Team size
  • Speak'n'hear assistive mobile app
    2023-ongoing
    Project duration
  • Speak'n'hear assistive mobile app
    R&D
    Project type
  • Speak'n'hear assistive mobile app
    Sweden
    Country

Solution

Speak’n’Hear offers a comprehensive solution tailored to its diverse user base. For users with hearing differences, it includes speech recognition, transforming spoken language into text, making conversations more accessible. Non-speaking users have access to a text-to-speech feature, enabling them to communicate verbally through typed text. People with communication differences can benefit from both functionalities, ensuring they have full support for their communication needs.

  • Accessibility

    Importantly, the app’s design adheres to global accessibility standards, making it not just user-centric but also universally accessible. This commitment to inclusivity ensures a satisfying and barrier-free experience for all users.

    Technologically, the app stands out with its ability to save conversation histories at the user’s request, enhancing user experience and providing a reference for past communications​​. It offers personalized settings, allowing users to indicate their specific needs.

  • Performance

    In terms of performance, the app is designed to handle multiple users simultaneously with a response time for speech recognition and text-to-speech features of less than two seconds, ensuring efficient and timely communication​​. This technical proficiency is coupled with robust safety and security measures, including encrypted user data and secure server communication

  • Problem Statement

    Individuals with hearing and speech differences face barriers in accessing health services, including communication challenges during consultations and limited health information access.

    Not all healthcare professionals are proficient in sign language, the primary communication method for many hard-of-hearing and non-speaking individuals.

    This can lead to challenges in accessing services and information, highlighting the need for more inclusive health services that accommodate diverse communication needs to ensure equitable healthcare access.

  • Speak'n'Hear Benefits

    • Enhances feelings of safety and security among individuals with hearing and speech difficulties
    • Highlights the collaboration between healthcare facilities and the broader community
    • Enables individuals with hearing and speech differences to seek medical assistance through standard methods, promoting equal access to healthcare services
  • Project Scope

    The goal is to develop a comprehensive telehealth app designed for individuals with hearing and speech challenges, providing them with accessible healthcare services. This app aims to bridge communication gaps, ensuring that hard-of-hearing and non-speaking people can request and receive medical assistance effortlessly.

Target Audience

  • Who will be using this product?

  • Who is your offering for?

The solution is dedicated to the deaf person. They are the recipient of the solution.

Audience Needs

  • What does your audience need?

  • What are their pain points?

Pain Points

    • Lack immediate sign language interpreter access, medical information for deaf patients.
    • Emergency systems are not designed for the deaf
    • Leading to potential misunderstandings during medical consultations.
    • Health information often isn’t available in accessible formats
    • Face difficulties in managing healthcare appointments and procedures
  • Needs

    • Require accessible communication, including sign language interpretation, text-based options, and real-time interpreting tools.
    • Quickly contact emergency services using methods like SMS, email, and GPS tracking.
    • Health information needs to be in understandable formats
    • Appointment management

Key Functionality

  • How does the app address user needs and challenges?

  • SMS or email-based emergency service contact

  • Real-time text/audio interpreting tool

  • GPS location tracking

  • Simplified appointment scheduling system

  • Vibrating alert notifications for diverse purposes

  • Digital transcription of laboratory and diagnostic Results

Team Insights

  • Our team sees this project as more than just a technical endeavor; it’s a personal mission, especially as it was inspired by the cooperation with a team member with hearing and speech challenges. This collaboration has been a driving force, allowing us to leverage our skills and creativity to develop a truly impactful telehealth platform.

    Nikita Tkachov,
    CDO Beetroot

Bring your idea to life

Check some of the main reasons tech leaders and non-tech startup founders choose us for their custom software development solutions.