by On his YouTube channel, Anand Munje interviews entrepreneurs, academics and artists.
He communicates with them even though it is difficult to speak because of his cerebral palsy, but it is possible thanks to the software developed by his brothers Arun and Amit Munje — Aihearu.
It says “I hear you,” and takes Anand’s words and turns them into text on the screen during his show Andes world.
The Aihearu brothers integrated into Zoom video conferencing, and three dots appear on the screen as the software processes Anand’s voice and then displays it as text.
“Talking to people in my own voice without Amit’s help is a new thing for me,” he said in the episode where he interviewed his brothers. Amit often acted as Anand’s interpreter.
For his channel, Anand prepares and practices his questions. This allows him to fine-tune the vocabulary used by the software to interpret his voice.
CBC News met Arun at home in the Ottawa suburb of Kanata for a video interview with Anand in India.
In a departure from how CBC usually conducts interviews, Anand received some questions a day in advance so he could prepare answers the same way he does for the show.
“The best part is, it can recognize my voice,” he said in one of his prepared answers. “I hope that millions of people with speech impairments can use the app. I want others to be able to speak independently.”
We also tried the “open mode” to cover some questions that were not given to him beforehand.
“I’m very nervous,” he said through the software in open mode. msgstr “This is the first time I have used open mode.”
Arun, who previously worked at a teleconferencing startup and founded Aihearu as a side project, explained Anand’s “open-mode” efforts to match words against a huge word dataset.
“This is a completely open vocabulary. That’s why there may be some or more errors,” said Arun. “The context is unknown [of] what are we talking about.”
Anand Munje speaks using the Aihearu interface integrated with Zoom. The software displays his words and displays an ellipsis as it processes his speech. (Zoom)
Open mode made some mistakes and Arun had to help interpret, a sign of work to come. For example, “it’s 10 pm” became “it’s a poem.”
Anand admits that managing the software vocabulary is sometimes a challenge.
But while there may be challenges for long, more spontaneous conversations, Anand is able to use it for regular activities like managing smart home devices.
Arun said artificial intelligence allows them to build and refine the model that recognizes Anand’s speech more quickly, rather than “feeding” his voice patterns into the program.
He said the motivation was to help his brother express himself, but he hopes Aihearu will be able to help people with various speech impairments or even thick accents.
“What we have done is that we will use Anand so that we get a personal experience of all the problems and how we can overcome them,” said Arun.
‘Leading a fuller life’
Claire Davies says people with speech impairments need more options to be able to express themselves independently and participate in society.
“The further there is in the future, the more we can enable others to interact with people with speech impairments,” said Davies, who runs the Assistive Technology Building and Design Laboratory at Queen’s University .
“That allows them to get a job, it allows them to interact in social environments, it allows them to work with their co-workers and it allows them to live a fuller life.”
Davies said it is especially valuable that the technology comes from the family’s life experiences and their understanding of how Anand expresses himself.
“They better understand how to interpret that information and that allows the software to learn more effectively about what is being said,” she said.
She said other users could also benefit from their work.
Claire Davies, associate professor at Queen’s University, says people with speech impairments need more options to express themselves. (Submitted/Claire Davies)
In open mode, Anand was able to answer a question about how he feels about his brothers’ hard work.
“I am very grateful and very proud of my brother[s],” Anand said through the app.
Arun was even more shocked that the software produced that result.
“I push myself when I see him get caught [phrases] in that open mode because he hasn’t entered anything.”
Arun said they are testing Aihearu with a handful of other speech impaired people through charity, but each user needs to be set up manually.
“The next step is to make it easily self-usable by anyone, so they can … download and use it.”
