9 C
New York
Monday, March 27, 2023

This Hearing Aid Can Translate For You—and Track Steps, Too

logoThe AI Database →


Personal services

End User



Consumer services

Health care

Source Data


It’s too loud for me to hear inside the Cupertino coffee bar, but Achin Bhowmik says it doesn’t bother him. He’s got a superpower, he says. If I look closely—very closely—I can see the tiny plastic tubes reaching from his ear canals to small devices hidden behind his ears. The hearing aids are running machine-learning algorithms that continuously monitor his “acoustic environment” to help him hear what he wants to hear. In the coffee shop, the devices decide this is a “speech in noise” situation, and automatically dampen the sound of background chatter and espresso machines, and focus four directional mics (two in each device) to amplify my voice instead.

But that’s not the cool part. While other high-end hearing aids have included similar technology in recent years, Starkey Hearing Technologies’ new Livio AI hearing aids also count your steps and track how much time you spend talking to people rather than in isolation. They can detect when the wearer has fallen, and with an impending software update will be able to notify a loved one or 911. They can even listen to another language and whisper a near real-time translation in your ear, Star Trek style. While some or all of these features have been available in consumer devices known as “hearables,” they’ve never before been packed into a hearing aid—a government-approved medical device that has to be small, comfortable, and include batteries that last for days rather than hours.

If Starkey delivers on all these promises, the Livio AI, which Bhowmik unveiled last week at Starkey’s headquarters near Minneapolis, could bust the musty old hearing-aid industry out of its niche of selling to … well, musty old people. Today, only 50 million of the estimated 466 million people with some hearing loss use hearing aids. By adding other capabilities, Starkey is hoping to finally make it OK for all those untreated people to buy a product they haven’t wanted to admit they need. It’s a page out of the Apple playbook. “When Steve Jobs launched the iPhone, he totally disrupted a perfectly good cell-phone market by making it into a multipurpose device” that combined a phone, an internet communicator and iPod into one product, says Bhowmik, Starkey’s chief technology officer. “What Apple did to the smartphone, we’re going to do to the hearing aid.”

That’s big talk, but Starkey has as good a chance as anyone to pull it off, and the most reason to try. The 5,000-person company is one of five firms—the other four are based in Europe—that sell more than 90 percent of the world’s hearing aids. It may be the most innovative of the lot, having introduced everything from a 90-day trial to all-but-invisible in-the-ear-canal models. If any company is going to figure out how to grow the highly profitable, $7 billion-a-year hearing aid business, Starkey isn’t a bad bet.

New Rivals Emerge

Trouble is, Starkey may be competing with some much tougher rivals before long. As personal assistants such as Siri and Alexa change how consumers access information, Silicon Valley’s most powerful companies are working on ear-based devices of their own. Apple is interested in creating “hearable”-style products with sensors to track various health metrics, say people familiar with Apple’s thinking. Amazon is determined to free Alexa from the smart speaker and get her into an in-ear device, where she can fill your shopping orders wherever you are, according to people familiar with its plans. Google has various projects to do the same for search and other services.

The giants’ interest coincides with passage last year of the Over-The-Counter Hearing Aid Act, which will create a new class of hearing aids that won’t be regulated as medical devices, for people with mild to medium hearing loss. That will free others to market sexy new ear-based products as hearing aids—for a lot less money and less hassle than a conventional hearing aid. Today, you need to be tested by an audiologist, who charges $3,500 on average for his service and the hearing aid. Backers of the law, which will go into effect by mid-2020, expect over-the-counter models to be available at big-box retailers and pharmacies and pretty much anywhere else that you can buy cheap reading glasses today, for $500 or less.

Starkey executives insist the competitive threats are overblown, because hearables will be aimed at people who now don’t wear hearing aids. They say the Livio AI is simply the latest, and biggest, step of CEO William Austin’s long-term plan to make the hearing aid more than just a hearing aid. Inspired by his work at a tiny hearing-aid repair shop in 1967, Austin gave up on his plan to become a doctor and bought the outfit for $13,000. Thanks to innovative products, customer service and countless celebrity endorsements—Austin has personally fit five US presidents, the Dalai Lama and entertainers from Gene Autry to Ozzie Osbourne—the company has reached nearly $1 billion in annual sales and made him a billionaire.

Throughout, Austin pressed his engineers to take advantage of the ear’s potential for monitoring health metrics such as heart rate, and for real-time language translation. “I’ve been waiting so long for this—to be able to help people live healthier, better lives,” Austin told a raucous crowd at Starkey headquarters during the launch of the Livio AI, via videoconference from Madagascar.

Such talk may sound like typical corporate blather, but Austin is not your average businessman. He was in Madagascar last week to fit locals with free hearing aids. For the past decade, he’s left most of the day-to-day management to others while he and his wife Tani travelled with Starkey Foundation staffers giving the “gift of hearing” from Ramallah to Rwanda (and making plenty of syrupy videos to prove it). The Austins typically spend more than 150 days a year on such missions, working long days in school cafeterias and community centers. “I sensed a great sense of purpose when I met him,” says Satjiv Chahil, a veteran Silicon Valley marketer who was introduced to Austin by Bill Clinton at a Clinton Global Initiative conference in 2013. “It was confirmed when he travelled to parts of India, where I was born, where I’m afraid to go, to give hearing aids to people who can’t afford shoes.”

Austin began reaching out to Silicon Valley years ago. In 2013, Starkey began working with Apple on a “Made for iPhone” program that allowed hearing-aid users to have phone calls and music streamed directly into their hearing aids. Soon after, Chahil was hired as an adviser to help connect Starkey around Silicon Valley, and begin honing a product strategy. Chahil had a hand in turning the PC from a boring text-centric device into a multimedia appliance capable of playing music, movies and multimedia software, by pushing Apple to add CD-ROM drives and Quicktime video software to Macs in the early 1990s. He saw the same opportunity if hearing aids could be redefined to do more than amplify sound for disabled people. “Most technologies get deployed for a single purpose, and people tend to fight against expanding from there,” says Chahil. “When it happens, new worlds suddenly open up.”

A Scandal Within

Beginning in late 2015, however, Starkey was forced to focus its attention on the current world. A few days before Christmas, Austin fired longtime President Jerry Ruzicka, who was escorted off the premises by police. A year later, Ruzicka and five others were charged with a multi-headed plot to defraud the company that included setting up sham distributors, doctoring pay records to award themselves cars and cash, and forging Austin’s signature to grant two of them equity in a Starkey affiliate. Ruzicka and another man were later convicted of charges including mail and wire fraud. In court, however, Ruzicka—who was evidently angry over Austin’s intention to promote his stepson, Brandon Sawalich, to be the company’s next president—took some shots of his own. In a filing, he accused Sawalich of being a serial sexual harasser and questioned Austin’s sanity by ridiculing his enigmatic spiritual beliefs. Austin freely tells listeners—including on the stand during the trial—that God once told him he would die on Nov. 11, 2011, but that a young Mexican boy who spoke to angels later told him he’d have more time to accomplish his good works.

Amid this chaos, two Starkey efforts to redefine the hearing aid flopped. In 2016, the company set out to create a hearable called Wave that would have been marketed more like a souped-up set of wireless earpods, with batteries that could last days on a single charge. Since they wouldn’t fully plug the ear canal, wearers wouldn’t need to remove them when they wanted to talk to someone. Chahil lined up Robert Brunner, Apple’s former head of industrial design and designer of the first Beats headphones, to design the devices. But when engineers couldn’t get the battery life where it needed to be, Starkey cancelled the project.

The company then focused on a partnership with German startup Bragi, an early maker of a non-FDA-approved, consumer-priced hearable. Bragi’s device included hearing-improvement features even for people without diagnosed hearing loss, as well as activity tracking, calorie counting, a heart-rate monitor and later language translation. Starkey invested in Bragi, and the companies introduced a Frankenstein of a device in May 2017 called the “Dash Pro tailored by Starkey,” that was only available through audiologists. The goal was to meld the best features of a hearable and of a hearing aid. In reality, it highlighted their biggest flaws—the unreliability, lack of comfort, and poor battery life of a hearable, with the price and hassle of a hearing aid.

“The demographic that audiologists target is the white, older, and wealthy consumer that needs to address hearing loss,” says KR Liu, a hearing technology executive and disabilities advocate, who has worn hearing aids since she was 3. “The Dash was aimed at a younger audience that had no clue who Starkey was, and who had no interest in paying an audiologist $500 for an ear mold.”

By then, Austin was getting serious about adopting Silicon Valley’s ways. For starters, he wanted to hire a chief technology officer with a proven record for applying cutting-edge technologies in actual products. When Bhowmik got a call in early summer from a recruiter, he quickly declined. He was having a fine time running Intel’s 1,400-person Perceptual Computing Lab, which uses artificial intelligence to give machines human-like understanding of the world around them—so a drone can avoid a flagpole, say, or a robot can respond to an alarm. After hanging up, he recalled that he’d written a paper about Austin in graduate school, which compared mission-oriented founders of public companies (Andy Grove and Bill Gates) and private companies (Amar Bose and Austin). Intrigued, Bhowmik flew to Minneapolis, where Austin told stories of how Starkey had helped the people whose photographs hang in his clinic—not just celebrities, but bright-eyed infants hearing their first sound. Two months later, Austin sealed the deal over drinks in Sawalich’s basement. “You’re using your passion for AI to enhance machines’ understanding of the world,” Austin told him “Why not use it to enhance people’s understanding of the world? There are a lot of people who could use the help.”

Bhowmik quickly got to work on adding new capabilities to Starkey’s next hearing aid, which was already being developed. That process typically takes 18 months or more, mostly to meet the FDA’s testing requirements. But that doesn’t apply to non-hearing capabilities, for which Starkey received a waiver from the FDA. Bhowmik reorganized many of Starkey’s technologists into seven “tiger teams”—including teams for fall detection and language translation—and told them to quickly get him prototypes, or proof of why that wasn’t possible. At Starkey’s annual Vegas Expo in January, he all but promised a product like the Livio AI within a year.

Adding Features Without Hurting Battery Life

In actuality, the deadline was late summer, when audiologists and other customers expect new products from vendors. And there was lots to get done. In terms of hardware, they had to add an inertial sensor—basically, an accelerometer and a gyroscope crammed into one chip—to the device. They had to create the new apps, and figure out which functions should run on the device, and which could be handled elsewhere. Most important, they had to preserve the device’s 45-hour battery life.

The result is a product that capitalizes on the ear’s natural advantages for certain tasks, such as detecting when people fall. According to National Council on Aging, a person over 65 goes to the ER due to a fall every 11 seconds—and half of them will die within the following year. Some people buy fall-detection monitors that notify a loved one or 911. But people often resist using them, and Bhowmik says an ear-based device will be more accurate. While the hardware is in the Livio AI, Starkey hasn’t turned on the service yet because it hasn’t finished field testing.

The sexiest feature of the new model is the ability to translate 27 languages in near real-time. Bhowmik knew building a translation service into the devices would consume too much power and drain the batteries quickly, and that Starkey couldn’t to spend billions, as Google did to build its Google Translate cloud service. So he employed Silicon Valley thinking and tapped Google’s Translate service, allowing a device that runs on a half-watt of power to tap into server farms that consume as many gigawatts as small cities. Bhowmik says the translations typically arrive in less than half a second, though of course the feature only works when you’ve got a internet connection.

While the Livio AI is a technical achievement, it’s far from a guaranteed hit. As with the ill-fated “Dash Pro tailored by Starkey,” the Livio AI will only be available through the audiologists that bring in most of the company’s sales. That means it will cost at least $3,000 and easily as much as $7,000 to get a pair. That might appeal to some people who need a hearing aid. No doubt, that’s a growing market, given all those aging boomers, not to mention the 17 percent of teens with some noise-induced hearing loss from listening to earphones. But until the company works up the nerve to expand beyond audiologists and sell cheaper products through big-box retailers and other channels, Starkey’s multi-function devices will only be the iPhone of hearing aids for a small segment of society.

“There’s a huge opportunity for ear-based technology that nobody has carved out yet, and there’s no reason a small Minnesota company that nobody has heard of can’t do it,” says Gene Munster, a long-time consumer tech analyst with Loup Ventures. “But to appeal to people who aren’t old and rich, it has to be in a $300 product, not a $3,000 product.”

Sawalich, who became Starkey’s president in July 2017, the same month Bhowmik was hired, makes no excuses for this strategy. He points out that 91 percent of hearing-aid owners are satisfied with them, despite the cost and hassle. He argues that anyone with serious hearing loss wants help from an expert, and that doesn’t mean the clerk at the nearest Best Buy. “Look, we can double or triple the size of our company without having to take on Apple any other big tech company,” says Sawalich. “In fact, I’d rather collaborate with those guys.”

That’s wise, but that plan will only work if Starkey keeps innovating with Silicon Valley-style urgency. That’s why Bhowmik is there. He says the company is already working on follow-up products, including ones that add sensors for monitoring heartbeat. He’s added dozens of techies from Silicon Valley, Israel, and elsewhere—and let them stay where they were rather than come to Minnesota—to help fill out more of Austin’s dream. Over time, Starkey hearing aids could include biometric sensors to be on the lookout for signs of epileptic seizures or strokes. It could add apps to help people improve their sleep, or even predict falls—by whispering to the wearer to sit down if it detects an irregular gait. “We’re going to invent some things that are going to surprise even tech’s biggest players,” says Bhowmik.

Otherwise, the surprise may be on Starkey.

Related Articles

Latest Articles