5 C
New York
Friday, March 22, 2024

Meet the Man Who's Building Sonos' Audio Internet

Nick Millington had just started to feel like a successful software engineer. It was early 2003, and at age 26 he already had a fruitful career at Microsoft. He feared his one-bedroom apartment in Redmond, Washington—filled with a mishmash of bargain-bin furniture and DIY shelving made of cinderblocks and wood planks—wasn't representative of his upward mobility. He wasn't a starving student anymore, he was an adult. So Millington did what we all do once we're able: He bought a larger, more elegant apartment and filled it with brand-new stuff.

"I was starting to think more about the design of my home," Millington says. "You know, upgrading to the fancier Ikea furniture that looked nice."

One nagging problem was how to rig up his music system. Digital music was exploding on the internet, and Millington had spent many hours methodically trying to collect every Billboard Top 40 hit from 1945 onward, amassing a wide range of other, more eclectic music files along the way. But listening to his treasure trove of tunes felt like living in the dorms again. All of his MP3s were stored on the boxy PC he kept in his living room. "At the time, my music setup was a Gateway 2000 tower PC and then I had a laptop," he says. "I would do terminal server into the PC and play MP3 files there. That wasn't cutting it for me."

Around that same time, a Santa Barbara, California, startup called Sonos was hiring engineers. Sonos courted Millington with a simple pitch: It wanted to make digital music players that would look respectable on his "fancier" Ikea shelves, wirelessly access his MP3s without requiring him to touch a PC, and play all of his music in every room of his house. Millington wanted in.

Now, 15 years later, Nick Millington is the chief product officer at Sonos: the guy responsible for all of the company's speakers and software. When I meet him at Sonos' East Coast headquarters in Boston, he's dressed like an archetypical engineer, in a plaid, short-sleeved shirt. Framing his round face is brown hair that's kept in check but seems to be waiting for the chance to poof and curl. He looks both completely put together and a dash disheveled all at once.

Millington is named as an inventor on 69 Sonos patents, with 33 others pending—a significant portion of the company's entire portfolio, which numbers around 500 patents. He also developed the core networking framework that keeps the playback of multiple Sonos speakers throughout a home in perfect sync. His work was central to the company's early adoption of apps and its expansion from amps to speakers. Yet, even though his efforts have proven critical to the company's success, Millington tends to credit others a lot more than himself. He seems more concerned with solving problems than bragging about his solutions, which may be why he's never sought a greater spotlight or given an in-depth interview before talking to WIRED.

Since 2005, Sonos has sold 19 million audio devices—each of them a representation of the work done by Millington and his team—into 7 million homes. The same speakers still top many best-of guides. By most measures, it's a story of success.

>

But today, Sonos is sailing rougher seas. Cofounder and CEO John MacFarlane resigned in early 2017 and was replaced by COO Patrick Spence. The company has also had more than one round of layoffs as it has dealt with an evolving competitive landscape. The rise of the voice-assistant smart speaker, in particular, took the company by surprise. Just this year, shortly before going public, Sonos released a new soundbar that interacts with voice services from Apple, Amazon, and soon, Google—the same tech giants who have charged forcefully into the multiroom wireless speaker market Sonos built.

It will be up to Millington and his product team to chart a course through the choppy, frenemy-filled waters ahead. To do it, he may help guide the company into new places altogether. Until now, all Sonos products have been shackled to rooms inside your home. In the future, Millington hints, Sonos may create its first speakers and audio products designed to leave the house.

Millington is used to leaving houses. As the son of a British Diplomatic Service officer, he spent his entire childhood globetrotting, but it was his time as a kid in Japan in the 1980s—an era when the country was the worldwide epicenter for electronic innovation—that set him on a path toward tech.

"There was an area of Tokyo called Akihabara, which is where there were all of these tech companies and these stores where you could go buy endless arrays of parts, and incredibly cheap floppy disks and all sorts of equipment," Millington says. "I used to go almost every weekend with a few geeky friends of mine and check out various developments and things like that."

He learned about networking early on as well, subscribing to the first dialup internet service provider in Japan, called TWICS. He ran a Tokyo PC users group on a bulletin board system that predated the web. Eventually, he landed at Duke University in the US and then went to Microsoft when he graduated in 1998, where he worked on SharePoint, the software maker's online collaboration platform.

When Sonos was founded, in 2002, there was no streaming music. No Spotify. No Pandora. AOL dialup was the most popular way to access the internet, and many families didn't even have Wi-Fi yet. iTunes was popularizing the concept of legal music downloads, and file-sharing services like Kazaa were growing in popularity as the music industry reeled in the wake of Napster.

>

The four founders of Sonos, led by MacFarlane, recognized that digital music would increasingly become a bigger part of consumer's lives. Their big idea was an ambitious plan to make it possible for anyone to set up a multiroom home speaker network for digital music. At the time, multiroom systems could be purchased, but they were inaccessible.

"The technology was very cumbersome," Millington says. "It was difficult to set up. It was typically the domain of high-end installers charging very high prices for stuff that didn't always work terribly well. Sometimes you had to actually rebuild your home to put in the necessary wiring and speakers."

Their plan was to democratize the whole stack. Instead of requiring dedicated wiring or a team of professional installers, Sonos would create Wi-Fi amplifiers ("ZonePlayers") that you could tether to the speakers you already owned. Set a Sonos amp in any room of your house, and digital music could be summoned from your computer using a dedicated wireless remote control. You'd be able to move your speaker setup to new rooms, alter which speakers were grouped together, and take the whole arrangement with you if you bought a new home.

By early 2003, the founders had hired Andy Schulert to head up product development. He quickly called on Millington, an old Microsoft colleague, to help solve the most daunting problem the company was facing: how to develop networking tech that would flawlessly sync multiple amplifiers together, transport music between them, and keep them connected and updated through the internet—all over Wi-Fi, which was far less mature back then. Despite having no audio experience "except for 10 years of piano lessons," Millington moved to Santa Barbara and taught himself what he needed to know about audio synchronization in a matter of weeks.

"We had a saying in the early days of Sonos that if there's one thing that absolutely has to work in version 1 of the software it's the ability to upgrade to version 1.1," Millington says. "But if there are two things that have to work, then the audio transport is definitely the core part of it. I have always liked to gravitate toward the core problem where 'If we don't solve this, we have no product' and make sure that it's handled in the best possible way."

Syncing amps (and their speakers) wasn't easy. One of the big challenges with multiple speakers is dealing with the accuracy of the human ear, which can quickly detect audio that's out of sync.

"The way that you perceive stereo [sound] is with differences in time when the signal arrives at your right ear and your left ear," Millington explains. "If that is moving around or it's off, it will seem like the sound is coming from a different location, and it can be quite quite disconcerting. You need to synchronize it to less than a millisecond of accuracy to have it be a really pleasurable experience."

To get multiple speakers to sync that closely, Millington developed a method of time-stamping all the music traveling between speakers, thereby holding each speaker accountable. Timestamping made it virtually impossible for Sonos ZonePlayers to get out of sync.

>

The team made another important choice around this time. Instead of designating a permanent master ZonePlayer that centrally ran the entire network, the team created a distributed network in which every Player acted on its own and intelligently communicated with the others—no easy task. For example, if a user had five ZonePlayers hooked up, Millington couldn't let all five of them fetch music from the internet. It would suck up too much bandwidth and potentially crush a home network. So he developed a "delegation" process that allowed every ZonePlayer to dynamically assign duties to one another. If one ZonePlayer was removed from the network, another one could pick up the slack and take over its duties—even fetching the music for all the Players, if necessary.

Unfortunately, none of this worked over Wi-Fi yet. John MacFarlane was adamant that the whole system work wirelessly, so Millington turned to mesh networking. This method offers a way to wirelessly connect devices in an ad hoc fashion, so you don't need to rely on a central traffic point like a router to keep the network humming. Millington taught himself mesh networking in about six weeks.

By early 2004, Sonos' wireless mesh networking system was working. Owners would be able to run up to 32 Sonos players in their home, grouping and ungrouping them at will to bundle rooms together, play the same music across an entire floor of their house, or use each player separately.

But code that works in the lab still has to pass real-world tests. Millington and the crew began traveling to homes with different Wi-Fi setups near Sonos' East Coast headquarters in Boston and its West-Coast headquarters in Santa Barbara. They had to figure out what microwaves and cordless phones might do to a Sonos player under the same roof. In the early days of wireless networking, many products didn't use Wi-Fi as frugally as they do today, and some were major bandwidth hogs, causing a lot of headaches for the team.

It was tempting to take the easy road and blame someone's Wi-Fi for everything that went wrong. But for Sonos to succeed as a product, it had to operate in less-than-ideal wireless environments, and several months of troubleshooting ensued. The intense amount of testing the team went through in that prelaunch phase has been immortalized in code; all Sonos products are packed with onboard Wi-Fi diagnostic tools that can send reports to customer service reps when speakers start having problems.

Space Oddity

The first Sonos ZonePlayer (ZP100) hit store shelves in early 2005 as part of a $1,199 bundle that included two amplifiers and a physical wireless remote controller.

That wireless controller had a screen and direction pad so you could play music without a PC. Some engineers affectionately referred to it as a "Russian iPod" because of its chunky, jog-wheel-bedecked design. But it was useful, displaying album art and song titles on a small screen, and giving users the ability to group and ungroup speakers. The company also made a strong effort to simplify setup with three-step instructions.

At launch, the ZP100 was a niche proposition. Wireless multiroom systems were entirely new. Sonos players, though cheaper than a professionally installed wired system, were still fairly expensive, and the fact that they required you to purchase your own speakers was tough to communicate to a mass audience. Hardcore audiophiles likely understood it, but many of them were already in the market for (or owned) a professional setup.

Still, for a startup like Sonos, it was a promising debut. The team believed the product worked really well and was reliable, and initial sales were at least decent. As word of mouth began to spread and new ZonePlayer amps arrived, Sonos gained more attention.

All-In on iPhone

With the concept of networked audio proven and Sonos' amp business up and running, Millington was promoted to director of advanced development and architecture in 2006. In his new role, he assembled a small team of half a dozen engineers to create bold, innovative product ideas—a skunkworks team of sorts. While the rest of the company maintained and improved those wireless amps, he began working on new concepts.

One of his first projects turned out to be a critical milestone for Sonos.

With Millington at the helm, Sonos launched its first iPhone app in late 2008—the same year the App Store launched. The company debated whether to charge for the app (nobody knew how much apps should really cost back then) but decided to make it free.

Millington credits the other founders, particularly MacFarlane, for the brisk adoption of iOS. MacFarlane's "a guy who lives three or four years in the future, and he takes for granted things that don't actually exist yet, is kind of how I would describe his mindset," Millington tells me with a smile. "He really gave us the push in that area."

The app eliminated the need for a host PC or that "Russian iPod" wireless controller. In the years that followed, Millington and the team continued to flesh out the app and began adding direct access to music streaming services as listeners stopped hoarding MP3s on their home computer and began turning to services like Pandora, Rhapsody, and Spotify.

Sonos also made a critical decision around this time that continues to define what the company is about: It chose to remain an open platform. The company decided against making its own music service and instead began working to support every audio service on the market in a completely neutral way.

"Sonos is a level playing field for these services to compete for users' attentions and subscriptions. We've never taken any money from a music service or promoted any one of them [over the others]," Millington says. "I think the services appreciate that."

The Sonos app eventually grew to support around 100 services globally, more than any similar platform.

The First Sonos Speaker

The next milestone project for Millington's team was a full-fledged speaker for Sonos. In early 2008, he hired an audio engineer named Chris Kallai, a self-described "audio nut" who had spent time at Harman and Velodyne.

In its early years, Sonos focused on amplifiers instead of speakers because it seemed too difficult for an unknown brand to launch a speaker as its first product. There were a lot of established companies in the space. Executives also believed that users and reviewers tended to judge speakers differently than amps. Amps are almost always evaluated objectively. With speakers, however, each listener tends to favor their preferred sound signatures.

"There was just a lot of snake oil and folklore around which ones sound good and which ones don't, and things like that," Millington says. "You know the nice thing about an amplifier is you can measure whether it sounds good or not. Either it reproduces the input or it doesn't, whereas with a speaker it's much more subjective."

To solve the problem, the company decided to avoid creating a "Sonos sound" of its own. Kallai, Millington, and others decided Sonos speakers would try to replicate what recording engineers heard in the studio as they recorded albums. They assembled a group of recording artists and engineers to help. Several notable names in music, including Rick Rubin, joined the group. (Giles Martin, son of Beatles producer George Martin and overseer of many recent Beatles remasterings, currently heads it up.)

With Kallai's help, Sonos shipped the $399 Play:5 speaker in 2009. Millington and others described this as a turning point for the company because of how much it simplified the Sonos proposition. Combined with the new iPhone app, it was a speaker that worked out of the box and sounded fantastic. It could be used alone or be networked with as many as 31 pieces of Sonos hardware—other Play:5 speakers or older ZonePlayers. It also got more capable over time, thanks to firmware updates that downloaded and installed from the app—refreshing all speakers at once.

"In some ways you can think of [the Play:5] as the first smart speaker in the sense that it's internet-connected, runs software, connects to music services, and can make music itself rather than being hooked up to an external amplifier and speakers," Millington says.

The Play:5 earned relatively high marks from reviewers, who liked its sound and features. It helps that the Play:5 stood out among what seemed like a sea of home speakers with iPod docks or then-subpar Bluetooth radios.

Let the Good Times Roll

The success of products like the Play:5 and iPhone app led to a promotion for Millington. He was put in charge of the entire product department at Sonos in early 2010.

Throughout the next few years, more and more music listeners began to rely on their phones to stream songs, and for anyone who liked using an app, Sonos became an attractive idea. The company enjoyed a wave of growth and recognition as a leader in multiroom audio during these years, adding speakers like the petite $200 Play:1 (the top-selling Sonos speaker) and Playbar soundbar to its lineup. Eventually, Sonos came full circle and revamped the Play:5, giving it touch controls and a modern exterior.

In early 2014, Sonos redesigned its smartphone app, adding a universal music search that let you hunt through all your music services at once. Later that year, an update eliminated the need to physically plug one of your speakers/amps into your home router to create a Sonos network. All of a sudden, every Sonos player could just connect via Wi-Fi, further simplifying setup.

But at the end of 2014, things began to change. Amazon released a small voice-powered speaker called the Echo. It didn't sound very good, and Sonos didn't recognize it as a threat. But the Echo quietly kicked off an entirely new wave of smart speakers powered by voice control. Just as things were beginning to get comfortable, the ground beneath Millington and his team's feet began to shift.

After I finish my multihour chat with Nick Millington in the "Fenway" meeting room at Sonos' Boston office, hardware lab manager Jim Weineck whisks me away to give me a full tour of the company's labs, where products, new and old, are put through their paces. The facility is intense. Many testing chambers look like massive bank vaults from the outside, and it's not uncommon to see huge foam cones protruding from the walls and ceilings.

In one chamber, speakers are stress-tested by having special pink noise tones piped through them for several months at a time. Pink noise sounds a lot like white noise, but it contains frequencies that are better for testing audio systems. (I was told that, in the Santa Barbara offices, Sonos had 64 Play:1 speakers playing audio at maximum volume for 12 months straight using a custom-designed tone. This tone, called "life test noise," simulates a bunch of songs across all genres at once, and pumping it through the speakers for a year can simulate 10 years of playback.)

In another room the size of a walk-in freezer, a huge circular array of probes studies how well the Wi-Fi antennas in Sonos products pick up and emit a signal. A monitor shows me a 3-D Wi-Fi cloud for the Sonos Playbase, which appears to have a tough time picking up signals directly below it. Other rooms test for things like the long-term effects of extreme temperatures, static electricity and how it impacts touch controls, and unintentional radiation.

On the tour, one engineer tells me Sonos speakers are packed with more antennas and connectivity tech than they actually need. The team even tries to squeeze in features that may not be used yet, knowing they might be activated in the future via a software update. Whenever Sonos releases new updates, it takes pains to make sure older hardware still works reliably. Millington and other employees say there are still ZP100s out there, serving up music in 2018 just like they did in 2005. Quite a few of them, actually. Sonos claims that 93 percent of all the players it has sold are still in use—a figure that stands out in a tech world where internet-connected products increasingly seem to die on a whim.

Weineck leads me into Sonos' anechoic acoustic chamber, my favorite part of the tour. It's a two-story vault with a door so heavy it needs to be operated electronically. Inside, the room is completely silent. The walls and ceiling are covered with bundles of foot-long, gray, triangular prisms that absorb all sound and cancel any reverberations. The floor is a trampoline-like mesh material topped with a metal wire grid. If you peer through the mesh, you can see that you're suspended about 10 feet off the gray, foamy ground. In the center of the room is a pedestal where a speaker sits with an arced pole of microphones in front of it. These mics capture and map the sound that comes out of a speaker.

>

Standing in a silent chamber is oddly unnerving. Weineck tells me that with the lights out, people start to go crazy after a few minutes in the vault because they lose all sense of space and direction. All they can hear is their own heartbeat. As soon as he tells me this, I too swear I can hear my own organs pulsating.

Other areas of the facility hold Faraday cages that block outside signals to create a pure environment where Sonos can test the Wi-Fi access points of its speakers (a necessity in a building filled with hundreds of internet-connected devices). A 3-D-printing room lets designers quickly mock up new product ideas.

Weineck describes a couple of rooms as "teleconferencing on steroids" thanks to their electronic whiteboards, highly sensitive directional microphones, and a surgical camera mounted to the ceiling. The camera is precise enough to zoom in on the threading of a screw. The Santa Barbara offices have rooms identical to these here in Boston, enabling worldwide nitpicking as teams on each coast work together to perfect the look or sound of a new speaker.

Notes of humanity peek through the sterile jungle of lab equipment on concrete floors. I notice jokes posted on the walls and oddly placed toys, like a shark sitting atop test equipment, hinting that there is at least a little time for play in the audio labs. In a woodworking lab, some employees pulled a prank on an engineer's tendency to meticulously label his supplies by labeling absolutely everything in the room when he was on vacation, including his sink and chair. They laughed, telling me it was to help him "get back up to speed quickly."

There are also Obi-Wan Kenobi photos saying "this is not the room you're looking for" cheekily plastered on some of the more mysterious doors in the labs during my visit. Like a good stormtrooper, I move along, but I also wonder what the audio Jedi are up to in those secret labs.

For a company born by looking ahead, Sonos was late to recognize the importance of voice controls in speakers. Though the Amazon Echo launched way back in 2014, Sonos just began selling its first voice-enabled products in the past year, with the Sonos One and the new Sonos Beam. The shift has forced Millington and his product team to rethink what a Sonos speaker should do all over again.

In some ways, Sonos is still ahead. The multiroom capabilities in the Google Assistant app are already starting to get pretty good, but Amazon's Alexa is plagued by a lot of headaches when it comes to multiroom and third-party speaker support. Neither Google's nor Amazon's product supports as many streaming services as Sonos, and the only streaming service supported by Apple's Siri-enabled HomePod speaker is Apple Music.

>

"Smart speakers can do a ton of things, [but] the killer application is music," Millington says. "I think second only to, you know, 'Alexa set a timer for 30 seconds,' music is the predominant application of smart speakers. Once you've put multiple speakers into an environment, you have to deal with the multiroom issues. They've gotta be synchronized, you've got to be able to group them, and you've got to do it in a way that's decentralized, that doesn't involve any kind of server."

Much like they did by supporting all music-streaming services equally, Millington and his crew have chosen to remain agnostic when it comes to voice assistants. While almost every other voice-activated speaker focuses solely on supporting a single service, Sonos plans to support the three biggest—Alexa, Siri, and Google Assistant—by the end of the year. The team even has a name for the body of content we all access on voice assistants: the sonic internet.

"In many ways, speakers are the browsers for that sonic internet," Millington says. "They're the thing that lets you go and connect to all the content that's out there. Now, imagine a browser that could only go to Amazon.com or could only go to Google.com or could only go to Apple.com. That's a rather limited experience."

Supporting more than one assistant in a single speaker adds compatibility problems.

"We're working with Google Assistant alongside Alexa and nobody's ever really thought about how to get those things to coexist before," Millington says. "What does it even mean to ask Alexa to 'order an Uber' and then ask Google, 'When's my Uber going to be here?' We're starting to think through a bunch of those types of issues. That for me is where the innovation often happens."

Walk This Way

Until now, Sonos products have always been made to live in rooms of your home. The company's mantra has long been "fill every home with music." But that mission may be too limiting for Sonos today, Millington says.

The next step for Millington might be taking all those voice assistants on the sonic internet, and the entire speaker platform Sonos has developed, outside the home for the first time.

"One of the key transitions that we talk about is from home to everywhere," Millington says, choosing his words carefully. "The home isn't the only place where you listen to music. There are many places where you listen to music. So I would say, without giving the blow-by-blow of everything in our road map, that's one of the key themes that we're thinking about."

When I ask again, he clarifies his words but won't commit to any future products or whether they are ideas or actually in development.

"Over time, everywhere that you might want to enjoy music—in different rooms of your house as well as outside the home—we want to have a product that serves that scenario really well, and also any content that's relevant to you," he says. "We want to make it as easy as possible for you to summon that up wherever you are. All of our work is going into those areas. And again, when I say content it's not just music. It's sonic culture umbrella in general: podcasts, entertainment, TV soundtracks, things like that."

Does that mean Sonos is planning to make a set of headphones in the future? Maybe a battery-powered portable Sonos speaker? Something else entirely? We'll have to wait and see, but thinking about how Sonos might work outside the home is exciting.

Since there is no reliable Wi-Fi outside, Sonos products would need to tether to your phone via Bluetooth, or maybe they might directly connect to LTE service—though that, admittedly, feels like a long shot. What we know is that these are now all things Sonos is pondering as well.

For Millington, the best part about the future of Sonos is how it will improve every product the company has already shipped.

"Personally, I'm incredibly proud of the fact that you can use the latest iPhone and our app to control a Sonos player that you bought back in 2005, and listen to Spotify, when none of those technologies even existed at the time."

Whether Millington and Sonos can maintain that mindset and keep all of their new products alive for more than a decade—all while pursuing the kind of growth that's demanded of a publicly traded company—is a question no voice assistant can answer just yet.

Related Articles

Latest Articles