David Tejeda helps deliver food and drinks to tables at a small restaurant in Dallas. And another in Sonoma County, California. Sometimes he lends a hand at a restaurant in Los Angeles too.
Tejeda does all this from his home in Belmont, California, by tracking the movements and vital signs of robots that roam around each establishment, bringing dishes from kitchen to table, and carrying back dirty dishes.
Sometimes he needs to help a lost robot reorient itself. “Sometimes it’s human error, someone moving the robot or something,” Tejeda says. “If I look through the camera and I say, ‘Oh, I see a wall that has a painting or certain landmarks,’ then I can localize it to face that landmark.”
Tejeda is part of a small but growing shadow workforce. Robots are taking on more kinds of blue-collar work, from driving forklifts and carrying freshly picked grapes to stocking shelves and waiting tables. Behind many of these robot systems are humans who help the machines perform difficult tasks or take over when they get confused. These people work from bedrooms, couches, and kitchen tables, a remote labor force that reaches into the physical world.
The need for humans to help the robots highlights the limits of artificial intelligence, and it suggests that people may still serve as a crucial cog in future automation.
“The more automation you inject into a scenario, the more, at least for now, you need those humans there to handle all the exceptions and just watch and supervise,” says Matt Beane, an assistant professor at the University of California, Santa Barbara, who studies robotic automation of manual work.
Human operators have been a feature of some commercial robotic systems for more than a decade. A few years ago, as new robots emerged in different workplaces, it seemed as if human helpers might be just a stopgap, helping until AI improves enough for robots to do things for themselves.
Now, Beane says, it seems that this workforce will continue to grow. “They're cleaning up after the robot,” he says. “They are the human glue that allows that system to function at 99.96 percent reliability, according to reports given to some VP of automation somewhere.”
Beane says the smartest companies will use input from human operators to improve the AI algorithms that control their robots most of the time. Each time a person labels an object—a chair for example—in an image, it can help train the machine-learning algorithm that the robot uses to navigate.
But training AI this way is challenging, and there seems to be no shortage of new tasks for people to do. Beane says he has yet to come across a company that has successfully replaced human operators by having them train an AI algorithm.
Tejeda works for a company called Bear Robotics. The company’s cofounder and chief operating officer, Juan Higueros, says it is ramping up production of robots to meet growing demand, and also plans to hire dozens more robot operators.
“I do think this is going to become a very important aspect of how robotics companies that are in both structured and unstructured environments are going to have to operate,” Higueros says. He says the company has found an ample supply of workers in pockets of the US, including Texas and Utah.
Remote robot work is a growing category in job listings, especially at robotics startups looking to put systems in new settings that present challenges for AI. Perceiving, interpreting, and operating in an ever-changing environment remains an unsolved problem in AI and robotics, despite some impressive progress in recent years.
Another sign that remote robot wrangling is taking off is interest from some startups focused on the problem. Jeff Linnell, who previously worked on robotics at Google, left to found Formant in 2017, when he realized that more remote operation would be needed. “There are all sorts of applications where a robot can do 95 percent of the mission and a person can pick up that slack,” he says. “That's our thesis.”
Formant’s software combines tools to manage fleets of robots with others to set up teams of remote robot operators. “The only way you get to an economy of scale over the next decade, in my opinion, is to have a human behind it, managing a fleet,” he says.
Another sign that this kind of work could become more common is interest from network providers. At Newlab, an “innovation center” in New York, Verizon is helping startups test robots connected using 5G wireless technology. 5G’s faster speed and lower latency than older networks allows robot operators to move more computation and sensing to the cloud, paving the way to remotely operate robots more reliably. In February, Verizon acquired incubed IT, a company that makes software for managing and controlling robots. Elise Neel, VP of new business, says Verizon hopes to develop products that will help industrial customers manage a fleet of robots more effectively.
Some of the technology used to remotely operate robots comes from the world of self-driving cars. Autonomous vehicles often need some form of human assistance in the form of safety drivers inside cars, but also remote operators that help when a vehicle gets confused and grinds to a halt.
Elliot Katz is cofounder and chief business officer for Phantom Auto, which makes software for remotely operating autonomous cars as well as forklifts for customers including the logistics giant Geodis and delivery robots for Postmates.
Katz says companies have traditionally been drawn to remote operation because of safety and to tap into new workers and scale up capacity when needed. But he says the pandemic, which has hastened a revolution in remote work, has accelerated things significantly. “All of our customers are focused on ‘How do we drastically reduce the number of workers in our facilities,” he says.
People who have lost jobs may be attracted to operating a robot remotely, he says. “You have so many people, mainly due to the pandemic, that are home, unemployed,” he says. “We can train people who are willing to do this job.”