9.3 C
New York
Thursday, March 28, 2024

Watch a Robot Dog Learn How to Deftly Fend Off a Human

logoThe AI Database →

Application

Hardware

End User

Research

Sector

Research

Source Data

Sensors

Technology

Machine learning

Machine vision

Robotics

Study hard enough, kids, and maybe one day you’ll grow up to be a professional robot fighter. A few years ago, Boston Dynamics set the standard for the field by having people wielding hockey sticks try to keep Spot the quadrupedal robot from opening a door. Previously, in 2015, the far-out federal research agency Darpa hosted a challenge in which it forced clumsy humanoid robots to embarrass themselves on an obstacle course way outside the machines’ league. (I once asked you, dear readers, to stop laughing at them, but have since changed my mind.) And now, behold: The makers of the Jueying robot dog have taught it a fascinating way to fend off a human antagonizer who kicks it over or pushes it with a stick.

A team of researchers from China’s Zhejiang University—where the Jueying’s hardware was also developed—and the University of Edinburgh didn’t teach the Jueying how to recover after an assault, so much as they let the robot figure it out. It’s a dramatic departure from how a hardware developer like Boston Dynamics goes about teaching a robot how to move, using decades of human experience to hard code, line by line, the way a robot is supposed to react to stimuli like, um, a person’s foot.

But there’s got to be a better way. Imagine, if you will, a soccer team. Midfielders, strikers, and a goalkeeper all do generally soccer-esque things like running and kicking, but each position has its own specialized skills that make it unique. The goalkeeper, for instance, is the only person on the field who can grab the ball with their hands without getting yelled at.

In traditional methods of training robots, you’d have to meticulously code all of those specialized behaviors. For instance, how should the actuators—motors that move a robot’s limbs—coordinate to make the machine run like a midfielder? “The reality is that if you want to send a robot into the wild to do a wide range of different tasks and missions, you need different skills, right?” says University of Edinburgh roboticist Zhibin Li, corresponding author on a recent paper in the journal Science Robotics describing the system.

Li and his colleagues started by training the software that would guide a virtual version of the robot dog. They developed a learning architecture with eight algorithmic "experts" that would help the dog produce complex behaviors. For each of these, a deep neural network was used to train the computer model of the robot to achieve a particular skill, like trotting or righting itself if it fell on its back. If the virtual robot tried something that got it closer to the goal, it got a digital reward. If it did something non-ideal, it got a digital demerit. This is known as reinforcement learning. After many of such guided attempts of trial and error, the simulated robot would become an expert in a skill.

Compare this to the traditional line-by-line way of coding a robot to do something as seemingly simple as climbing stairs—this actuator turns this much, this other actuator turns this much. “The AI approach is very different in the sense that it captures experience, which the robot has tried hundreds of thousands of times, or even millions of times,” says Li. “So in the simulated environment, I can create all possible scenarios. I can create different environments or different configurations. For example, the robot can start in a different pose, such as lying down on the ground, standing, falling over, and so on.”

Once the eight algorithmic experts were trained up, they needed to learn to work together as a team. So the researchers combined them into an overarching network to act as a kind of coach or team captain. This allows the Jueying’s artificial brain to tap into the knowledge of each expert—how to run, or turn, or right itself. “The coach or the captain will tell who is doing what, or who should do work together, at which time,” says Li. “So all experts can collaborate together as a whole team, and this drastically improves the capability of skills.” For example, when the robot is falling down and needs to recover, the system can detect that movement and trigger the expert that handles balancing.

If you take a look at the GIF above, at top left you’ll see bars that correspond to the eight different experts. As the virtual robot moves through the simulated environment, playing fetch with the floating green ball, the network turns the influence of each expert’s specialty up or down, depending on what skill is needed at a given moment.

“This is a useful alternative to training the robot for each task it will encounter,” says University of Southern California biomedical engineer Ali Marjaninejad, who researches quadrupedal robots but wasn’t involved in this work. A challenge, Marjaninejad adds, will be reducing the amount of computation the robot requires to train in simulation, in order to make the process more efficient for practical applications.

The Jueying team’s researchers could then port what the digital robot learned in simulation into an actual robot in the real world. In the video above, the “coach” in the robot dog’s brain is coordinating with the AI experts to help the machine maintain its footing as it walks over stones. And when a professional robot antagonizer pushes the Jueying over, the robot pulls itself back on its feet—no small accomplishment for a four-legged robot in any environment, much less one that requires clambering over rocks. (Imagine how easy it would be for you to twist your human ankle here.)

Not content to let the robot off easy, the human can grab a handle on the machine’s head, forcing it into a face-plant. Each time, the plucky robot climbs back onto its feet—and not because the researchers coded this reaction to this particular variety of assault, but because the Jueying is now consulting with its AI experts in the art of locomotion. When the researchers made the robot walk on slippery surfaces or on grass, instead of stones, it also adapted to that. Basically, it’s become robust to the unexpected.

The general idea of this research is to get robots to learn to locomote in much the way human toddlers do. The traditional way of coding a robot to move is to load a machine with assumptions about how the real world works—say, how a foot might grip hardwood floors and carpeting differently—and to give it point-by-point instructions. But imagine telling a toddler: To climb these stairs, you’ve got to move your arms and legs just so. They don’t follow directions; they learn to move by trial and (sometimes painful) error. They have to learn by experience which surfaces are slippery and which are rough, and how they should adapt their movements accordingly.

Similarly, says Li, a machine can’t easily adapt to its environment just by following a script, because real-world forces and surfaces are unpredictable and enormously complex. “All these assumptions completely break down when you enter the wild, because you don't have complete information about that,” says Li. There’s just no way for roboticists to fully characterize the chaos of the real world for machines to understand. So the solution is to let the Jueying learn as people do—thanks to good old trial and error. “A grand picture or grand vision,” Li adds, “is that we will have more intelligent machines, which are able to combine flexible and adaptive skills on the fly, to handle a variety of different tasks that they have never seen before.”

Aspiring professional robot fighters, take note.

Related Articles

Latest Articles