Cloud services host vast quantities of valuable information, making them perpetually attractive targets for hackers. Attackers regularly develop new and clever ways to access cloud accounts—or find ones that have been left exposed—and exfiltrate data. Those in charge of protecting cloud accounts have their own methods of shoring up defenses and securing account perimeters. But just in case someone slips by, they also lay the digital equivalent of a booby trap or a trip wire to sound the alarm on any interlopers. They're called honeytokens.
A honeytoken can be any data planted to attract hacker interaction. You might, for instance, send yourself an email marked "Important bank stuff," and put in a link that's really a honeytoken, to let you know if your account gets breached. In the cloud, honeytokens are often authentication credentials that look like the keys to the kingdom, but actually act as canaries in the coal mine. It's a clever ruse, and a vital one given the stakes of cloud security.
But researchers from the network security firm Rhino Security Labs have made the troubling discovery that attackers can identify many honeytokens planted in Amazon Web Services, the largest cloud provider, and silently avoid them while going about their nefarious business. It's like a mouse that learns to grab the cheese without tripping the trap.
"You as the defending company put these keys out there so when I as the attacker grab them you’re alerted and you know that I’ve compromised that area," says Ben Caudill, the founder of Rhino Security Labs. "But the problem we've found allows us to do a universal bypass where we can take those keys and without actually triggering the honeytoken. We can identify that it’s booby-trapped, and avoid those AWS services that it would otherwise trigger."
The problem Caudill and Rhino Security Labs penetration tester Spencer Gietzen discovered has two components. AWS manages honeytokens through an auditing and compliance service called CloudTrail, but there are a handful of niche services that CloudTrail doesn't support. Since CloudTrail doesn't extend its visibility features to them, it also doesn't create logs for activity connected to these services—and for hackers, no logs means no trace.
The second component the researchers discovered is that certain failed AWS queries provide a lot of information in their error messages—what the researchers call a "verbose error message." One thing the errors show is the "Amazon Resource Name," or the name of the credentials you used to send the query. The Amazon Resource Name will also reveal if you're using a honeytoken. As a result, an attacker could intentionally produce errors to check the identity of credentials they encounter, and see whether they are honeytokens. And all of this can happen without CloudTrail having any record of it.
"It’s both the fact that AWS doesn’t have logging on those services and the fact that certain error messages from AWS show you which user you are," Gietzen says. "The API functionality's universal response means that I can use an unsupported CloudTrail service as the attacker to get information back. And there’s no way for you as the defender to know I did it."
Both the security company Thinkst, which offers a honeytoken service called Canary, and the enterprise software developer Atlassian, which oversees the open source honeytoken project SpaceCrab, are making changes to remediate these issues as much as they can. But Caudill and Gietzen note that a full fix to the larger conceptual issue can only come from architectural changes to AWS. Amazon did not return a request for comment by publication.
"This is a fundamental issue within AWS that I think is not known well enough and can be exploited by an attacker," Caudill says. "People rely on these defenses, but there is an inherent risk."