A little after 6 pm ET on Wednesday, the system started blinking red for iOS developer Clay Jones. Like many devs, Jones uses a Google product called Crashlytics to keep tabs on when his app stops working. Out of nowhere, it registered tens of thousands of crashes. It also pointed to the cause: a chunk of code that Jones’ app incorporates to let people log in with their Facebook accounts.
By 6:30 pm, Jones had filed a bug report about the flaw in Facebook’s software development kit on GitHub, the code repository. He provided succinct answers to a standardized form:
What do you want to achieve? We are using FBSDK in our app as an authentication option.
What do you expect to happen? I would like FBSDK to not crash.
He wasn’t alone. According to widespread reports and the web monitoring service Down Detector, prominent iOS apps like TikTok, Spotify, Pinterest, Venmo, and more experienced issues on Wednesday. Many users found that they crashed whenever they tried to open the apps, whether or not they used Facebook to log in. “Please move slower and break fewer things,” wrote one GitHub commenter. “Thank you.”
“Yesterday, a new release of Facebook included a change that triggered crashes in some apps using the Facebook iOS SDK for some users. We identified the issue quickly and resolved it,” Facebook said in a statement.
That change was quite small, given its outsized impact. “It was something like a server value—which was supposed to provide a dictionary of things—was changed to providing a simple YES/NO instead, without warning,” says iOS developer Steven Troughton-Smith. “A change that simple can break an app that isn't prepared for it.”
The use of SDKs, not just from Facebook but in general, is commonplace in part because of the convenience. In the same way that you might assemble a car using parts from other manufacturers with particular expertise, developers build apps with outside code, especially from ubiquitous online companies like Facebook and Google. An SDK means that much less work you have to do yourself.
“Pretty much all these apps—Pinterest, Spotify, a lot of the big ones—use the Facebook SDK for the login button,” says Jones. “You’ll see ‘Login With Facebook.’ Everyone has it, super common, great for sign-up rates because it’s just a one-click thing.”
And lots of apps that don’t use Login With Facebook still use the SDK, which is why the issue Wednesday was so widespread. “It is extremely common for apps to connect to Facebook, regardless of whether they use a Facebook-related feature, mainly for ad attribution,” says iOS security researcher Will Strafach, whose Guardian Firewall app automatically blocks online trackers. “It’s something people are not made aware of, and what’s more frustrating is that attempting to block it will break things a user may actually want, such as Login With Facebook.”
But for developers, using an SDK also means ceding control when things go wrong, both in identifying the problem and resolving it. Even though Crashlytics identified the problematic code right away, those details were of little help to Jones and others. “It’s Facebook’s code,” says Jones. “It’s not like it’s something we wrote or something we know a whole lot about. You can try to parse out what’s going on by how the code is written, but it’s not our code.”
Facebook’s not the only company to experience this specific category of woe. In late April, the Google Maps SDK had an issue that similarly caused apps that use it to crash on opening. Wednesday’s incident is worth flagging, though, not only because of its widespread impact but because it serves as a reminder of just how far Facebook’s reach extends. Not only that, but several developers commenting in Jones’ GitHub bug report noted that the crashes seemed to indicate that the Facebook SDK was sending information back to the company’s servers every time the app opened, activity that they—and almost certainly their users—found surprising at best.
"By default, the SDK is configured to send events for app install, app launch, in-app purchases, and SDK crashes," says Facebook spokesperson Tom Parnell. "Developers can disable these events, add other events from a standard library we include, or create their own custom events."
That’s another potential downside of using any SDK: You often don’t have a good read on exactly what it’s up to.
“It’s actually uncommon to open an App Store app and not have it make a connection to an analytics service to report the install along with other telemetry, as well as reporting app use on a continued basis,” says Strafach. “What’s interesting here is that people were forced to notice this now because of the crash—downside of arbitrarily including globs of code in your app without reviewing it for bugs.”
The good news is that Facebook did fix the issue with haste, as far as these things go. Jones says it took about two hours for things to return to normal. (The fix itself came through sooner, but it took time to propagate.) It’s maybe useful, though, not to move on too quickly from the outage as a case study in how your favorite apps are actually built, how interconnected every aspect of the internet is, and how so many roads lead back to Facebook—whether you have an account or not.