When it comes to evaluating if a movie is a hit or a miss, at least commercially, there’s a very easy metric for that: box office numbers. Movies need to make money; to make money, they need to sell tickets at theaters. Ticket prices can vary depending on where you live, the time of day, regular or plush reclining seats, and so on, yet they’re a pretty standard—and reliable—way of figuring out whether a film release is actually successful. Television is a little trickier; Nielsen ratings, the standard for TV, are statistics based on a sample size of households that provide Nielsen Media Research with their viewing habits. The figures aren’t as precise, but they work—at least for the advertisers looking to buy airtime based on the viewership of popular shows.
>
How does it work, then, for a streaming service? Well, as Netflix revealed this week, the answer is however the companies want to! Following the launch of the original series The Witcher, as well as the premiere of the Michael Bay-directed, Ryan Reynolds-starring action flick 6 Underground, the streamer issued a fourth-quarter earnings report this week boasting to shareholders that the Henry Cavill-starring fantasy series was “our biggest season one TV series ever” (emphasis Netflix’s). According to the streaming firm, that meant 76 million households watched the show in its first four weeks. 6 Underground, meanwhile, was viewed by 83 million in the same amount of time.
But pull out your salt mill and prepare your asterisks for those huge numbers. Netflix—which only selectively releases viewing numbers for its most popular shows and films—has an ever-evolving definition of what a “view” really means. Previously, when the company announced its 2018 original film Bird Box had brought in 80 million viewers in the four weeks after its premiere, the caveat was that the number reflected users who watched at least 70 percent of the film. For this new crop of viewership data—which notably doesn't include bragging rights for The Irishman, Marriage Story, or The Two Popes, the three Netflix films up for Academy Awards next month—the metric is very different. As The Hollywood Reporter notes, “Now, it will count anyone who ‘chose to watch and did watch at least 2 minutes’ of a title as a viewer. … Per Netflix, it is ‘long enough to indicate the choice was intentional’ but more accurately shows popularity than the 70 percent threshold, which may have negatively impacted longer projects.”
Two minutes? Two minutes!!! That is a bizarrely low threshold for entry into, say, a 10-part fantasy drama featuring a character that could best be described as “Legolas with an Equinox membership” with episodes that run between 47 and 67 minutes. Can one even get past the opening credit sequence? The trailer for The Witcher runs two minutes and 20 seconds. Meanwhile, two minutes of The Irishman, which runs three and a half hours, amounts to 0.95 percent of the total film. You’d barely get through the opening tracking shot at the assisted living facility where an aging Frank Sheeran (Robert De Niro) is primed to reflect back on his long life of crime. I like to imagine me, an occasional movie critic, turning off a film after two minutes and writing a review. I mean, according to Netflix’s data, I’ve seen it. Two minutes is enough!
>
(OK, this is where I include a possibly deranged parenthetical aside to say, once again: Two minutes??? Is that possibly a reasonable amount of time to consume or engage with anything? Imagine enjoying anything that lasts two minutes [winky-face-emoji]. It is absurd! It's a reminder that Netflix is, at heart, a technology company, and can just say whatever it wants and most people will reply “Mmm yes, of course, you’re right, let’s disrupt our collective concept of metrics and statistics and data valuation forever.” Yes, the internet is crushing our attention spans, but 6 Underground isn’t a TikTok—or even a YouTube video, for that matter. Perhaps expecting every viewer to finish what they're watching is asking a lot, but maybe count their view after 10 minutes, or 15? Imagine putting in two minutes of work and saying, “Well, the intention was there, so I don’t see why it doesn’t count?” If this paragraph even makes it into this esteemed publication, it might just be because my editor has already shrugged, released a giant sigh, and said “Eh, I’ve been staring at this for two minutes, I’m sure the rest is fine, whatever. Hit Publish!”) [Editor's note: I read it, even this paragraph. Gave it two minutes of rock-solid edits. Circle, send message.]
Look, understandably there’s no one perfect, absolutely correct metric. That’s something I gathered from the first four weeks of my college statistics class before I dropped out and changed my major to English, because I like to read books and hate math. The point is to find the most accurate one as possible. Nielsen ratings are based on a sample that’s supposed to reflect American TV viewing habits. A ticket sale is a ticket sale. Your visit to this very page, dear reader, earned this post a pageview. (Thank you!) But not everyone who visits a website on the internet reads the whole thing. A ticket sale doesn’t guarantee that a moviegoer didn’t walk out early in disgust. And I’m not entirely convinced that Nielsen boxes aren't delivered specifically to households full of Big Bang Theory fans.
Netflix, however, is playing fast and loose with its metrics—and its data. We know the company knows what we watch; it's watching us as we’re watching its shows. Does that inform the content the streamer produces? Absolutely. But these figures still have so many holes in them. For one, the company doesn’t count multiple users who log in to the same account (meaning the data might suggest that the parents of millennials who borrow their folks’ passwords are actually watching Cheer and The Circle). The data is also misleading because it completely inflates the popularity of, well, everything. No broadcast network could claim to its advertisers that two minutes of intentioned viewing should count toward a show’s popularity, especially since the ad breaks come much further into an episode of analog television.
Sure, Netflix is not courting advertisers, but the service is hoping for more subscribers and more investors. I can’t help but think back to the federal lawsuit against Facebook wherein the social network was accused, in 2016, of inflating video views on the site. Facebook claimed it was a calculation error, but by the time it was revealed to advertisers many media executives had already rushed to pivot their digital strategies to video—a grand move that cost many content creators, many of them my fellow journalist peers, their jobs. (Facebook claimed the suit was “without merit,” but settled last fall.)
While the streaming service’s liberal data might not affect the average user, surely there are some people who might want to raise a dubious eyebrow—and hold it there long enough for Netflix to see.
Updated 1-24-20, 2:45 pm EST: This story was updated to correct the name of the star of 6 Underground.
Updated 1-27-20, 8:40 pm EST: This story was updated to clarify the details of the Facebook lawsuit.