Netflix (NFLX -0.20%) changed a lot of things when it pioneered the subscription video on demand (SVOD) streaming space back in the late 2000s. Its success helped spark the cord-cutting trend and forced stakeholders such as Disney to change their strategies for the new era. Among those affected was Nielsen, the consulting group that has long been evaluating the success of TV broadcasts with its eponymous ratings system.
Neilsen has made some efforts to better capture statistics in the streaming era, but much of what we've learned so far about the successes and failures of various services come from the streaming giants themselves. That often means taking what we hear with a grain of salt. Netflix's self-reported numbers have typically looked pretty good, though a closer evaluation of the methodologies behind some of those numbers makes things a bit less clear. And as questionable as some of Netflix's past proclamations have been, they're nothing compared to what's coming. The company just announced a new metric that will consider a show or movie to have been "viewed" if it is watched for more than two minutes.
Image source: Getty Images.
A history of bad math
Netflix's metrics have never been all that good. While frustrating, that's understandable: It has no particular reason to be advertising any of its own failures. And a rosy view of its originals is a good thing to encourage in a fractured streaming era that has seen quality exclusives become more important than ever. Plus, even before the flight of licensed content was on Netflix's mind, the service was incentivized to hype up original series that would keep viewer's focus on Netflix's content instead of pricey, licensed alternatives.
This is how we ended up with a phenomenon like Bird Box. A sci-fi/horror flick starring Sandra Bullock, Bird Box got decidedly lukewarm reviews from critics. But Netflix used a review embargo, an unhyped release, and -- almost certainly, though the company won't say so -- Netflix's recommendation algorithm to turn the forgettable film into a hit. Nearly 26 million people watched Bird Box in its first week, according to Nielsen. But Netflix (and this is the important part for our purposes) issued its own figure ahead of Nielsen's, declaring with pride that 45 million people had watched the film in its first week. That's a difference of 19 million, which is roughly the population of Chile.
For Netflix, missing the mark by a Chile's worth of viewers was a feature, not a bug. The company counted as a viewer any account that watched at least 70% of the film's run time. Netflix's PR push hyping Bird Box's popularity was presumably intended to get more Netflix viewers to watch Bird Box -- and more non-viewers to sign up.
With shows, Netflix's metrics were even worse. The 70% mark still applied, but shows needed only hit that mark within a single episode. That meant 35 minutes of the first episode of Stranger Things' third season would mark a Netflix user as a viewer of the entire season -- which, in total, is seven hours and 23 minutes long (35 minutes is just 7.9% of the season).
From dumb to dumber
The lesson from all of the above is that Netflix's viewer metrics have always been pretty specious. But Netflix is really taking things to the next level now: The company says it will now count a title as having been viewed if it plays for two minutes or more. Thirty-five minutes was once enough to count as a view for Stranger Things, but now two minutes -- less than half of 1% of the season -- will be enough.
The change in methodology is explicitly designed to pad Netflix's numbers. According to Netflix's own assessment, viewership figures will be 35% higher under the new system. This math suggests that if Bird Box had been lucky enough to be measured within these parameters, Netflix would have claimed roughly 61 million Bird Box viewers for the first week, which amounts to well more than double the Nielsen figure -- an extra 35 million, which falls somewhere between the populations of Saudi Arabia and Canada. And that's all assuming that Netflix isn't downplaying the impact of its new metric, which -- given the context here -- is perhaps a bit credulous of us.
Netflix's explanation for its new metric is that reaching the two-minute mark establishes that a view was "intentional." That's a necessary distinction, because Netflix autoplays titles that viewers linger on while choosing what to watch. If you take a break from browsing Netflix to head to the bathroom, you had best hurry unless you want to return as a card-carrying viewer of Big Mouth.
Investors should ignore Netflix's numbers
Netflix's new metrics are self-evidently useless. If they're better than nothing, they're only barely so. It's a virtual certainty that Netflix turns to more useful figures than this when determining which shows ought to be renewed for new seasons -- the company isn't going to renew something that 90% of viewers quit in disgust five minutes into episode one. But you can bet that Netflix will count those views for PR purposes. Like so many other tech companies, Netflix seems to have little use for transparency.
To review: Netflix's viewer metrics, long questionable at best, are now acts of imagination with no real use to anyone -- except, of course, Netflix's PR team. In all fairness to the company, there's no real incentive for honesty besides (one hopes) a desire to avoid over-promising and over-projecting. Netflix is simply talking itself up, and it's up to independent groups like Nielsen to put real figures together.
The problem is that there are a lot of challenges inherent in trying to rate streaming audiences. Nielsen has some metrics, but nothing has taken off to become as widely cited as the group's legacy TV ratings. While the metrics struggle to catch up, we're left without much insight into how popular this stuff really is -- in fact, thanks to Netflix's changes, we have less today than we did last month.
