Netflix (NFLX +3.23%) enjoys a reputation for big hits. But it also has a reputation for being cagey about exactly how big those hits are. We still don't know exactly how many people watched Bandersnatch, for instance, and we know that Netflix's numbers for Bird Box were born of a very specific methodology.
So it came as a bit of a surprise when Netflix announced that it would be releasing top 10 lists, ranking content watched on its platform. But don't go wild just yet: This is a test program, and for now it's only happening in the United Kingdom.
Image source: Getty Images.
How Netflix's new ranking program works
Netflix's new program looks like a fun way to track content popularity, but it doesn't necessarily look like it uses the most rigorous methodology. Here's how it works, per Netflix's official U.K. and Ireland Twitter account: "A quick FYI: to compile these lists we look at the most-watched individual season of a show, film or special (regardless of when it launched). 'Watched' means members finished at least 70% of one episode."
In other words, users don't even have to watch one full episode for a given season of a show to claim a viewer. And there doesn't appear to be anything to differentiate between a binge-watching mega-hit and a show that a lot of viewers give up on three-quarters of the way through the pilot.
Dividing things by seasons hurts long-running programs, too. For example, outside analysts have pegged the U.S. version of The Office as the most-watched show on Netflix here in the United States (Netflix hasn't released any numbers of its own).
But under Netflix's just-announced methodology, all of those The Office super-fans would be binge-watching nine separate seasons, not one show. Plus, a user who watched The Office's first season in full over and over again would count as much as a user who watched 75% of one episode of, for instance, Our Planet, which ranked tops among documentaries in Netflix's most-watched list for the U.K. in April.
Netflix is using fuzzy math in its self-reporting. It did say in a tweet that this is "a test as we want to make sure this information is useful for you. So we may tweak it in the weeks ahead. eg. we’re not including kids right now. And if people don’t find it useful we'll stop."
Still, it's fair to wonder if there's a reason Netflix chose to count this way.
Netflix's uncomfortable position with licensed content
Netflix's decision to reveal some user habits now is particularly interesting given its looming crisis with licensed content. With a big influx of new streaming services coming from major media companies, Netflix is poised to lose a lot of shows: The Office (U.S.), for instance, is an NBC show, and NBC's parent company, Comcast, is planning a streaming service of its own for 2020. That planned service will almost certainly be The Office's new home as soon as Comcast can manage it.
Netflix's original content, once a cost-saving luxury, now looks like a survival necessity. When NBC, Disney, and others make off with their own titles, original content will be virtually all that Netflix has left. That's not good for Netflix, where users still watch a lot of licensed content. So why would Netflix want to share data about user habits now?
One possibility: Netflix's methodology makes things look rosier for Netflix than they really are. The U.K. Netflix catalog is a lot smaller than the U.S. catalog, meaning that Netflix originals are a great percent of the whole -- all the better for dominating a top 10 list.
Note that Netflix originals take up seven of the 10 slots in the U.K.'s most-watched series list.
Top ten most-watched series in the UK in April:
— Netflix UK & Ireland (@NetflixUK) May 9, 2019
1. Black Summer
2. After Life
3. Riverdale
4. Chilling Adventures of Sabrina
5. Star Trek: Discovery
6. Lunatics
7. Bodyguard
8. Santa Clarita Diet
9. Bonding
10. Brooklyn Nine-Nine
More fuzzy numbers from Netflix
It may be exciting to hear about new measurements from Netflix, but the reality is that the company is still being stingy with us. U.K.-only numbers using this method tell us little about what's actually happening on Netflix. To the extent that these numbers mean anything, they hint that more common-sense metrics would reveal something uncomfortable about Netflix's position in its larger stateside market as it prepares for a potentially calamitous departure of licensed content.
