- “Doctor Who”
Netflix’s data wizard says we’re close to a future where algorithms know almost everything we want.
“A very realistic vision is we should get to the point where you just turn on your Netflix app and automatically a video starts to play that you’re very happy with,” says Vice President of Innovation Carlos Gomez-Uribe. “If you’re not, you may have to flip once or twice and end up with something that you’re very, very happy with. Only in, say, 10-to-20 percent of sessions [will you] escape into … browse mode.”
Netflix has been creeping toward that goal for more than a decade.
There was the Netflix Prize, launched in October 2006, which aimed at predicting how much users would like videos. As the company shifted into streaming video, it turned to engagement data to predict what users really wanted to watch. Last year saw the introduction of a major new algorithm that chooses between other algorithms to find the best recommendation. And many more improvements big and small.
“It’s just a matter of time,” says Gomez-Uribe.
You can judge for yourself how close Netflix is to this goal anytime you open the app. The video in the first slot of the top row tends to be the one Netflix thinks you’re most likely to watch right now.
Gomez-Uribe talked through the evolution of Netflix recommendations in a recent call with Business Insider.
Beyond the stars
Back when Netflix was just a DVD-delivery company, it relied on star ratings filled out by members to analyze and predict what people would like.
The problem was that people don’t always want to watch five-star videos: a member might give “The Godfather” five stars and “Friends” three stars but still be more interested in watching “Friends.”
“One of the challenges with stars is some people interpret it as the following: if you were a movie critic, how would you rate this movie?” says Gomez-Uribe. “Are they telling you how much they like the video or are they telling you how much they think a movie critic would like the video? That makes it not very useful, frankly.”
Thankfully, Netflix has a lot more data these days.
“[S]ince we started with internet television, since we know exactly what you’re watching and how much of it you’re watching on what device and so on, the engagement data … turns out to be a much more important and useful predictor of what you like and what you will watch than the stars,” says Gomez-Uribe.
Netflix’s engagement data includes every play, pause, search, and click from 83 million current members and past ones too. Machine learning does wonders to spot patterns in this data, using similarities between one user and groups of users to make predictions.
Netflix’s most important metric now is what you’re most likely to watch and stick with for a while.
“The more you watch, up to a point, the better the outcome,” says Gomez-Uribe. “If we show you something that leads you to watch six minutes and then abandon it, that’s a failure from our point of view. It’s even worse than you not watching it, because we just made you waste your time on something you didn’t enjoy.”
The algorithms are smart enough to account for changing habits during the week.
“We know most people are complex and diverse and that they are happier with a product that gives them a wider array of stories and types of stories,” says Gomez-Uribe. “Maybe on the weekends, you’re more open to discovering a video, and you’re fine spending more time choosing what to watch … whereas during the week maybe you had a long day at work and you literally have only 45 minutes to relax, you can’t be spending a lot of time choosing what to watch and then discovering something that doesn’t work for you, so instead you may be more likely to just continue watching whatever it is you like to watch, whether it is ‘Family Guy’ or ‘Archer’ or ‘Gilmore Girls.'”
As for the stars, they’re still there for now. You might even find them useful, helping you distinguish between trashy and classy suggestions. Still, the data wizard confirmed that Netflix was thinking about getting rid of them entirely.
Predicting what people want to watch is one thing: getting them to watch it is another. That’s where Netflix relies on a vast degree of personalization and optimization through the app. For instance:
Personalized rows. Netflix settled long ago on rows of similar videos as the best way to present content on the home page. Different rows are powered by different algorithms, combining what Netflix knows about you with other criteria that might be helpful in finding shows.
“We don’t know how to come up with a mathematical formula that optimizes everything at the same time,” says Gomez-Uribe. “What we do is instead develop recipes to satisfy each use case.”
- Continue Watching rows predict how likely you are to return to videos, looking at factors like time elapsed since viewing, point of abandonment, and watching patterns since. Popular on Netflix and Trending Now rows consider your interests and what is popular in general or recently. Top Picks rows are based almost entirely on your interests. Genre rows consider your interests in a certain category. Because You Watched/Liked/Added rows consider similarity to something you’ve interacted with in the past and what you like in general.
Personalized home page. Starting in 2015, Netflix unleashed something called the page generation algorithm to construct the optimal mix an order or rows for each user.
The page-maker grabs the highest-scoring of thousands of possible rows, making sure to include a diversity of options on the page, and puts the highest scoring row on top.
Video scores generally decline as they move down and to the right on the home page.
It’s not hard to imagine in the future that Gomez-Uribe described that that top-scoring video might one day be all you see when you open the app.
Optimized evidence. Netflix uses data to figure out the visuals and information to provide to get you to choose a video.
Some of this is personalized. For instance, the system uses what it knows about you to decide whether to show to a movie won on Oscar or say that it is similar to something else. Netflix also famously made ten cuts of the first “House of Cards” trailer to appeal to different audiences.
Some of it is more generally optimized. For instance, the image that shows up for a video may change if A/B testing shows that another image is better in your region. Or a different image will show up depending on what row it shows up in, looking to emphasize a particular aspect of a video.
Billion dollar algorithms
Netflix has said the combined effects of personalization and recommendation are worth more than a billion dollars every year.
What’s so valuable about them?
For one thing, they let Netflix get more value out of every item in its library. The company tracks this with something called the effective catalog size, which shows how viewing is spread across the catalog. Here’s a chart showing the effective catalog size when making personalized recommendations and when basing recommendations solely on popularity.
This means Netflix doesn’t have to license as much content to keep people happy. Not surprisingly, the company also uses data insights to determine what content is really worth adding to the catalog.
“You can imagine that we could have a million low-quality videos that almost no one wants to watch and then say, hey, we have the largest catalog in the world and advertise that and so on, and obviously that would be pretty egregious. What matters is how many videos are people actually watching,” Gomez-Uribe says.
Despite reports that Netflix’s total amount of licensed content has declined, Gomez-Uribe suggests that the effective catalog size is increasing.
“I’m pleased with how that number has evolved, but I can’t give any comments on that,” he said.
Netflix’s recommender system also leads to better recommendations. One metric that tracks this is the take-rate, which measures the fraction of recommendations offered resulting in a play. Here’s a chart showing the take-rate starting with the best recommendations with personalization turned on and off (with the highest score normalized to equal 1).
A high take-rate is key to keeping users happy. Netflix says the typical subscribers will give up after 60-to-90 seconds if they haven’t found a good video. If that happens often, then subscribers will quit the service.
How happy are users these days?
Even more important for Gomez-Uribe is hours watched per subscription per day. This figure grew 13% in 2015 from 1.6 to 1.8.
Speaking for one user, I can say that I would choose Netflix over any other video service.
I watch about two hours a day, and the more I learn about the algorithms, the more I trust them. Does Netflix really think I’ll like “Scrubs”? Fine, I’ll give it a try.