Quora Questions are part of a partnership between Newsweek and Quora, through which we'll be posting relevant and interesting answers from Quora contributors throughout the week. Read more about the partnership here.
Answer from Alon Amit, Former Facebook Product Manager.
Forget Facebook. How would you decide how to rank stories in someone's news feed?
Suppose I give you a random Facebook user - someone you've never met before, never heard of, and know nothing about. Let's call her Alice. Alice has 300 friends (just below average) and likes 15 pages. At this moment, she's opening Facebook on her browser, and you find that there are about 100 stories available for her - all the stuff her friends posted over the past 12 hours since she last checked in.
Further, suppose you have access to the entire history of Facebook's usage logs. All users, the stories they've seen, the ones they liked, commented on, shared, clicked, came back to, all that. You have Alice's own history, as well as everyone else's.
Oh, and you have about 50 milliseconds to make up your mind. Alice is not going to appreciate waiting for her news feed to load while you scratch your head.
What would you do?
Well, the candidate stories are not made equal, are they. Some are just text, some have photos, some have links to YouTube videos. Some are short, some long. Some have already been liked, commented on and shared around. Two of the stories are by Ewen, a bloke whose stories Alice tends to like. Another is by someone she has 50 common friends with. Still others are by her mother-in-law.
So, presumably, you would try to learn how all those different factors influence the likelihood of people to engage with the stories. Well, engagement may or may not be your goal - there's a separate question of what the goal needs to be, how you measure your success.
There's no shortage of data. Throughout history, how much more likely are stories with photos to get liked than stories without? How about stories by "close friends"? Rather than the overall stats, how do things look like for Alice herself? For girls of Alice's age?
Whatever you learn from studying all those massive amounts of data, you'd want to distill that into a very simple set of rules. Something like "For each story, take the number of likes Alice has ever made with the writer plus 1, add 7 if the thing has a video, multiply by 23 if it's a video of a cat, subtract 600 if the word 'panties' is mentioned, and then rank all the stories by that score." Or maybe "First check if it's a photo. If yes, follow these instructions, but if not, follow those other instructions". Remember, you only have 50 milliseconds to decide, there's no time for deep pondering. Keep it simple.
The field of study people use to do such things used to be called "statistics", but nowadays it's more fashionable to call it " ". The goal of machine learning is to do what we just described: look at historical data, produce simple rules that can be used to maximize some value that you need to define, and continuously improve the rules and their parameters to get the most value for everyone.
Many modern systems for making user-facing decisions - what search results to show you, how to rank your feed stories or your news stories or your ads - many such systems follow the paradigm we just described. The technical details of producing the distilled formula from the data can get quite hairy, but the core idea is essentially just what you would have done if you had this data and a lot of time on your hands.
Almost any machine learning paradigm can be used here. It's very, very rare that a specific category of algorithms like "decision tress" is the only one suitable for a general task such as ranking stories. You can do engagement prediction using stochastic regression, boosting, gradient descent or whatnot and use the predicted scores for ranking. You can use SVMs. You can use random forests. I've seen each and every one of those employed at one point or other at Google or Facebook.
The choice of algorithm matters less than the architecture and infrastructure for moving the data around and getting the predictions out on time, the feature selection, feature engineering and quality controls, and a thousand other details. The ML paradigm matters but not as much as you may imagine.
Read all the response to the question "How does Facebook decide the order of events on the news feed?"