October 21, 2010
Our focus here at Outbrain is trying to help people find interesting content through algorithms and automation. Since all of us who work here are rabid readers ourselves, we intuitively felt that the standard approach you see on existing article pages– showing a list of links related to what you just read, or popular headlines of the day– wasn’t cutting it. After reading a long piece about the BP oil spill, did I really want to read another one? No, not so much. As a reader, what I would appreciate is a link to something within my broad scope of general interests (politics, media, literature, UNC basketball….), but that was new and interesting and definitively *not* what I had already read.
After several years of testing different algorithmic approaches to this issue across thousands of publisher sites, big and small, we now have a lot of data supporting our initial hypothesis: the best way to deeply engage audiences is to find content that is interesting to them, but not necessarily related to the page they are on. While we have tens of algorithms behind the scenes, and new ones being tested every month, they can largely be grouped in the following four major buckets:
Popularity: recommending content that is trending up in popularity on the site
Contextual: recommending content related to the page the person is currently on
Behavioral: recommending content based on audience dynamics. For instance, finding content that people with similar reading habits have been consuming, that is not mainstream popular, and that the person has not read before
Personal: recommending content within broad categories that the person frequents but not necessarily related to the page they are on at present
When evaluating success, we look at a couple of metrics.
1. How frequently do people click on links based on the algorithmic approach (CTR, or “click through rate”)
2. How many more pieces of content does the person consume on the site *after*clicking on the link (what we call PVAC, or “pageviews after the click”)
We boil these two data points together to evaluate the total page views we can produce for every thousand recommendations we serve. Below is a chart of the results across our entire network:
Pretty quickly you’ll see that recommending content that is contextually related falls on the low end of the range, while approaches that help people discover content about a new subject (behavioral) perform much better. If you look within the data further, you’ll find even more interesting nuggets.
For example, articles recommended because they are related to the page itself (contextual) suffer not so much on CTR– where they are only slightly below average– but primarily on how many page views they yield post click. On average, someone who clicks a related link will read 1.9 more pieces of content within that session whereas someone who clicks on what we call a “discovery” link through a behavioral algorithmic approach will read 2.6 more pieces of content during that session.
That’s a pretty huge jump. What accounts for it? Still guesswork, but our hunch is that if you can open up a new subject matter for the person to explore, they get more use out of the site as a whole. While if you tunnel vision them down the same old road, pointing to another story about the same subject, they tire more quickly and look elsewhere for the next interesting thing.
In the content world– which is very different than the search world– the key is to unearth what’s interesting, not necessarily what’s related. We are constantly working here on new ways to identify the interesting for each visitor to a site and to bring it out front and center, keeping them happy, engaged and coming back for more. If anyone has ideas about additional ways to pinpoint what’s interesting through technology, we’d love to hear them!