In a blog post published yesterday, OkCupid revealed it’s been lying to some of its users just to see how manipulating their experience could make the site better at matchmaking.
The public’s reaction to OK Cupid’s admission of the kind of A/B testing that Facebook caught hell for has been much more muted. It turns out, treating your users like guinea pigs is ok as long as people already like your website (nobody really likes Facebook — they’re just trapped by the network effect).
So what exactly did they do? From the blog post:
The ultimate question at OkCupid is, does this thing even work? By all our internal measures, the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on. But in the back of our minds, there’s always been the possibility: maybe it works just because we tell people it does. Maybe people just like each other because they think they’re supposed to? Like how Jay-Z still sells albums?
To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.
† Once the experiment was concluded, the users were notified of the correct match percentage.