Need Yelp? Study Shows Search Ads Work for Small Business

Not that long ago, if you were looking for a good restaurant to try, chances are you would ask someone whose opinion you trusted—a family member, friend, neighbor, or co-worker. 

Today, that information is quite literally at your fingertips. Online search engines, review sites, and mobile apps provide instant access to listings, ratings, and reviews of restaurants specializing in virtually any kind of food you crave in any particular neighborhood. 

But if you happen to own a small restaurant with favorable reviews in a city like New York, Chicago, or San Francisco, where there are thousands of restaurants to choose from, potential diners may never see your listing among the search results. 

One option is paid search advertising, which brings your business to the top of the list of results when relevant keywords are searched. In 2015, online advertising accounted for roughly $60 billion in spending in the United States alone. About half of this amount was spent on paid search advertising. 

But how effective is paid search advertising? Does it actually lead more consumers to your business? 

My colleague Michael Luca, an associate professor at Harvard Business School, and I conducted a large-scale field experiment involving more than 18,000 restaurants and 507 million advertising exposures on the popular consumer review platform Yelp.com to help answer those questions. As part of our experiment, we treated 7,210 restaurants to three months of paid search advertising on Yelp. 

Our study showed that paid search ads on Yelp resulted in a 25 percent increase in page views, or the number of clicks on the business’s Yelp page. And it led to an increase of 9 to 18 percent in people who demonstrated an interest in going to the restaurant by requesting directions, going to the restaurant’s website from its Yelp page, or directly calling the restaurant from the mobile app or the restaurant’s mobile site on Yelp. 

We also considered the number of reviews left on Yelp during the advertising period. If more people are going to the restaurant as a result of advertising, it would make sense that the restaurant would get more reviews during that period. Indeed, we found that the restaurants treated to free advertising during our experiment experienced a 5 percent boost in the number of reviews they received. 

Our Methods

It is challenging to estimate the effectiveness of advertising using observational data. The correlation between advertising and revenue does not imply causality. For example, a business that chooses to start advertising during the holiday season may see a boost in its revenue and infer that the advertising is effective. However, we cannot tell whether the change in revenue during the holiday season compared to before is really due to advertising or merely reflects the seasonal trend. 

In a similar manner, measuring the effects of advertising by comparing the revenues of restaurants that advertise versus ones that do not relying on observational data could lead to biased estimates because there are unobserved, underlying factors that drive the business’ advertising decision that also correlate with the business’ revenue. 

To avoid biasing the estimates of advertising effectiveness, we randomized which of the 18,295 restaurants would get ads and which wouldn’t. We only included restaurants that were eligible to purchase search advertising on Yelp, which requires verified listing information, including photos and a minimum number of trusted reviews with good ratings. 

The treated restaurants received standard search advertising packages offered by Yelp. The standard package is sold for $400 a month and promises the businesses a minimum number of paid impressions. Geoff Donaker, Yelp’s COO at the time, agreed to allow us to randomly assign three months of free advertising to 7,210 restaurants. Before we launched our experiment, Yelp agreed that we had complete independence to publish the results, regardless of the outcome. 

As a detail in the implementation, the restaurants were not told that they were being given the free ads. At the end of the three months, we compared the results for the treated restaurants that were given the free advertising package and those in the control group that did not receive free ads. 

While the restaurants with free ads saw a spike in page views, consumer interest in getting directions, visiting the website, calling the restaurant, and leaving reviews during the period of the advertising, it should be noted that those increases disappeared immediately when the advertising stopped. This suggests that ads temporarily increase the exposure of the advertised businesses that would otherwise not be discovered, but the effect is short-lived. 

That being said, the noise in the outcome measures makes any effect that is not large enough difficult to detect statistically. So the effect afterward could just be small and not detectable. Also, it is possible that a customer who visits the restaurant due to the search ads may return to the restaurant later without using Yelp. This benefits the restaurant in the long-term, but is not captured in our Yelp outcome measures. 

The finding of our research contrasts with an earlier, influential study involving eBay that showed that paid search advertising for large, well-known brands through brand keywords has zero effect on revenue. We found that advertising on sites like Yelp can have a large impact for smaller and lesser known businesses. 

And that ultimately can benefit consumers by helping them discover businesses they may have otherwise overlooked. The more choices consumers have, the better.