How the battle for local search will be won

Part 1: Local search is starting to get more social
Part 2: How the battle for local search will be won
Part 3: Google Hotpot a strong competitor to Yelp
Part 4: Statistics on business and consumer engagement in local search
Part 5: Foursquare 3.0 takes mobile ball to a whole new level

Local search has changed dramatically in the last decade. Gone are the days when you could buy a generic database from a mailing list provider, slap maps on it and have a local search solution. Social networks, mobile phones and businesses themselves are changing and enriching local search.

These are the key factors that will define success in local search going forward:

User generated content and engagement

Yelp and foursquare's mobile apps allow users to add photos.

Yelp and foursquare's mobile apps allow users to add photos.

The best local search databases are content rich. They include attributes such as hours of operation, friendliness of the place to kids and pets, whether there is outdoor seating, etc. Many of these attributes are collected by users themselves. Increasingly, this is being done on mobile phones — people can update data before they’ve even left the place.

Users also help to maintain the quality of the databases. In my research, there wasn’t a single case where Yelp or foursquare didn’t have a place I was looking for. There were quite a few that I couldn’t find in Google Places and Facebook. For the U.S., the best database of restaurants and bars is at Yelp. New places are often in Yelp’s database as soon as the place opens. (Sometimes even before the official open, as people participate in friends and family dinners and soft launches.) Foursquare’s data are also comprehensive, but are cluttered by users who try to exploit the service’s game mechanics by creating extraneous venues.

Users can also report businesses that have closed, helping alleviate the frustration of driving to a business only to find out that it is no longer around. Check in data on foursquare and Yelp can also identify anomalies. (e.g. a check in stream that suddenly stops can indicate that a business has closed.)

Photos are key components of some of these databases. The growth of smartphones will only further this trend. Some venues on Yelp have a hundred or more photos. Yelp reports that its users are uploading a photo on average once every 30 seconds. Foursquare recently introduced photos. Google is sending professional photographers out to take pictures of top places. Specialized applications like foodspotting have small but loyal audiences who upload pictures of specific dishes at restauranst.

When it comes to mobile data collection, Google’s Hotpot is weak. New places can’t be added and photos can’t be uploaded. It supports ratings, reviews and identifying problems such as closed businesses and duplicate venues.

But providing tools isn’t enough; it’s important to provide the right incentives.

Yelp has done a great job of providing ordinary users incentives to contribute to the maintenance of its database. It uses both social reinforcement and more tangible rewards. Yelp makes it easy for members to thank and compliment each other for reviews. Selected reviews are featured in weekly newsletters. Review often enough and you become a Yelp elite and have a badge on your profile.

Yelp employs community managers in its markets to help reinforce the community. Frequent events (including Yelp-hosted parties) provide more incentives to review and create adhesion among the community. Only a small proportion of the Yelp user base does any of these things. But you only need 1 person to provide value to millions. Yelp’s dedicated, engaged user base will be a significant barrier to other competitors in the space.

Business engagement

It’s taken a long time, but small businesses are finally engaging with consumers online. 60% of the businesses I looked at had Twitter feeds. (See my earlier posts: Why don’t local businesses use the Internet? and Twitter and foursquare: the tipping point to getting local business online.)

Businesses are using these tools to communicate specials, announce closings (e.g. for private events), run promotions, have contests and just engage with their customers. I’ve even seen businesses helping businesses; one business had electrical problems and another business offered her electrician’s number in response. Here is a snippet from a Twitter list I created of restaurants in Portland:

A snippet from a business list in Portland.

A snippet from a business list in Portland.

Beachwood BBQ

Beers on tap at Beachwood BBQ. Click the image for a live version.

This sort of real-time information can help sway a decision or prompt users to go out on a night when they would otherwise have stayed in. Radio Room in Portland does a great job of this with their Twitter feed.

The image to the right is an image from the Hops Cam at Beachwood BBQ in Seal Beach, Calif. It allows users to instantly see what’s on tap now. What spot now?, an iPhone app, allows users to see real-time cameras from various restaurants.

Although Google, Yelp and foursquare allow businesses the opportunity to claim their page, there is no mechanism to communicate with customers through their platform.

Businesses are claiming pages and providing enhanced attribute information. Nearly 2/3 of businesses I looked at have claimed their Google and Yelp pages.

Recommendations

Hotpot recommendations

This screenshot shows a recommendation in Hotpot based on other places I've rated.

To date, no one has done a great job of making recommendations based on a user’s preferences or social network. Local search has required users to sift through mounds of data or just go for serendipity (like in UrbanSpoon). Yelp and foursquare have had some form of social recommendations. Both will highlight recommendations from friends, but their social graphs haven’t been large or relevant enough.

This is a key focus area for Google Hotpot.

When you do a search, you might see recommendations based on other places you’ve rated.

Or you might see that a friend has rated the place. Unlike recommendations from strangers, this provides immediate context. I know some friends whose tastes are similar. If they like a place, I know the chances are good that I will like a place. Negative affinity can be helpful, too. There are a few people whose tastes are so divergent that I know not to go someplace they rate highly.

Pictures also play a big part in decision making. Local search has long relied on textual data because it’s been easy and available. But visuals are a key part of the experience when it comes to dining and nightlife. They can answer questions like “Is this place fancy or a dive?” and “Would this place be a great place for an anniversary dinner?” much quicker than text reviews can. See Picturing a new vision for local search. Pictures are also much easier to go through on a mobile device.

Making intelligent recommendations requires having a lot of data. The easier you can make it, the more participation you will have. Few people will go through the trouble of writing detailed reviews, but 1-click ratings can provide important signals and will have a higher participation rate. See more on recommendation engines for local search.

Distribution

No matter how good your content is, it doesn’t matter if you can’t get it in front of people. Here, Google has an indisputable advantage. Google sites serve 170 million people in the U.S. Yelp reaches 26 million. (Many of these come through the help of Google’s search results.) Foursquare claims several million downloads. The difference in scale is enormous.

Google’s distribution advantage extends to mobile with prominent applications on iPhone and deep integration within the Android OS. Facebook is also a large player here, with more than 150 million unique mobile users worldwide. When they set their eyes to local, they will be a big player to watch.

Sharing

Local search often involves a shared experience. Plans are made and coordinated. So far, no one has really provided a great solution for this. Here’s a simplified version of how the process often works:

Step 1: Person A looks up a place on a local search site.

Step 2: Person A sends the place name via SMS to Person B (and C, D…).

Step 3: Person B gets the text message and looks it up in a local search site to find the address and look up information.

Step 4: Person B responds to Person A that it’s acceptable. (Or not, back to Step 1.)

Step 5: Person B then uses the site to generate driving directions.

This could be greatly simplified. Again, Google’s deep integration into Android provides an advantage. Person A could find and text the place information. The receiving phone would identify that the link is specially formatted and instead of presenting it at as an SMS, would present a Places page with pictures and reviews and an accept/reject button. Such sharing could also help Google build out a social graph.

Local search is starting to get more social

Part 1: Local search is starting to get more social
Part 2: How the battle for local search will be won
Part 3: Google Hotpot a strong competitor to Yelp
Part 4: Statistics on business and consumer engagement in local search
Part 5: Foursquare 3.0 takes mobile ball to a whole new level

Social search has been talked about for several years now as the wave of the future. We’ll get better information with the help of our friends. Local is the ideal place to prove that out:

  • Most people tend to have a lot of friends in their immediate area.
  • Local search revolves around everyday experience.
  • The “answers” are based on opinions.

Google’s entry into the space is Hotpot, a local ratings and recommendations tool. It is spending a significant amount of money to promote Hotpot in the Portland and Austin markets.

Hotpot is clearly meant to compete with Yelp. To a lesser degree, it competes with Facebook Places and foursquare. (It’s funny how much the local search space has changed in the last few years. AOL, Mapquest, CitySearch and newspaper Web sites have largely dropped off the local map in recent years.)

It’s important to set the context in this fight: Google is already the undisputed leader in local search. Despite the attention that other sites get, Google is the number one place people go to get local information. More than 20% of Google queries are local in nature. Google Search serves about 170 million users. I bet 99.99% of them have done a local query. Yelp serves 26 million users in the United States. But many of those users come through the Google front door. (Partly because Yelp is one of the best SEOs out there.)

There are two core problems to be solved in local search. Providing someone additional information on a business whose name they know and providing guidance to those who are open to suggestions on a business.

Business name searches

The first problem is largely solved, despite the fact that the scope of the problem has increased. Just a few years ago, it meant providing someone the name, address, phone number and a map for a business. Today, it increasingly includes providing hours of operation, attributes such as romantic, kid-friendly, links to make reservations and menu information.

Distribution and integration helps Google capture business name searches. You can use the browser’s search box and Google.com to get your answer. With an Android phone, it’s even simpler. Press a button, speak your search and the answer appears.

Google can answer most of the basic questions about many businesses in the United States. Yelp has the best data out there for restaurants and bars in the United States. I’ll get to the reasons why later.

Google has difficulty with non-standard venues. For example, in Portland, it does poorly with food carts. In most cases, I don’t advocate manually updating a database to address localized concerns. But given the amount of money that Google is spending on promotion in Portland and the importance of food carts in the city’s dining scene, they should follow the advice of an Oregon company and “Just do it.” A basic effort could be done in a day by using online resources. A street team could hit all of the major food cart areas and provide enhanced data such as hours and pictures in a few days. (While also handing out Google stickers.)

Discovery searches

The other core problem in local search is discovery — helping to find an appropriate answer when they only have a few parameters or no clue what they’re looking for. These are the questions like “I want a kid-friendly pizza place nearby.” “I want to go to some place fancy,” “I’m looking for a special night out on the town.”

This is an area that Yelp excels at but Google generally sucks at.  The problem with Yelp (and the opportunity for Google) is that getting the most out of Yelp requires a lot of work from the user. Yelp has an incredible amount of rich data on local businesses. But it’s too much. It’s overwhelming to see hundreds of reviews. Using Yelp also means trusting people you don’t know, whose tastes may be very different from yours. And it means dealing with the snarkiness of reviewers who often spend more time talking about their life stories and girlfriend problems than the business they’re supposedly reviewing.

Yelp has introduced a number of tools over the years to alleviate this problem. It does data summarization across reviews so that you can see at a glance what are the things most frequently mentioned about the restaurant (e.g. popular menu items). You can see a distribution of the ratings to see how consistent a restaurant is. You can also see ratings trends to see if the restaurant is getting better or trending downward.

But often, people just want a few options. Too much choice and too much data is overwhelming. People don’t want to spend 30 minutes figuring out where to go. We’ve been getting recommendations from Amazon and Netflix for decades. “People who liked X also liked Y.” “Based on your previous ratings, here are places we think you’ll like.” This is especially important in mobile, where people are often more hurried and the screen real estate in which to read is limited. That’s what Google is trying to do with Hotpot.

In some ways, this is an easier problem to solve than Web search. If you’re looking up answers for Jeopardy, there is usually only one right answer. And if Google can’t find it, you know right away. For a discovery-oriented local search, there is more than one right answer. And if the answer isn’t what you were expecting, you won’t know for hours and you might not blame Google. (The restaurant might have had an off night.) For more details, see my earlier post about making intelligent recommendations in local search.

Picking the right social graph

In order to make the best recommendations, you need data. You need data from the user about their preferences and you need a good social graph from which to present options. The more data the better.

This is a significant challenge for Google. Other companies in the social space such as foursquare, Gowalla, Quora and Instagram, have piggy-backed off Facebook’s social graph. That’s not an option for Google. And I’m not willing to spam all of my friends to invite them to use Google Hotpot. The advent of Facebook Connect has made such spamming less socially acceptable. As a result, I have exactly one friend on Hotpot — and he’s a Google employee.

Foursquare’s social graph is OK, but it’s a bit small given the current focus on check ins. The number of people who I want to be able to see where I’m at in realtime is fairly small. But I’d be comfortable sharing historical data on reviews and ratings with a much larger audience.

Facebook’s social graph is ideal for this application. It has a lot of personal connections, including both close and loose connections. The loose connections are important because they help provide coverage that you might not have in your tighter friend circle. For example, the data to make recommendations for Indian restaurants in Paris might be from a former colleague who now lives in Paris.

In the next part of this series, we’ll look at some of the key success factors for local search.

Part II: How the battle for local search will be won

How Google could dramatically improve local search

A lot of companies have been spending a lot of time and effort in location-based services over the last couple of years. Whether it’s local search or check ins, the race to get people connecting with local businesses is on.

One ongoing challenge has been identifying where consumers are.  GPS has issues with power consumption, time to first fix and doesn’t work indoors. Cellsite-based location is not precise enough. Even WiFi triangulation, which is the most effective way currently, isn’t precise enough given current deployments. In densely packed urban areas, you can still come up with a hundred or more businesses that you would have to pick through.

One way that Google (or Facebook or anyone with a strong brand) could solve this problem is to send WiFi beacons to local businesses. This is roughly how it would work:

  • Routers are sent to businesses. The MAC address of the router is recorded and correlated with the address that it’s shipped to.
  • The business receives it and plugs it into a wall outlet.
  • The router then transmits its information to nearby phones.
  • Those phones can narrow the list of potential businesses based on that information.

This doesn’t even require the business to have an Internet connection. The only requirement is that the device be powered. At scale, the device could be custom designed to eliminate the Ethernet jacks on routers. This reduces costs and makes the device look less intimidating to folks who aren’t tech savvy. If you wanted to get fancy, you could shape the device so it didn’t look like a router at all — maybe something like the Open sign that Google is giving away. This would have the added benefit of branding to the business’s customers.

With a per device cost of approximately $15 and a service life of about 3 years, we’re looking at a cost of $5/year. If you sent them to 500,000 businesses (the focus should be bars/restaurants in high density urban areas), it’s still a modest cost of $7.5 million to tap into the local market.

The pitch to local businesses would be something along the lines of “make it easier for Google users to find you.” It could be presented as part of a small business starter kit, complete  with Google Places window decals, a guide to online advertising, personalized information on how the business is currently rated on Google and online advertising credit for use on Google. It could also serve as the validation mechanism for businesses to claim their Places page. In my experience, packages are more likely to be opened than typical direct mail pieces.

While there has been a lot of talk about NFC for searching or tagging, it would require a change in user behavior and is likely to take 2-3 years before a sufficient number of NFC-enabled phones are in use in the United States.

Not only would this sort of network enable easier local search and check ins, it could be used to generate real time maps of where the most popular places in a city are. People could also use it to generate automatic check ins when they reach selected favorite places.

If this sounds crazy, consider that Google is already testing giveaways for businesses in the Portland area as it tests its Hotpot product. Businesses can order free sugar packets, mints, magnets, billfolds and more.

The biggest challenge with this approach is the risk of bad press given the kerfuffle regarding StreetView vehicles capturing WiFi data by mistake. Although this is in no way equivalent, the media have a hard time understanding that. (Not to mention that the original issue was really blown out of proportion.) This could be offset if Google made the database open to the public. Not only would this improve results for Google applications, but could be used by a wide range of devices to improve position accuracy. It would be the equivalent of Google launching satellites for the public’s benefit.

EVO vs. iPhone

I’ve been using an HTC EVO since last Friday. As an iPhone user for the last two years, this is the first Android phone that has appealed to me.  CrunchGear has a good comparison of the technical specs of the iPhone and the EVO.

The two biggest complaints others have voiced about the EVO are bulk and poor battery life. Yes, it is bulky. It’s the heaviest phone I’ve had in at least 5 years — at 6 ounces, it’s 25% heavier than the iPhone 4G. It’s width makes it more awkward to hold than an iPhone, but not uncomfortably so. But it also has a big, beautiful screen. Life is a tradeoff.

I haven’t had issues with battery life, but then I don’t talk a lot on my phone. Unlike with the iPhone, you can carry around a spare battery.

The other issue that has been mentioned regularly is the on-screen keyboard. The iPhone’s keyboard is less complicated, but the EVO let’s you accomplish more tasks (like entering numbers) without leaving the main keyboard. The one issue I’ve definitely noticed is that some keys on the left side haven’t been registering consistently. (e.g. “A” and “S”)

While others have railed against one or the other, the phones are different enough that they’re likely to appeal to different people. I’ve tried to identify those below.

For typical consumers, my recommendation would be the iPhone, provided that you’re in an area where AT&T’s network isn’t saturated. For me? I’ve got three more weeks to decide.

If you…

… have a lot of music or photos and like iTunes.

Go with the iPhone. I haven’t been able to find a decent media synchronization experience for EVO. I used my iPhone frequently for podcasts and those are easy to set up and synch with iTunes. I also synch photos from my computer to my iPhone. Again, not something I can do with the stock EVO.

… want to customize your phone experience.

Go with EVO. You can customize a lot of elements of how the phone operates. You can create themes for different uses, e.g. a work theme, play theme and travel theme. Each theme can have different applications, shortcuts and widgets. It’d be even nicer if you could change themes automatically based on time of day or location. (e.g. work theme while at the office)

… don’t want to know what a task manager is.

Go with iPhone. Ordinary users should never have to see things like com.google.android.apps.googlevoice. It’s difficult to figure out what apps are running on the EVO. That’s problematic because you could easily have an unknown app running down your battery.

… want something that looks pretty.

Go with iPhone. It’s hard to top Apple design. The EVO is bulkier and certainly looks more utilitarian than iPhone. The EVO screen also shows fingerprints a lot more than my iPhone 3G.

… give out your Google Voice number to friends, family and colleagues.

Go with EVO. The Google Voice integration is incredible. Calls you make can be routed through GV automatically. Calls are logged correctly in the phone and on the GV site. Voicemail is also seamlessly integrated. Text messages aren’t integrated into the phone’s messages app.

… want a broad selection of apps.

Go with iPhone. Yes, it’s not open and yes, Apple can arbitrarily reject apps. But iOS has many more apps written for it. While many of the major apps are on both platforms, I couldn’t find equivalents for flickr or Zipcar on Android. Google Voice is the key exception of an app that’s on Android but not iPhone.

For gamers, the iPhone advantage is even stronger. With the gyroscope on iPhone 4, gaming will only get better.

…  like flickr, Facebook and Twitter.

Go with iPhone. The Facebook and Twitter apps for iPhone are much more polished than their Android counterparts. For example, on the Facebook app, clicking on a link someone has shared sends you on an infinite loop between the shared item and the person’s wall.  (Google VP Vic Gotundra recently gave a Facebook intern an HTC Evo in hopes of getting a better experience on Android.) I couldn’t find an official flickr app for Android.

HTC includes some tools for all three networks that integrate them into the phone’s UI. For example, contact lists from all three can be integrated with the phone’s main contact list. This sounds great — and is the right direction for phones — but the software isn’t ready for prime time. I often see the same people listed 3 or 4 times. (You can manually consolidate these for each person, but that’s a lot of work.) If you set up favorite people, you’ll see when they’ve updated their social networks. Background downloading of status updates also takes a toll on battery life.

… have terrible AT&T coverage.

Go with EVO. AT&T’s networks in SF and NY are overloaded and getting data connections or making a call can be a real challenge.

I’ve had few issues with Sprint’s network. Sprint also includes roaming on Verizon’s network.

… want something that “just works” out of the box.

Go with iPhone. The stock EVO is much more customizable than a stock iPhone. With customization always comes complexity. When iPod came out, a lot of techies criticized it for being a dumbed down MP3 player. Other MP3 players of the time had FM radios! They didn’t tie you into one company! But by stripping away all those extra features, Apple created something that just worked for the most common tasks for most people.

Same is true with iPhone. Owning the entire stack gives Apple a huge advantage in creating a user experience that just works across its enormous userbase. Video calling will work the same across all iPhone 4s. Not true with Android.

With HTC’s Sense UI, Android, Sprint customizations and apps all playing a part, the EVO experience doesn’t hold together.

Although features like social networking integration will be important, what HTC has done with EVO is too confusing for most people.

… want to be able to connect your laptop, iPad or other devices.

Go with EVO. Although AT&T is now offering tethering, they’re charging an extra $20 a month and the usage still counts against your 2GB data limit. For$30 a month, Sprint offers unlimited data and a wireless hotspot that supports up to 8 devices simultaneously. If you don’t need that, you might be able to use an app like PDANet to tether your laptop without paying the $30 a month.

… talk a lot, text a lot, use a lot of data or use navigation and want to economize.

Go with EVO. Sprint’s pricing plans are generally cheaper than AT&T for heavy users. For $80, Sprint includes unlimited nights (beginning at 7pm vs. 9pm for AT&T) and weekends, unlimited calls to any mobile phone (vs. just AT&T customers), unlimited texting (an additional $20 on AT&T) and navigation (extra $10 on AT&T). Sprint also has generous corporate discounts that can knock up to 25% off the bill. Low volume users who can get by with less than 250MB of data a month are better off with AT&T.

… are a world traveler.

Go with the iPhone. With GSM, you’ll at least have the option of international coverage in most countries, even if you have to pay exorbitant roaming rates. Of course, it’s best to unlock your phone and use local carriers if you’re spending any amount of time outside the country.

… are uncertain.

Try EVO. Sprint offers the most generous return policy in the business. You have 30 days to decide whether you like it. If you don’t, you can take it back and you won’t pay anything. They won’t even charge you for the service you used. AT&T will charge you for the service, plus the activation fee, unless you return within 3 days. Sprint’s early termination fee is also lower, $200 vs. $325.

NOTE: Comparisons here are based on a stock iPhone vs. a stock EVO.

Geo-enabled Twitter comes alive on Twitter Maps

Bing's Twitter Maps show you what's going on

Bing's Twitter Maps show you what's going on

I’ve been playing with Bing’s Twitter Maps lately and it’s one of the better implementations of Twitter’s geo APIs that were introduced last fall. It shows tweets and foursquare checkins within the last 7 days plotted on the map. Google Maps recently introduced a similar feature, but it seems to only show items that are fed through Google Buzz (including tweets that people have configured to send to Buzz).

Some future applications of geo-enabled Tweets:

  • Events. For last-minute party goers, a real time view of what’s going on around town, complete with pictures and real-time reactions.
  • Ticket scalping. Rather than walk around for blocks talking to scalpers about what they have, glance at a list of tickets posted. The information transparency would result in a higher price to sellers and a lower price to buyers than what scalpers typically offer. (In my experience at baseball games, scalpers usually ask at least 3x what they paid.)
  • Finding a place to go. When in new cities, it’s often hard to figure out where to go — what are the lively neighborhoods at night. By looking at a map of recent tweets, you could quickly discover where people are still awake.
  • Read reviews from friends. Geo-enabled tweets filtered by those you follow would provide socially relevant recommendations.
  • Offers from local businesses. These could be persistent or distressed inventory. Slow night? Tweet an offer to draw in customers.
  • News. Twitter has long been used for user-generated breaking news. With geo-enabled tweets, breaking news could be aggregated by location in addition to hashtags. The biggest stories could be identified by an increase of tweets from a location (versus normal) and retweet frequency. News from media outlets could also be plotted.
  • Construction and accident information. Avoid bottlenecks by seeing tweets from fellow drivers, DOTs and news sites.
  • Trip sharing. Find others at the airport headed your way, cutting costs and reducing pollution.

And, of course, there’s friend finding, which is the most talked about use of geo-enabled tweets.

So far, the percentage of tweets I see with geo information is tiny (>1% of those I follow). But as more and more geotagged data is put into Twitter, the key will be applications providing the right tools to filter all of that data. At a minimum, we’ll need the ability to filter by time of tweet, people we’re following, hashtag and application (e.g. foursquare).

Unfortunately, bing’s Twitter Maps doesn’t seem to be available where real-time information would be most useful — on mobile devices.

More on: geotagging, social networkingTwitter