Google Threatens Real Estate Professionals. Or Does It?
There’s a debate raging in the real estate world. It isn’t pretty and it isn’t simple. The National Association of REALTORS® (NAR) has supported a regional association’s decision that Google is a scraper site. The Indianapolis Metropolitan Board of REALTORS® (MIBOR) has issued cease and desist orders to a number of its members, telling them to implement robots.txt directives that prevent Google from indexing certain property listings. NAR has tentatively agreed with MIBOR’s claim that the Google index is nothing more than a receptacle of stolen content.
Photo by Phil Scoville
via Creative Commons
This week is the REALTORS® Midyear Legislative Meetings & Trade Expo. The annual event brings together NAR leadership, and it is expected that real estate professionals affected by the decision will ask the association to reconsider. The impending judgment is being closely watched by real estate professionals the nation over as the final outcome could either set a long-standing course for progression or futile resistance.
I consulted BCI tech writer Paula Allen, who worked for a real estate industry vendor for 12 years, for her take on why the NAR and other professional real estate associations are getting their pantaloons in a bunch. Aren’t they aware that getting listings to show up as Google results is as good as free marketing, I asked? Is this just another case of overzealous, old-school types clinging to the past? Where real estate professionals were once the sole owners of listing information, was the solution to educate them on the power of search technology?
As Paula sees it, there’s certainly a measure of that mentality getting in the way here. But there could be an even bigger problem that needs to be addressed before Google can be seen as a partner and not the enemy. To explain this requires a little background.
A multiple listing service, or MLS, is a handy cooperative that allows real estate agents to view and share listings with each other. Starting from the old days of the printed MLS book and continuing with today’s Internet-based systems, access to the MLS data was long restricted to subscribers and is still governed by a strict set of rules about who can show which property listing information to whom. When competitive third-party real estate sites like Yahoo! Real Estate and Zillow started popping up, the MLS established the Internet Data Exchange (IDX), an authorized way to share MLS listings online. Those brokers who opt in to their MLS’s IDX have their listings show up on other participating brokers’ sites. In exchange, their site can advertise the listings of other member brokers as well. In this way, real estate professionals hoped to make listings available to people searching online while still maintaining some control over the information. But, apparently, it wasn’t enough control.
As a member of the IDX, the displayed contact information for listings on your site is your own. So if broker Sally and broker John both participate in IDX, John’s listing on Sally’s IDX site will show up with Sally’s contact info. That way, Sally will get the call, and potentially be able to represent an interested buyer. This is fair enough if someone searching Sally’s site has come upon the listing.
However, things start to get fuzzy if that IDX listing is indexed in Google and someone finds Sally that way. According to the MLS, the listing doesn’t belong to Sally; it belongs to John. John retains the control to put the listing where he wants. If he’s smart, he’ll submit his listings directly to Google and other search engine real estate services to get the traffic himself. And if he opts in to the IDX, then he has chosen to show his listing in that way. However, he hasn’t allowed Sally to get all the credit for his listing in the SERPs.
Photo by FaceMePLS via Creative Commons
While on the surface it appears to be a simple “get over your power trip” diagnosis, there are a number of complicated implications in allowing Google to index IDX listings as they currently stand. I initially thought I’d berate NAR, demand that they embrace search engine technology as free marketing or be doomed to failure! But under the current system, indexed listings could be unfair to the rightful content owners.
The solution, in my assessment, will be two-fold. There will need to be a restructuring of the IDX system so that the contact info listed is that of the selling agent. Then, if IDX listings are indexed, credit will go where it is due. In the meantime, agents should embrace the power of search engine marketing and submit their property listings to sites like Google and Yahoo. This way they can proactively ensure they get all the inquiries. With an improved IDX system and a partnership mentality in place, search engine marketing education can take root. Down the road real estate professionals may just wonder what all the fuss was all about.
Update: For more on the real estate industry’s search marketing challenges, please see our series Jumping Into the Real Estate Fray, parts 1 and 2.
16 Replies to “Google Threatens Real Estate Professionals. Or Does It?”
We are hearing about news coming from Google in this area in the next 60 days. Don’t view them as a threat but rather a large efficient syndication opportunity for our listings to provide better more effective exposure for our listings.
Todd, you said: “… The way in which the IDX provider is publishing content (so that Google can read it) opens the door to scrapers as well. That’s not in compliance with the IDX policy voted upon by our members.”
This demonstrates that you, and the rest of NAR and MIBOR who are raising this as an issue, are totally out of touch with the technical realities. Malicious scrapers can easily extract and use IFrame links from any website to go directly to the IDX vendor site to get the same data. IFraming the IDX, thereby hiding it from Google and the other search engines, who actually play nice about it, or putting a robots noindex tag in the header, does absolutely nothing to protect the data from malicious scrapers. I’m sure that they have long ago figured out how to get what they want from any IFramed IDX, and the ignore noindex tags as well.
In addition, I have checked into four public IDX websites operated directly by Associations and MLS companies, not 3rd parties or brokers, and found that their IDX websites also display property addresses, MLS numbers and other MLS content in clear form, not under an IFramed wrapper, and all with robots index tags in their property details page’s head section. Therefore, would it be NAR’s intent to require them to also IFrame their content, and/or put noindex in their robots tags, even though neither of those actions will actually prevent malicious scraping?
I think the google directory just serves as better exposure. Obviously ever agent in Missouri wants to rank for the search of the cities and properties near them. Since there are literally thousands of agents with poor websites this will never happen. SOmeone needs to be the go to source.
With the listing showing up in Google as they are seen by the Realtors, themselves, the Realtors wind up losing one of the advertised benefits of working with them as opposed to looking for a property on your own: having access to information about homes and properties that the general public does not.
This isn’t how those of us fighting this issue see it. We want the listings to show up in Google – all over the place. By allowing Google to index a site that contains an IDX feed of my listing, I am increasing my advertising, not weakening it. I want the world to see my listing and not just be able to find it on my site via my Google rankings. The more agents that have indexable IDX feeds, the more exposure for my home. Sure, I might lose the #1 spot for a listing, but my goal is not SEO, my goal is to get the home sold. That’s my duty to my clients and what puts food on my table.
The ones that seem most worried about the issue are those that use many of the tricks and tactics that Cal mentions above. They are trying to get a warm body to convert into a lead. Me, I’m just trying to do what I told my sellers I’d do – sell their home.
Like Cal, I would like to clear up a few misconceptions that are out there…
First, concerning the MLS. Most Consumers AND Agents don’t understand what the MLS is.
“The Multiple Listing Service is a means by which authorized MLS broker participants establish legal relationships with other participants by making a blanket unilateral contractual offer of compensation and cooperation to other broker participants; by which information is accumulated and disseminated to enable authorized participants to prepare appraisals and other valuations of real property for bonafide clients and customers; by which participants engaging in real estate appraisal contribute to common databases; and is a facility for the orderly correlation and dissemination of listing information among the participants so that they may better serve their clients, customers and the public.” – http://www.sandicor.com/rules-regulations/2007-APPROVED-RULES0808a.pdf
Therefore, the MLS was established as a means for agents to cooperate with other agents. It is an agent to agent offer of compensation and Co-Operation.
What it has become with the growing popularity of the internet… It has become a means by which internet savvy Brokers and Agents can set up virtual offices to compete with the traditional Brick and Mortar Real Estate Brokerages. This is a threat to those Brick and Mortar Businesses
What it is NOT… It is not, and never was intended to be, an advertising tool for listing agents to get more buyers to call them directly (and potentially NOT be represented by a Buyers Agents). Remember it was and still is simply an offer for compensation and Co-operation between agents.
As Cal stated; the consumer does NOT benefit when they are NOT being properly represented (in my opinion Dual Agency is not in anyone’s best interest except the Listing Agent).
What is Dual Agency? Dual Agency is the listing agent attempting to represent both the buyer and the seller. Folks, the listing agent has a responsiblity to get the most money they can for the Seller. The only incentive they have in working both sides of a transaction is to Keep ALL the Commission (P.S. in almost every case there is no reduction in the amount of commission if the buyer also uses the same agent as the seller).
To all you buyers out there – the total commission the seller pays has already been determined by the listing contract. The buyer thinking that they may save some commission is one of the buyers biggest misconceptions.
Finally, to Todd
Todd – do we really need another lawsuit from the Department of Justice in order for NAR to do the right thing here? http://www.usdoj.gov/atr/public/press_releases/2005/211008.htm
Heck, if NAR really wants to end these silly squables then the National Association of Realtors should outlaw Dual Agency.
You wrote, “As we have continually remarked in other blog posts covering this story, we are not classifying Google as a scraper.”
While the NAR may not have directly said, “Google is a scraper”, Cliff Niersbach, NAR’s Vice President of Board Policy & Programs, DID say, “Consequently, NAR staff responded to questioners that the requirement to prevent scraping includes indexing.”
So our duty to prevent scraping includes preventing indexing by Google and other search engines. That is the crux of this issue. Let’s not get bogged down in semantics. Preventing indexing by legitimate search engines was never the intent of the original policy, yet it has now become a part of the policy.
And let’s be honest here — there is nothing anyone can do to prevent someone from scraping a site if they really want to.
Hi Todd. Thanks for dropping a comment on the blog. We reported on the adjustment to NAR’s search engine policy in a later post, http://www.bruceclay.com/blog/archives/2009/05/jumping_into_th_1.html .
I’m the social media manager for the National Association of Realtors. As we have continually remarked in other blog posts covering this story, we are not classifying Google as a scraper. The way in which the IDX provider is publishing content (so that Google can read it) opens the door to scrapers as well. That’s not in compliance with the IDX policy voted upon by our members.
Thanks, Cal. You’ve obviously got a lot of experience with this issue and your points provided excellent background to explain the politics of the situation. Much appreciated!
You can see that we dove into this issue more deeply during the week. We’d love to hear your thoughts on the problems and solutions laid out in the series: http://www.bruceclay.com/blog/archives/2009/05/jumping_into_th.html and http://www.bruceclay.com/blog/archives/2009/05/jumping_into_th_1.html .
Just to clarify a few points:
Brokers own the listings, not the agents.
All IDX solutions are required to display the name of the listing Broker and do.
Brokers (or their seller clients) may Opt Out of IDX completely or by the listing. However, I can’t think of a better way for a seller to loose faith in a Broker/Agent than for them to limit the visibility of the listing to the buying public.
If as suggested – “There will need to be a restructuring of the IDX system so that the contact info listed is that of the selling agent”, why in the world would tech savvy agents expend thousands of dollars and thousands of hours working on websites to advertise another agents listing and then send the website visitor to that agent?
There are two sides to every transaction, the buyer side and the seller side. If the listing agent has both the buyer and seller (dual agency), then the dual agent can not provide the same level of disclosure to either the buyer or the seller. It is beneficial to the buying public to be able to be represented by their own buyer agent if they so choose.
There are many tech savvy agents that seek to work with buyers. If the listing agents in their greed to take both sides of the transaction and possibly to the detriment of their clients want to shut off data to the IDX sites, that is their perogative.
If listing agents want to shut out buyer agents and return to the days when only the seller’s interests were represented, then that needs to be disclosed to their sellers, the properties Opted Out of the IDX, and then they can sit back and wait and wonder why their listing agent is not getting any buyers from the internet shopping public.
If as suggested, “the listing agents contact information is to be included on the IDX”, rest assured I will not be providing free advertising services for those agents or their clients.
I know the Bruce Clay organization is cognizant that well performing websites require quality, relevance, and investmet of large amounts of time in addition to money.
Part of the contingency that is opposed to the indexing is the same group that goes to listing appointments saying “I have a website”, yet fail to disclose that there is no traffic to it and that it is not even in the Google index. Heck, you can’t even find the agent by name in Google search, much less the property.
Again, they should feel free to drop the listings from the IDX. These people will be the same ones that post almost no information about the property and the “MLS required” single photo anyway.
These will be the same agents that put out flyers with no prices so you have to call them to see that the home you were interested in is not in your price range (or not listed anymore). The same ones that forget to mark their listings Pending and Sold so they keep getting exposure to themselves. Also, the same ones that buy ads in the Real Estate Book with no address for the property, again so that you have to call them to find out that the cute house is 60 miles away from where you work!
Listen folks, take the IDX indexing away, there are still going to be greedy and lazy individuals that are going to sell property owners on their services and the properties will not show up in searches because they are a)too lazy b)too dumb c)too greedy to put up they very information that might bring them site visitors. Information such as prices, addresses, multiple photos, adequate property details.
Believe me, you just don’t know how many times buyer agents with successful websites are asked for pictures of something that might be important to a home buyer – yes like kitchens, living rooms, master bedrooms – and then ask the listing agent to provide them and they are never forthcoming making the buyer agent look inept when it is the listing agent that has done his seller a disservice. The information on their websites is as basic as the information in their old newspaper and Real Estate Book adverts and will never be indexed because there are no keyword phrases present. They think a pretty graphic rich webpage will get them pasted all over Google, yet there is not one word about the geographical area that they serve.
I say encourage them to opt out of IDX!
Virginia, thank you for this highly timely and interesting post. We have a few real estate developments as clients and the other issue with this is the tremendous competition that will be further placed on FSBO’s (for sale by owners) this may also include new home developments that are not listed in MLS, and if they are being managed by a broker, it will cause these new home developments to have to rethink their entire sales process. Something like this is going to really cause a shake up in the way all real estate parties handle their business. Certainly interesting.
Google really threatens the real estate. I have been working in an advertising company and most of our customers were real estate agencies. The greatest decrease of customers or clients were those that emerged for advertising on Google for international adverting. That is one way of interacting in the real estate.
Thanks for the article, I agree with Paula in that I view google as a great free marketing resource for my listings.
I’ve been working with Realtor websites for the past 5 years, and the MLS boards are Greedy. Realtors are Greedy.
There. I said it. You can hate me for it but honestly I’m not surprised at the way that our housing market was full of corrupt Realtors and Mortgage Brokers who sold us all down the river.
That being said, MLS rules have always been restrictive, and made Real Estate websites complicated and difficult.
“I initially thought I’d berate NAR, demand that they embrace search engine technology as free marketing or be doomed to failure!”
You had it right. These MLS boards have been lagging far far behind the technology curve, and now are lashing out because they’re losing their power, and are making it even harder for Realtors to make a living in such a tough economy.
Another reason Realtors are probably up in arms is because they pay exorbitant fees in order to be a part of the MLS service, and in many area you don’t have the option of opting out. That membership is/was supposed to provide exclusivity to the information contained within the MLS system. With the listing showing up in Google as they are seen by the Realtors, themselves, the Realtors wind up losing one of the advertised benefits of working with them as opposed to looking for a property on your own: having access to information about homes and properties that the general public does not.