This demonstrates that you, and the rest of NAR and MIBOR who are raising this as an issue, are totally out of touch with the technical realities. Malicious scrapers can easily extract and use IFrame links from any website to go directly to the IDX vendor site to get the same data. IFraming the IDX, thereby hiding it from Google and the other search engines, who actually play nice about it, or putting a robots noindex tag in the header, does absolutely nothing to protect the data from malicious scrapers. I’m sure that they have long ago figured out how to get what they want from any IFramed IDX, and the ignore noindex tags as well.

In addition, I have checked into four public IDX websites operated directly by Associations and MLS companies, not 3rd parties or brokers, and found that their IDX websites also display property addresses, MLS numbers and other MLS content in clear form, not under an IFramed wrapper, and all with robots index tags in their property details page’s head section. Therefore, would it be NAR’s intent to require them to also IFrame their content, and/or put noindex in their robots tags, even though neither of those actions will actually prevent malicious scraping?

With the listing showing up in Google as they are seen by the Realtors, themselves, the Realtors wind up losing one of the advertised benefits of working with them as opposed to looking for a property on your own: having access to information about homes and properties that the general public does not.

This isn’t how those of us fighting this issue see it. We want the listings to show up in Google – all over the place. By allowing Google to index a site that contains an IDX feed of my listing, I am increasing my advertising, not weakening it. I want the world to see my listing and not just be able to find it on my site via my Google rankings. The more agents that have indexable IDX feeds, the more exposure for my home. Sure, I might lose the #1 spot for a listing, but my goal is not SEO, my goal is to get the home sold. That’s my duty to my clients and what puts food on my table.

The ones that seem most worried about the issue are those that use many of the tricks and tactics that Cal mentions above. They are trying to get a warm body to convert into a lead. Me, I’m just trying to do what I told my sellers I’d do – sell their home.

Like Cal, I would like to clear up a few misconceptions that are out there…
First, concerning the MLS. Most Consumers AND Agents don’t understand what the MLS is.

“The Multiple Listing Service is a means by which authorized MLS broker participants establish legal relationships with other participants by making a blanket unilateral contractual offer of compensation and cooperation to other broker participants; by which information is accumulated and disseminated to enable authorized participants to prepare appraisals and other valuations of real property for bonafide clients and customers; by which participants engaging in real estate appraisal contribute to common databases; and is a facility for the orderly correlation and dissemination of listing information among the participants so that they may better serve their clients, customers and the public.” – http://www.sandicor.com/rules-regulations/2007-APPROVED-RULES0808a.pdf

Therefore, the MLS was established as a means for agents to cooperate with other agents. It is an agent to agent offer of compensation and Co-Operation.

What it has become with the growing popularity of the internet… It has become a means by which internet savvy Brokers and Agents can set up virtual offices to compete with the traditional Brick and Mortar Real Estate Brokerages. This is a threat to those Brick and Mortar Businesses

What it is NOT… It is not, and never was intended to be, an advertising tool for listing agents to get more buyers to call them directly (and potentially NOT be represented by a Buyers Agents). Remember it was and still is simply an offer for compensation and Co-operation between agents.

As Cal stated; the consumer does NOT benefit when they are NOT being properly represented (in my opinion Dual Agency is not in anyone’s best interest except the Listing Agent).

What is Dual Agency? Dual Agency is the listing agent attempting to represent both the buyer and the seller. Folks, the listing agent has a responsiblity to get the most money they can for the Seller. The only incentive they have in working both sides of a transaction is to Keep ALL the Commission (P.S. in almost every case there is no reduction in the amount of commission if the buyer also uses the same agent as the seller).

To all you buyers out there – the total commission the seller pays has already been determined by the listing contract. The buyer thinking that they may save some commission is one of the buyers biggest misconceptions.

Finally, to Todd

Todd – do we really need another lawsuit from the Department of Justice in order for NAR to do the right thing here? http://www.usdoj.gov/atr/public/press_releases/2005/211008.htm

Heck, if NAR really wants to end these silly squables then the National Association of Realtors should outlaw Dual Agency.

You wrote, “As we have continually remarked in other blog posts covering this story, we are not classifying Google as a scraper.”

While the NAR may not have directly said, “Google is a scraper”, Cliff Niersbach, NAR’s Vice President of Board Policy & Programs, DID say, “Consequently, NAR staff responded to questioners that the requirement to prevent scraping includes indexing.”

So our duty to prevent scraping includes preventing indexing by Google and other search engines. That is the crux of this issue. Let’s not get bogged down in semantics. Preventing indexing by legitimate search engines was never the intent of the original policy, yet it has now become a part of the policy.

And let’s be honest here — there is nothing anyone can do to prevent someone from scraping a site if they really want to.

Brokers own the listings, not the agents.

All IDX solutions are required to display the name of the listing Broker and do.

Brokers (or their seller clients) may Opt Out of IDX completely or by the listing. However, I can’t think of a better way for a seller to loose faith in a Broker/Agent than for them to limit the visibility of the listing to the buying public.

If as suggested – “There will need to be a restructuring of the IDX system so that the contact info listed is that of the selling agent”, why in the world would tech savvy agents expend thousands of dollars and thousands of hours working on websites to advertise another agents listing and then send the website visitor to that agent?

There are two sides to every transaction, the buyer side and the seller side. If the listing agent has both the buyer and seller (dual agency), then the dual agent can not provide the same level of disclosure to either the buyer or the seller. It is beneficial to the buying public to be able to be represented by their own buyer agent if they so choose.

There are many tech savvy agents that seek to work with buyers. If the listing agents in their greed to take both sides of the transaction and possibly to the detriment of their clients want to shut off data to the IDX sites, that is their perogative.

If listing agents want to shut out buyer agents and return to the days when only the seller’s interests were represented, then that needs to be disclosed to their sellers, the properties Opted Out of the IDX, and then they can sit back and wait and wonder why their listing agent is not getting any buyers from the internet shopping public.

If as suggested, “the listing agents contact information is to be included on the IDX”, rest assured I will not be providing free advertising services for those agents or their clients.

I know the Bruce Clay organization is cognizant that well performing websites require quality, relevance, and investmet of large amounts of time in addition to money.

Part of the contingency that is opposed to the indexing is the same group that goes to listing appointments saying “I have a website”, yet fail to disclose that there is no traffic to it and that it is not even in the Google index. Heck, you can’t even find the agent by name in Google search, much less the property.

Again, they should feel free to drop the listings from the IDX. These people will be the same ones that post almost no information about the property and the “MLS required” single photo anyway.

These will be the same agents that put out flyers with no prices so you have to call them to see that the home you were interested in is not in your price range (or not listed anymore). The same ones that forget to mark their listings Pending and Sold so they keep getting exposure to themselves. Also, the same ones that buy ads in the Real Estate Book with no address for the property, again so that you have to call them to find out that the cute house is 60 miles away from where you work!

Listen folks, take the IDX indexing away, there are still going to be greedy and lazy individuals that are going to sell property owners on their services and the properties will not show up in searches because they are a)too lazy b)too dumb c)too greedy to put up they very information that might bring them site visitors. Information such as prices, addresses, multiple photos, adequate property details.

Believe me, you just don’t know how many times buyer agents with successful websites are asked for pictures of something that might be important to a home buyer – yes like kitchens, living rooms, master bedrooms – and then ask the listing agent to provide them and they are never forthcoming making the buyer agent look inept when it is the listing agent that has done his seller a disservice. The information on their websites is as basic as the information in their old newspaper and Real Estate Book adverts and will never be indexed because there are no keyword phrases present. They think a pretty graphic rich webpage will get them pasted all over Google, yet there is not one word about the geographical area that they serve.

Go figure!

I say encourage them to opt out of IDX!