Engines Banning ODP Clones Or Penalizing Duplicate Content?
Barry at Search Engine Roundtable is pointing to an interesting thread that broke out on WebmasterWorld after one user conducted a ‘study’ that he says proves that sites that use the ODP database are often banned by the search engines.
Sparked by his own site’s banning, this member took an alphabetical list of sites who use the open directory’s data and checked their status in Google, Yahoo and MSN by using a site search command. According to the study, approximately 50 percent were banned by at least one search engine, with 37 percent being banned in Google alone. The site owner said it did not seem to matter if the site was an exact clone, or if it was just using a small amount of ODP data. The punishment was the same: banned.
The site owner defined “banned” as being completely removed from the indexed, delisted, backlisted or censored. He said banned did not include sites that returned only a small number of pages or pages he felt the search engines simply had just not indexed.
The owner says his concern has more to do with ‘fundamental issues of fairness and deception’ that with his own site’s banning.
“The whole idea and promise of the "Open" Directory Project was that the data was to be freely available for use by any web site. This is effectively a fraud if only Google and their friends can use "Open" Directory data without risking being banned by Google.”
The ‘data’ this member brings is slightly suspect (and just a bit biased). And without more information I’m not sure what it really proves. Did the banned sites have anything else in common besides the shared ODP data? Can the banning really be linked to the shared data? Are the sites primarily composed of ODP data (straight clones) or do they also have rich, original content? Is this really a banning or just an example of sites being penalized for duplicate content issues? I don’t think the results found in this study can say either way.
Users who want to weigh in on the discussion should head over to the WebmasterWorld forums.