As search engines improve their ability to find and index content, site owners enjoy the benefits of seeing even dynamic parts of their websites getting found and ranked. Does this mean you can relax? Not at all.

Mike Moran noted that before 2005, no search engine could index dynamic content. This meant site owners with dynamic content they wanted to see indexed needed to make various adjustments to ensure that content would be visible. For example, a page with a Flash video might include a text version of the video's content below it for the search engines to find. Now, though, for Flash videos that contain text, “Google can index a lot more of it than ever before,” Moran observed.

Not only that, but “dynamic content generated from databases is indexed better than it once was, so it is less important to hide dynamic URLs than in years past,” Moran continued. And back in November, Google's Matt Cutts stated that the search giant is now indexing Facebook comments. What all this amounts to is that the Googlebot spider understands more JavaScript than it ever did before.

While this is good, it's still not ideal. As Moran points out, the Googlebot may be better at spotting and indexing dynamic content, but that doesn't mean it picks up on all of it – and “if it omits any of your content, you're losing something.” If you want to make sure the spiders find and index ALL of your content, you need to keep using techniques that avoid dynamic content.

You should also keep in mind that Google may be the biggest search engine around, but it's hardly the only game in town. Moran observed that Bing handles nearly a third of the searches performed in the U.S., and plenty of other search engines top the list in other countries. Baidu, for example, is still the dominant search engine in China. The point is, most of these search engines can't handle dynamic content, which means their users won't be able to find your dynamic content. You're effectively hiding your content from them if you don't convert your content in some way. 

Moran revealed that your losses, when not converting your dynamic content, may end up being worse than you think. “When the spider fails to identify dynamic content, you might lose a lot more than a few words on a page. If that content contains links, the spider might not see whole pages on your site, and whatever pages THEY link to,” he explained.

So yes, it's true that search engine spiders are smarter now than they used to be, but they still won't spot all of your dynamic content. Rather than believing they'll catch everything you've put up, it's safer to operate on the assumption that they'll miss your dynamic content. That means continuing to use the appropriate techniques to make your dynamic content visible to spiders that can only see static content. Someday, we'll no longer need to do this – but that day isn't today, or this month, or probably even this year. So make sure your content is visible to all of the search engines. Good luck!

DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware.

More Search Engine Tricks Articles
More By terri