Deliver Your Web Site From Evil (Part 2)
(Page 1 of 7 )
Here are some tips to protect your content from being ripped off.Other sites can easily outrank you in SERPs for your own content. This can happen if they have higher PageRank than you, or get spidered by Google first.
1. Set up a bad-bot-banning script on your site.
There's one offered in this forum: http://www.webmasterworld.com/forum88/10425.htm
This is to bar leechers. You use 'robots.txt' to disallow a subdirectory. After two weeks, you set up the bad-bot script. The idea is that a bot that accesses a subdirectory, which you have disallowed, is a bad bot. It's just hoovering up your data. It's not from a legit search engine. So it's probably a competitor, or a leech.
The script rewrites your .htaccess file to forbid the bot access to your site altogether. Webmasters are keen to ban some bots to save bandwidth also.
Furthermore, you can ban individual downloader softwares by their HTTP_USER_AGENT environmental variable:
More Website Content Articles
More By Jase Dow