Disallowing bots (was Re: Man, they'll try anything to hack your system...)

[Larry Cook]

Fred wrote:
> I've been debating if I should 
> disallow all the other bots since they do put quite a load on my servers. 

My understanding is that you do this with robots.txt which the bots and 
spiders read.  So it's basically an honor system that keeps out the good ones. 
  How do you keep out the bad ones, the ones that ignore robots.txt?

Larry