OT: Spamming Log Files?

[Jason Stephenson]

bscott at ntisys.com wrote:
> On Wed, 30 Apr 2003, at 10:41pm, jason at sigio.com wrote:
>>I didn't go to the web site, but this doesn't appear to be the typical 
>>entry for a bot or a site checker ...
> 
> 
>   I did go to the web page, and that does appear to be what it is.  The page
> in question simply explains what is going, and how to tell them not to
> spider your site (using the "robots.txt" file).  Based solely on what is on
> that page, they appear to be pretty legit.

Ah-ha! But, they don't look for a robots.txt file. I have entries in my 
log file for the spiders that do look for robots.txt, and they don't 
have one. They appear to be saying one thing and doing another.

I'm not saying that they aren't legit, just that they appear to be 
advertising themselves in my log file.

Like I said, it doesn't really harm me. I was just wondering what others 
thought and if we have a new trend in www advertising starting. 
Actually, I've suspected this of another site/service in the recent past.

I really don't care if they do it. I just think it's an interesting 
tactic. Well, I guess it bothers me enough to mail the list about it.