robots.txt problem with google?

Bill Ricker bill.n1vux at gmail.com
Thu Dec 14 09:30:42 EST 2006


> > My syslog tells me I have currentlt 303 hits from googlebot

> Are any of them for robots.txt?  Just curious.

Good question! The follow-up is, is robots.txt 444 or 600 ?
Maybe you can see it and Google can't.

Or maybe it's the other robots.txt.

If this is the same host as your email, which it probably is since
there is a ./album/ there, you may be surprised at what you see at
   http://www.syslang.net/robots.txt
- - - -
User-agent: *
-EOF-

No Disallow made it to "production".

You should have just gotten hits from pool-72.blah-blah.bos*.verizon.net ...

 --
Bill
n1vux at arrl.net bill.n1vux at gmail.com


More information about the gnhlug-discuss mailing list