robots.txt problem with google?
    Bill Ricker 
    bill.n1vux at gmail.com
       
    Thu Dec 14 09:30:42 EST 2006
    
    
  
> > My syslog tells me I have currentlt 303 hits from googlebot
> Are any of them for robots.txt?  Just curious.
Good question! The follow-up is, is robots.txt 444 or 600 ?
Maybe you can see it and Google can't.
Or maybe it's the other robots.txt.
If this is the same host as your email, which it probably is since
there is a ./album/ there, you may be surprised at what you see at
   http://www.syslang.net/robots.txt
- - - -
User-agent: *
-EOF-
No Disallow made it to "production".
You should have just gotten hits from pool-72.blah-blah.bos*.verizon.net ...
 --
Bill
n1vux at arrl.net bill.n1vux at gmail.com
    
    
More information about the gnhlug-discuss
mailing list