robots.txt problem with google?
Steven W. Orr
steveo at syslang.net
Wed Dec 13 15:04:16 EST 2006
On Wednesday, Dec 13th 2006 at 14:41 -0500, quoth Ben Scott:
=>On 12/13/06, Steven W. Orr <steveo at syslang.net> wrote:
=>> I have a big dir full of pictures and my robots.txt file excludes it. So
=>> now I just noticed that google is crawling through it. Anyone know what I
=>> shoul do?
=>
=>Start here:
=>
=>http://www.google.com/support/webmasters/bin/answer.py?answer=35301&topic=8459
=>
=>-- Ben
Ok. Here's my robots.txt
User-agent: *
Disallow: /album/
My syslog tells me I have currentlt 303 hits from googlebot
(http://www.google.com/bot.html)
since Monday.
The link above has nothing to do with them violating the robots.txt file.
--
Time flies like the wind. Fruit flies like a banana. Stranger things have .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net
More information about the gnhlug-discuss
mailing list