Home Articles FAQs XREF Games Software Instant Books BBS About FOLDOC RFCs Feedback Sitemap
irt.Org

BBS: Re: robots.txt - September 14, 1998 at 20:22:28

You are here: irt.org | BBS | Re: robots.txt [This BBS is closed]

Posted by Don Rowlett on September 14, 1998 at 20:22:28:

In Reply to: robots.txt posted by Michael B on September 14, 1998 at 16:16:23:

: I'd like certain files on a site to be excluded by search engines. I realise this involves a robots.txt file - can someone please teach me how to create one and where it should go?
: Thanks,

: Michael

Try this URL: http://info.webcrawler.com/mak/projects/robots/faq.html

Follow-ups:

You are here: irt.org | BBS | Re: robots.txt [This BBS is closed]

©2018 Martin Webb