Page 1 of 1

Bots [Solved]

PostPosted: Wed Jan 03, 2007 4:24 pm
Author: DjPorkchop
Ok I have searched not this site BUT the other one too and turned up no results. HOW do I stop bots from visitng my page? Is there a way to do this? Ive been watching one sit on my site now for 4 days! I dont need bots. Dont want them have nouse for my page being indexed by anyone.

Is it just as wise to ban google, yahoo, jeeves , etc, etc? I run a website with VERY limited users and personally dont give a rats fat Arse about rankings at all. All that amounts to is status and Im not concerened with my websites status on the net.

And bots adding, ignoring and all that is just getting old. I got lists of bots that i have to ignore 1 at a time. YUCK!

Help!!! Please???? <img>

Re: Bots

PostPosted: Thu Jan 04, 2007 1:57 am
Author: Teelk
If you have no need for bots you can ban them using ACP>Security>Special and entering their User Agent name in the "Block User Agents" section. You'll have to google a list of User Agent names for search engines, I don't know what they are specifically.

Re: Bots

PostPosted: Thu Jan 04, 2007 5:55 am
Author: DjPorkchop
Ok I can do that Thx Teelk.

Search Engine: User-Agent
AltaVista: Scooter
Infoseek: Infoseek
Hotbot: Slurp
AOL: Slurp
Google: Googlebot
Overture: Slurp:
Lycos: Lycos
MSN: Slurp
Netscape: Googlebot
NorthernLight: Gulliver
WebCrawler: ArchitextSpider
Iwon: Slurp
Fast: Fast
DirectHit: Grabber
Yahoo Web Pages: Googlebot
Looksmart Web Pages: Slurp

Re: Bots [Solved]

PostPosted: Tue Jan 09, 2007 2:42 pm
Author: DjPorkchop
Just an FYI I found a great link that anyone wanting to exclude certain bots or all of them from your site. ALSO It has tutorials for making bot.txt files. What these do is allow you to create a text file for bots. What it essentially does is tell the bots which pages they can not index. By this I mean if you have certain pages you do not want them to index you can now take care of it through a bot.txt file much in the same manner as using a redirect in a .hta file. It's quite simple and bots have stopped visitng my site alltogether now. Which is what i was after to begin with.

Here is the link. [url=http]Robots.txt[/url]