Translate
Sub Menu
Links Menu
Online Users

In total there are 85 users online :: 2 registered, 0 hidden and 83 guests

Most users ever online was 372 on Thu May 09, 2019 3:50 am

Registered users: Exabot [Bot], Google [Bot] based on users active over the past 60 minutes

Last Online
In order to view the online list you have to be registered and logged in.



We are a free and open
community, all are welcome.

Click here to Register

how can i control bot info retrieved

General discussion of anything. Discuss a topic in and out of IntegraMOD

Moderator: Integra Moderator

how can i control bot info retrieved

PostAuthor: woo » Wed May 28, 2008 9:56 am

as much as i want bots to crawl my site for advertising on search engines, is there anyway we can control what information or pages/forums they can access on the site.
whilst checking up on what info the search engines were picking up, i noticed they were picking up on logins ect and stuff irrelevant to what readers want to see.
any help appreciated
thanks x
Last edited by woo on Wed Dec 31, 1969 5:00 pm, edited 1 time in total.

woo
Members
Members
 
Posts: 56
Joined: Mon Mar 24, 2008 3:01 pm

Re: how can i control bot info retrieved

PostAuthor: CaNNon » Wed May 28, 2008 10:21 am

google robots.txt <img>
Last edited by CaNNon on Wed Dec 31, 1969 5:00 pm, edited 1 time in total.
Image
Image
User avatar
CaNNon
Sr Integra Member
Sr Integra Member
 
Posts: 750
Joined: Thu Apr 19, 2007 12:15 pm

PostAuthor: woo » Wed May 28, 2008 10:44 am

il check that out, thanks caNNon x
Last edited by woo on Wed Dec 31, 1969 5:00 pm, edited 1 time in total.

woo
Members
Members
 
Posts: 56
Joined: Mon Mar 24, 2008 3:01 pm


Return to General Discussion

Who is online

Registered users: Exabot [Bot], Google [Bot]

cron