Translate
Sub Menu
Links Menu
Online Users

In total there are 82 users online :: 1 registered, 0 hidden and 81 guests

Most users ever online was 366 on Wed Dec 27, 2017 1:32 pm

Registered users: Google [Bot] based on users active over the past 60 minutes

Last Online
In order to view the online list you have to be registered and logged in.



We are a free and open
community, all are welcome.

Click here to Register

how can i control bot info retrieved

General discussion of anything. Discuss a topic in and out of IntegraMOD

Moderator: Integra Moderator

how can i control bot info retrieved

PostAuthor: woo » Wed May 28, 2008 9:56 am

as much as i want bots to crawl my site for advertising on search engines, is there anyway we can control what information or pages/forums they can access on the site.
whilst checking up on what info the search engines were picking up, i noticed they were picking up on logins ect and stuff irrelevant to what readers want to see.
any help appreciated
thanks x
Last edited by woo on Wed Dec 31, 1969 5:00 pm, edited 1 time in total.

woo
Members
Members
 
Posts: 56
Joined: Mon Mar 24, 2008 3:01 pm

Re: how can i control bot info retrieved

PostAuthor: CaNNon » Wed May 28, 2008 10:21 am

google robots.txt <img>
Last edited by CaNNon on Wed Dec 31, 1969 5:00 pm, edited 1 time in total.
Image
Image
User avatar
CaNNon
Sr Integra Member
Sr Integra Member
 
Posts: 750
Joined: Thu Apr 19, 2007 12:15 pm

PostAuthor: woo » Wed May 28, 2008 10:44 am

il check that out, thanks caNNon x
Last edited by woo on Wed Dec 31, 1969 5:00 pm, edited 1 time in total.

woo
Members
Members
 
Posts: 56
Joined: Mon Mar 24, 2008 3:01 pm


Return to General Discussion

Who is online

Registered users: Google [Bot]

cron