Sub Menu
Links Menu
Online Users

In total there are 317 users online :: 2 registered, 0 hidden and 315 guests

Most users ever online was 1091 on Wed Aug 16, 2023 5:27 pm

Registered users: Bing [Bot], Google [Bot] based on users active over the past 60 minutes

how can i control bot info retrieved

General discussion of anything. Discuss a topic in and out of IntegraMOD

Moderator: Integra Moderator

how can i control bot info retrieved

PostAuthor: woo » Wed May 28, 2008 8:56 am

as much as i want bots to crawl my site for advertising on search engines, is there anyway we can control what information or pages/forums they can access on the site.
whilst checking up on what info the search engines were picking up, i noticed they were picking up on logins ect and stuff irrelevant to what readers want to see.
any help appreciated
thanks x
Last edited by woo on Wed Dec 31, 1969 4:00 pm, edited 1 time in total.

woo
Members
Members
 
Posts: 56
Likes: 0 post
Liked in: 0 post
Joined: Mon Mar 24, 2008 2:01 pm
Cash on hand: 0.00

Re: how can i control bot info retrieved

PostAuthor: CaNNon » Wed May 28, 2008 9:21 am

google robots.txt <img>
Last edited by CaNNon on Wed Dec 31, 1969 4:00 pm, edited 1 time in total.
Image
Image
User avatar
CaNNon
Sr Integra Member
Sr Integra Member
 
Posts: 750
Likes: 0 post
Liked in: 0 post
Joined: Thu Apr 19, 2007 11:15 am
Cash on hand: 0.00

PostAuthor: woo » Wed May 28, 2008 9:44 am

il check that out, thanks caNNon x
Last edited by woo on Wed Dec 31, 1969 4:00 pm, edited 1 time in total.

woo
Members
Members
 
Posts: 56
Likes: 0 post
Liked in: 0 post
Joined: Mon Mar 24, 2008 2:01 pm
Cash on hand: 0.00


Return to General Discussion

Who is online

Registered users: Bing [Bot], Google [Bot]

cron