Not sure if this belongs here...about googlebot

epona45 · Post by **epona45** » Sat May 27, 2006 2:51 am

I just wanted to know if there's a way to keep a googlebot from crawling my forum. I'm not a computer whiz by any means so if you know how I can do this can you explain it in simple terms?

I read somewhere about robot.txt but I don't know exactly what this is. Also, I guess a better question would be, is it really a good idea to somehow block such a bot?

Thanks for any help

The KaN · Post by **The KaN** » Sat May 27, 2006 2:55 am

Here is all of Google's info on the topic.

Google Crawler Support

CTCNetwork · Post by **CTCNetwork** » Sat May 27, 2006 2:56 am

Hi,

Yes, create a robots.txt file and upload it to your hosts server in the phpBB folder or the one above it.

More info on robots.txt can be found Here...

Hope this helps,

Des. . .

ferrethouse2004 · Post by **ferrethouse2004** » Sat May 27, 2006 2:56 am

I just banned ask jeeves. 9 out of every 10 hits to my site was from jeeves bot and I get no real traffic from them. 54 gigs of bandwidth i've used this month and about 30 gigs of that was jeeves bot. keep googlebot and get rid of jeeves.

The KaN · Post by **The KaN** » Sat May 27, 2006 3:01 am

ferrethouse2004 wrote: I just banned ask jeeves. 9 out of every 10 hits to my site was from jeeves bot and I get no real traffic from them. 54 gigs of bandwidth i've used this month and about 30 gigs of that was jeeves bot. keep googlebot and get rid of jeeves.

30 GBs... WOW... Yea...

epona45 · Post by **epona45** » Sat May 27, 2006 3:06 am

Thanks everyone, I'm going to give this a try...wish me luck

tanfwc · Post by **tanfwc** » Sat May 27, 2006 3:07 am

This might make Google love your site.

http://www.gotaxe.com/phpbb-sitemap.php

advertisements