add baidu spider to list of spiders/robots phpbb3

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Suggested Hosts
Post Reply
bigone5500
Registered User
Posts: 84
Joined: Sat Apr 02, 2005 7:48 pm

add baidu spider to list of spiders/robots phpbb3

Post by bigone5500 » Sat Nov 28, 2009 6:17 pm

I want to add the Baidu Spider to the list of spiders/robots and be able to block it?

User avatar
Sam
Former Team Member
Posts: 2082
Joined: Thu Jan 22, 2009 3:59 am
Location: Sacramento, CA
Name: Sam Thompson
Contact:

Re: add baidu spider to list of spiders/robots phpbb3

Post by Sam » Sat Nov 28, 2009 6:44 pm

Why would you want to block it? IIRC, it is already on the list.
You can't block with phpBB, but you can with .htaccess.

Pony99CA
Registered User
Posts: 4783
Joined: Thu Sep 30, 2004 3:13 pm
Location: Hollister, CA
Name: Steve
Contact:

Re: add baidu spider to list of spiders/robots phpbb3

Post by Pony99CA » Wed Dec 02, 2009 10:55 am

SyntaxError90 wrote:Why would you want to block it?
Possibly because it's primarily used by Chinese people and the admin has no interest in catering to them because he runs a local board in some other country.
SyntaxError90 wrote:IIRC, it is already on the list.
It is on my board (it's listed first, even).
SyntaxError90 wrote:You can't block with phpBB, but you can with .htaccess.
Well, you sort of can. You could search for the user Baidu [Spider] and assign the "No Access" role to Baidu for all the forums. That should effectively prevent Baidu from crawling your forums, although it could still visit the Index and any non-forum links from there.

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

User avatar
Sam
Former Team Member
Posts: 2082
Joined: Thu Jan 22, 2009 3:59 am
Location: Sacramento, CA
Name: Sam Thompson
Contact:

Re: add baidu spider to list of spiders/robots phpbb3

Post by Sam » Thu Dec 03, 2009 8:34 am

Pony99CA wrote:Well, you sort of can. You could search for the user Baidu [Spider] and assign the "No Access" role to Baidu for all the forums. That should effectively prevent Baidu from crawling your forums, although it could still visit the Index and any non-forum links from there.
That approach should work, I just generally don't like doing individual permissions, you can easily loose track of who has what permission.

Peter77sx
Registered User
Posts: 3253
Joined: Wed Nov 09, 2005 2:51 pm

Re: add baidu spider to list of spiders/robots phpbb3

Post by Peter77sx » Tue Dec 08, 2009 3:02 am

For your .htaccess file
deny from 123.125.66 *

blocks about 15+ of it's search engines IP's
Which happen to be:
123.125.66.121
123.125.66.123
123.125.66.130
123.125.66.133
123.125.66.57
123.125.66.59
123.125.66.63
123.125.66.66
123.125.66.70
123.125.66.72
123.125.66.74
123.125.66.79
123.125.66.81
123.125.66.90
123.125.66.94
123.125.66.96
123.125.66.97

Post Reply

Return to “phpBB Discussion”