Page 2 of 8

Re: Bots Install Script

Posted: Sat Sep 20, 2008 10:58 pm
by lockylew
I've added all the files and went to www.mydomain.com/bots.php and got the following error:

Code: Select all

SQL ERROR [ mysql4 ]

Incorrect string value: '\xC7\x83 spi...' for column 'username_clean' at row 1 [1366]

SQL

INSERT INTO phpbb_users (username, username_clean, user_password, user_pass_convert, user_email, user_email_hash, group_id, user_type, user_permissions, user_timezone, user_dateformat, user_lang, user_style, user_allow_pm, user_actkey, user_ip, user_regdate, user_passchg, user_options, user_inactive_reason, user_inactive_time, user_lastmark, user_lastvisit, user_lastpost_time, user_lastpage, user_posts, user_dst, user_colour, user_occ, user_interests, user_avatar, user_avatar_type, user_avatar_width, user_avatar_height, user_new_privmsg, user_unread_privmsg, user_last_privmsg, user_message_rules, user_full_folder, user_emailtime, user_notify, user_notify_pm, user_notify_type, user_allow_viewonline, user_allow_viewemail, user_allow_massemail, user_sig, user_sig_bbcode_uid, user_sig_bbcode_bitfield, user_form_salt) VALUES ('Yahoo! SpiderMan', 'yahooǃ spiderman', '', 0, '', '00', 6, 2, '', 0, 'D M d, Y g:i a', 'en', 1, 1, '', '', 1221951401, 1221951401, 895, 0, 0, 1221951401, 0, 0, '', 0, 0, '9E8DA7', '', '', '', 0, 0, 0, 0, 0, 0, 0, -3, 0, 0, 1, 0, 1, 1, 0, '', '', '', 'af088d86714f083c')

BACKTRACE

FILE: includes/db/mysql.php
LINE: 158
CALL: dbal->sql_error()

FILE: includes/functions_user.php
LINE: 242
CALL: dbal_mysql->sql_query()

FILE: bots.php
LINE: 191
CALL: user_add()

FILE: bots.php
LINE: 207
CALL: add_bots2()
It isn't that duplicate entry problem so i'm confussed.

any help would be greatly appreciated.

Lew

Re: Bots Install Script

Posted: Sun Sep 21, 2008 12:09 am
by T50
hmm are you using mysql 4?
It seems the issue is with the ! on the yahoo bots. remove the ! from them in the file and comment out the first 6 bots on the list (the ones before the first yahoo bot)

Re: Bots Install Script

Posted: Sun Sep 21, 2008 1:29 am
by junktemp
Hi,

I am new to PHP; and a new phpBB forum owner. I have and regularly use phpBB powered boards.

Just by glancing at the SQL code - I could see the passwords are set to ' '.

Correct me if I'm wrong, but isn't this a security concern? Are BOTS a read only, or do they have post rights? If these 100+ bots are allowed to post, wouldn't that just open up the board to spam?

Are the BOTs group typically prevented from posting?

Re: Bots Install Script

Posted: Sun Sep 21, 2008 1:41 am
by T50
like i said before this mod doesn't mean that these bots will visit your forum

also bots don't have passwords, they should only have read only access to parts of your forum that you want to be indexed. but the admin can give them whatever access they want.

Re: Bots Install Script

Posted: Sun Sep 21, 2008 3:50 am
by ∞House
junktemp wrote:Hi,

I am new to PHP; and a new phpBB forum owner. I have and regularly use phpBB powered boards.

Just by glancing at the SQL code - I could see the passwords are set to ' '.

Correct me if I'm wrong, but isn't this a security concern? Are BOTS a read only, or do they have post rights? If these 100+ bots are allowed to post, wouldn't that just open up the board to spam?

Are the BOTs group typically prevented from posting?
The Bots Group is one of the Core Groups in phpBB3. It cannot be deleted because it was hard-coded into the software. On top of that, this particular group is there for a very specific purpose: the proper handling of webcrawler/spider/bots' sessions. So when these visitors (like Googlebot) do visit your forum, they will be recognized by their User Agent. - A user-agent is not something in phpBB, but rather a string provided to the webserver by the crawler/bot, or even the web browser that you're using at the moment. (My personal user-agent is "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 BETA!".)

What this MOD does is add a list of user-agents which are known to be used by these bots. So when they visit, they are simply identified and are handled as per the permissions set for the Bot Group. By default, there are no security issues what so ever with this. But you can theoretically give them more access to your board then you would normally want to under any foreseeable circumstance. - Such is with the Guest Group. You could give Guests ACP rights, for example. (Unless this has been denied at the file-level. I haven't personally tried, though it's a good example.)

So it's Ok. :geek:

Re: Bots Install Script

Posted: Sun Sep 21, 2008 4:42 pm
by lockylew
T50 wrote:hmm are you using mysql 4?
It seems the issue is with the ! on the yahoo bots. remove the ! from them in the file and comment out the first 6 bots on the list (the ones before the first yahoo bot)
That worked, Thanks for your help

and i'm using MySQL version 5.0.45.

last question, when i was trying to install this last night i deleted all of the original bots that came as standard with phpbb3 because along with the varieable string error i was getting the dublicate entry error, do you know how i can re-add all the standard bots?

thanks for your help

Lew

Re: Bots Install Script

Posted: Sun Sep 21, 2008 5:57 pm
by T50
yes, I'll have to find the code, for it, but I know it is possible as I did it before.

Re: Bots Install Script

Posted: Sun Sep 21, 2008 8:25 pm
by lockylew
T50 wrote:yes, I'll have to find the code, for it, but I know it is possible as I did it before.
ok, thanks very much for your help.

Lew

Re: Bots Install Script

Posted: Sun Sep 21, 2008 8:34 pm
by T50
replace the list of bots with this

Code: Select all

$bots = array(
		'AdsBot [Google]'			=> array('AdsBot-Google', ''),
		'Alexa [Bot]'				=> array('ia_archiver', ''),
		'Alta Vista [Bot]'			=> array('Scooter/', ''),
		'Ask Jeeves [Bot]'			=> array('Ask Jeeves', ''),
		'Baidu [Spider]'			=> array('Baiduspider+(', ''),
		'Exabot [Bot]'				=> array('Exabot/', ''),
		'FAST Enterprise [Crawler]'	=> array('FAST Enterprise Crawler', ''),
		'FAST WebCrawler [Crawler]'	=> array('FAST-WebCrawler/', ''),
		'Francis [Bot]'				=> array('http://www.neomo.de/', ''),
		'Gigabot [Bot]'				=> array('Gigabot/', ''),
		'Google Adsense [Bot]'		=> array('Mediapartners-Google', ''),
		'Google Desktop'			=> array('Google Desktop', ''),
		'Google Feedfetcher'		=> array('Feedfetcher-Google', ''),
		'Google [Bot]'				=> array('Googlebot', ''),
		'Heise IT-Markt [Crawler]'	=> array('heise-IT-Markt-Crawler', ''),
		'Heritrix [Crawler]'		=> array('heritrix/1.', ''),
		'IBM Research [Bot]'		=> array('ibm.com/cs/crawler', ''),
		'ICCrawler - ICjobs'		=> array('ICCrawler - ICjobs', ''),
		'ichiro [Crawler]'			=> array('ichiro/2', ''),
		'Majestic-12 [Bot]'			=> array('MJ12bot/', ''),
		'Metager [Bot]'				=> array('MetagerBot/', ''),
		'MSN NewsBlogs'				=> array('msnbot-NewsBlogs/', ''),
		'MSN [Bot]'					=> array('msnbot/', ''),
		'MSNbot Media'				=> array('msnbot-media/', ''),
		'NG-Search [Bot]'			=> array('NG-Search/', ''),
		'Nutch [Bot]'				=> array('http://lucene.apache.org/nutch/', ''),
		'Nutch/CVS [Bot]'			=> array('NutchCVS/', ''),
		'OmniExplorer [Bot]'		=> array('OmniExplorer_Bot/', ''),
		'Online link [Validator]'	=> array('online link validator', ''),
		'psbot [Picsearch]'			=> array('psbot/0', ''),
		'Seekport [Bot]'			=> array('Seekbot/', ''),
		'Sensis [Crawler]'			=> array('Sensis Web Crawler', ''),
		'SEO Crawler'				=> array('SEO search Crawler/', ''),
		'Seoma [Crawler]'			=> array('Seoma [SEO Crawler]', ''),
		'SEOSearch [Crawler]'		=> array('SEOsearch/', ''),
		'Snappy [Bot]'				=> array('Snappy/1.1 ( http://www.urltrends.com/ )', ''),
		'Steeler [Crawler]'			=> array('http://www.tkl.iis.u-tokyo.ac.jp/~crawler/', ''),
		'Synoo [Bot]'				=> array('SynooBot/', ''),
		'Telekom [Bot]'				=> array('crawleradmin.t-info@telekom.de', ''),
		'TurnitinBot [Bot]'			=> array('TurnitinBot/', ''),
		'Voyager [Bot]'				=> array('voyager/1.0', ''),
		'W3 [Sitesearch]'			=> array('W3 SiteSearch Crawler', ''),
		'W3C [Linkcheck]'			=> array('W3C-checklink/', ''),
		'W3C [Validator]'			=> array('W3C_*Validator', ''),
		'WiseNut [Bot]'				=> array('http://www.WISEnutbot.com', ''),
		'YaCy [Bot]'				=> array('yacybot', ''),
		'Yahoo MMCrawler [Bot]'		=> array('Yahoo-MMCrawler/', ''),
		'Yahoo Slurp [Bot]'			=> array('Yahoo! DE Slurp', ''),
		'Yahoo [Bot]'				=> array('Yahoo! Slurp', ''),
		'YahooSeeker [Bot]'			=> array('YahooSeeker/', ''),
	);

Re: Bots Install Script

Posted: Mon Sep 22, 2008 7:36 am
by cacc
Quick and easy install! Great work on a great mod. Thanks.

Re: Bots Install Script

Posted: Mon Sep 22, 2008 8:38 pm
by lockylew
T50 wrote:replace the list of bots with this

Code: Select all

$bots = array(
		'AdsBot [Google]'			=> array('AdsBot-Google', ''),
		'Alexa [Bot]'				=> array('ia_archiver', ''),
		'Alta Vista [Bot]'			=> array('Scooter/', ''),
		'Ask Jeeves [Bot]'			=> array('Ask Jeeves', ''),
		'Baidu [Spider]'			=> array('Baiduspider+(', ''),
		'Exabot [Bot]'				=> array('Exabot/', ''),
		'FAST Enterprise [Crawler]'	=> array('FAST Enterprise Crawler', ''),
		'FAST WebCrawler [Crawler]'	=> array('FAST-WebCrawler/', ''),
		'Francis [Bot]'				=> array('http://www.neomo.de/', ''),
		'Gigabot [Bot]'				=> array('Gigabot/', ''),
		'Google Adsense [Bot]'		=> array('Mediapartners-Google', ''),
		'Google Desktop'			=> array('Google Desktop', ''),
		'Google Feedfetcher'		=> array('Feedfetcher-Google', ''),
		'Google [Bot]'				=> array('Googlebot', ''),
		'Heise IT-Markt [Crawler]'	=> array('heise-IT-Markt-Crawler', ''),
		'Heritrix [Crawler]'		=> array('heritrix/1.', ''),
		'IBM Research [Bot]'		=> array('ibm.com/cs/crawler', ''),
		'ICCrawler - ICjobs'		=> array('ICCrawler - ICjobs', ''),
		'ichiro [Crawler]'			=> array('ichiro/2', ''),
		'Majestic-12 [Bot]'			=> array('MJ12bot/', ''),
		'Metager [Bot]'				=> array('MetagerBot/', ''),
		'MSN NewsBlogs'				=> array('msnbot-NewsBlogs/', ''),
		'MSN [Bot]'					=> array('msnbot/', ''),
		'MSNbot Media'				=> array('msnbot-media/', ''),
		'NG-Search [Bot]'			=> array('NG-Search/', ''),
		'Nutch [Bot]'				=> array('http://lucene.apache.org/nutch/', ''),
		'Nutch/CVS [Bot]'			=> array('NutchCVS/', ''),
		'OmniExplorer [Bot]'		=> array('OmniExplorer_Bot/', ''),
		'Online link [Validator]'	=> array('online link validator', ''),
		'psbot [Picsearch]'			=> array('psbot/0', ''),
		'Seekport [Bot]'			=> array('Seekbot/', ''),
		'Sensis [Crawler]'			=> array('Sensis Web Crawler', ''),
		'SEO Crawler'				=> array('SEO search Crawler/', ''),
		'Seoma [Crawler]'			=> array('Seoma [SEO Crawler]', ''),
		'SEOSearch [Crawler]'		=> array('SEOsearch/', ''),
		'Snappy [Bot]'				=> array('Snappy/1.1 ( http://www.urltrends.com/ )', ''),
		'Steeler [Crawler]'			=> array('http://www.tkl.iis.u-tokyo.ac.jp/~crawler/', ''),
		'Synoo [Bot]'				=> array('SynooBot/', ''),
		'Telekom [Bot]'				=> array('crawleradmin.t-info@telekom.de', ''),
		'TurnitinBot [Bot]'			=> array('TurnitinBot/', ''),
		'Voyager [Bot]'				=> array('voyager/1.0', ''),
		'W3 [Sitesearch]'			=> array('W3 SiteSearch Crawler', ''),
		'W3C [Linkcheck]'			=> array('W3C-checklink/', ''),
		'W3C [Validator]'			=> array('W3C_*Validator', ''),
		'WiseNut [Bot]'				=> array('http://www.WISEnutbot.com', ''),
		'YaCy [Bot]'				=> array('yacybot', ''),
		'Yahoo MMCrawler [Bot]'		=> array('Yahoo-MMCrawler/', ''),
		'Yahoo Slurp [Bot]'			=> array('Yahoo! DE Slurp', ''),
		'Yahoo [Bot]'				=> array('Yahoo! Slurp', ''),
		'YahooSeeker [Bot]'			=> array('YahooSeeker/', ''),
	);
Worked Perfectly.

Thanks for all your help.

Lew

Re: Bots Install Script

Posted: Fri Sep 26, 2008 1:31 am
by kramtronix
What is the benefit to having the bots?

Re: Bots Install Script

Posted: Fri Sep 26, 2008 2:21 am
by T50
kramtronix wrote:What is the benefit to having the bots?
I answered that like 2-3 times already

Re: Bots Install Script

Posted: Fri Sep 26, 2008 2:42 am
by ∞House
T50 wrote:
kramtronix wrote:What is the benefit to having the bots?
I answered that like 2-3 times already

Dee-dee-deeeee~ Image

Tim, you may want to make use of your reservation and stick that information in. :twisted:

Re: Bots Install Script

Posted: Fri Sep 26, 2008 2:54 am
by T50
Image
Totally forgot I had that post!