[CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Converting from other board software? Good decision! Need help? Have a question about a convertor? Wish to offer a convertor package? Post here.
Anti-Spam Guide
a1topdog
Registered User
Posts: 32
Joined: Sat Nov 26, 2005 10:49 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by a1topdog »

nneonneo wrote:@a1topdog: That error means that user emails cannot be converted. However, everything else should still be OK.

You can try regenerating the ADMINSID if you want to convert user emails. But, otherwise, you should still use the admin account to convert since it seems to be working better for you.
@nneonneo - I've sussed out why the second run without the ADMINSID had less posts and topic converted. The reason was some of the forum categories has restricted permissions for non admin users. I removed these permission restrictions via the forum ACP and rerun the script, again without a ADMINSID and all seems well.

Now onto the next stage of converting to phpbb2 to 3.......
LECoupe
Registered User
Posts: 3
Joined: Mon Oct 22, 2012 12:38 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by LECoupe »

nneonneo wrote:@LECoupe: ugh, the Forumer/Yuku board really is quite different (at least in appearance). There are some similarities to regular Yuku boards, though. Is there no way to "flip a switch" and get a regular Yuku theme?
Hmmm...I don't believe so but couldn't swear to it. I'll have a poke about in there but I'm not hopeful.
[edit: had a look, it's so broken that even applying their own skins doesn't work...it's puke green+black no matter what I do.]

I have a cunning plan though. 8-)

Forum Members I will do by hand - not too many of those.

For the categories/forums/topics/posts: You might have noticed (depending on when you looked) that I moved all the forums into a single category, and we only have a handful of forums anyway. Topics and posts...ack, there's hundreds of topics and ~12500 posts, which our members are pretty keen to hang onto. So I plan to hack away at your python script to hard code my one category and 5 or 6 forums in there and see how it copes with the topics and posts (your "grep list of doom" is a little confronting though.... :shock: )

Ugly, but if it works it'll do.

Now I just need to learn Python. :roll:
bertran
Registered User
Posts: 3
Joined: Sun Oct 21, 2012 3:25 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by bertran »

@bertran: I don't plan on doing that in the near future, as I don't have a lot of time available to devote to the converters, unfortunately.
Thank you anyway nneonneo :)
Adam Overmiller
Registered User
Posts: 1
Joined: Thu Nov 08, 2012 1:38 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by Adam Overmiller »

We currently have a massive forum with years of posts hosted on Forumer. We are looking at options to move somewhere where we have more control and a paid server for downtime accountability. Forumer does not allow you access to database to get a DB copy. However I have written a tool that parses the entire board page by page and can output any way I want with all the data needed (post id, user name, timestamp, content, signature, etc...).

Without purchasing I can't tell what your database schema is like or if it's possible to do some kind of manual database inserting to import all our posts. So is that something that is possible?
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by nneonneo »

@Adam Overmiller: The Forumer converter, attached to the first page of this topic, should do what you want. It downloads and parses every page (much like your tool does) and outputs a database dump ready for import into phpBB.
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
a1topdog
Registered User
Posts: 32
Joined: Sat Nov 26, 2005 10:49 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by a1topdog »

I''ve managed to set up my phpbb2 forum using the forumer crawler scripts. Many thanks for that. I have the problem that others have had where the members file upload over rides the default admin user on the new clean forum. Please can someone post up the phpbb2 SQL to either:

1. Update an existing user to admin permissions, or
2. Create a new user with admin permissions

Many thanks.
User avatar
HGN
Former Team Member
Posts: 4706
Joined: Wed Dec 03, 2008 1:53 pm
Location: The Netherlands
Name: Alfred

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by HGN »

a1topdog wrote:2. Create a new user with admin permissions
Try the following MySQL query. You may need to change the prefix of the tablename to match your tablename.
To avoid overwriting an existing user_id, user_id 8388607 is used.
The query will create an admin account with username admin1 and password admin. In case you keep this admin, please make sure to change the password, to prevent for illegal use of the account.

Code: Select all

INSERT INTO phpbb_users (user_id, username, user_regdate, user_password, user_email, user_icq, user_website, user_occ, user_from, user_interests, user_sig, user_sig_bbcode_uid, user_avatar, user_avatar_type, user_viewemail, user_aim, user_yim, user_msnm, user_attachsig, user_allowsmile, user_allowhtml, user_allowbbcode, user_allow_viewonline, user_notify, user_notify_pm, user_popup_pm, user_timezone, user_dateformat, user_lang, user_style, user_level, user_allow_pm, user_active, user_actkey) VALUES (8388607, 'admin1', 0, '21232f297a57a5a743894a0e4a801fc3', '[email protected]', '', '', '', '', '', '', '', '', 0, 0, '', '', '', 0,1, 1, 1, 1, 0, 0, 0, 1.00, 'D M d, Y H:i', 'english', 0, 1, 1, 1, '');
User avatar
HGN
Former Team Member
Posts: 4706
Joined: Wed Dec 03, 2008 1:53 pm
Location: The Netherlands
Name: Alfred

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by HGN »

Beside above mySQL query I found two other ways to retrieve administrator access. They can be found in this topic: viewtopic.php?p=2351630#p2351630
One is the Admin Toolkit for phpBB2 (still available), the other is a fix file to reset a users password to 'admin'.
a1topdog
Registered User
Posts: 32
Joined: Sat Nov 26, 2005 10:49 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by a1topdog »

@HGN

Thanks for the SQL , it works a treat.
Bosse999
Registered User
Posts: 1
Joined: Thu Nov 15, 2012 12:40 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by Bosse999 »

Hi

Trying to move som 40000 posts from phpBB2 where the owner will not release the data.
I have managed to get the converter running but it fails to fetch forumlinks. I guess a cnahge of the regexp is needed but I cant figure it out. Anyone see the problem?

BR
/bo

<tr>

<td class="row1" align="center" valign="middle" height="50"><img src="templates/subSilver/images/folder_big.gif" width="46" height="25" alt="Inga nya inlägg" title="Inga nya inlägg" /></td>

<td class="row1" width="100%" height="50"><span class="forumlink"> <a href="me-forum-2.html" class="forumlink">Här diskuterar vi ME/CFS</a><br />

</span> <span class="genmed"><br />

</span><span class="gensmall"><b>Moderator</b> <a href="profile.php?mode=viewprofile&u=3">Moderator</a></span></td>

<td class="row2" align="center" valign="middle" height="50"><span class="gensmall">1678</span></td>

<td class="row2" align="center" valign="middle" height="50"><span class="gensmall">18546</span></td>

<td class="row2" align="center" valign="middle" height="50" nowrap="nowrap"> <span class="gensmall">1352138337<br /><a href="profile.php?mode=viewprofile&u=2">Admin1</a> <a href="me-post-30376.html#30376"><img src="templates/subSilver/images/icon_latest_reply.gif" border="0" alt="Läs senaste inlägget" title="Läs senaste inlägget" /></a></span></td>

</tr>
a1topdog
Registered User
Posts: 32
Joined: Sat Nov 26, 2005 10:49 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by a1topdog »

I've got my phpbb2 forum up and running now , but the search function is not working when using the "seach for keywords" option for the converted posts. It does work for posts on the forum that were made after conversion. The search seems to be OK (or better) when using the "Search for Author" option. I did resync every sub forum via the ACP forum admin/management after the conversion. If there something else I need to do?
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by nneonneo »

@a1topdog: You will need to rebuild the search index in Admin->Maintenance->Database.

@Bosse999: Try the updated converter attached. You can easily edit the regexp used in this version. For your forum, you probably have to add

Code: Select all

    r'forum-(\d+)\.html',
to the re_forum_urls array, and

Code: Select all

    r'topic-(\d+)\.html',
to the re_topic_urls array.
You do not have the required permissions to view the files attached to this post.
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
a1topdog
Registered User
Posts: 32
Joined: Sat Nov 26, 2005 10:49 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by a1topdog »

@nneonneo: I'm using phpbb2 and there is no Admin->Maintenance->Database option in the ACP.
User avatar
HGN
Former Team Member
Posts: 4706
Joined: Wed Dec 03, 2008 1:53 pm
Location: The Netherlands
Name: Alfred

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by HGN »

Support for phpBB2 has ended. Any support requests regarding phpBB2 are limited to help with conversion to phpBB3.

You should convert to phpBB3 and in phpBB3 recreate the search index.
a1topdog
Registered User
Posts: 32
Joined: Sat Nov 26, 2005 10:49 pm

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by a1topdog »

HGN wrote:
a1topdog wrote:2. Create a new user with admin permissions
Try the following MySQL query. You may need to change the prefix of the tablename to match your tablename.
To avoid overwriting an existing user_id, user_id 8388607 is used.
The query will create an admin account with username admin1 and password admin. In case you keep this admin, please make sure to change the password, to prevent for illegal use of the account.

Code: Select all

INSERT INTO phpbb_users (user_id, username, user_regdate, user_password, user_email, user_icq, user_website, user_occ, user_from, user_interests, user_sig, user_sig_bbcode_uid, user_avatar, user_avatar_type, user_viewemail, user_aim, user_yim, user_msnm, user_attachsig, user_allowsmile, user_allowhtml, user_allowbbcode, user_allow_viewonline, user_notify, user_notify_pm, user_popup_pm, user_timezone, user_dateformat, user_lang, user_style, user_level, user_allow_pm, user_active, user_actkey) VALUES (8388607, 'admin1', 0, '21232f297a57a5a743894a0e4a801fc3', '[email protected]', '', '', '', '', '', '', '', '', 0, 0, '', '', '', 0,1, 1, 1, 1, 0, 0, 0, 1.00, 'D M d, Y H:i', 'english', 0, 1, 1, 1, '');
f/y/i - Just noticed any new user trying to register for my forum was unable to and was getting the following debug message:

"Could not insert data into users table"

Once I deleted this admin1 user which was inserted using the above sql the user registration process seems to be working again.

Return to “[3.0.x] Convertors”