[CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Converting from other board software? Good decision! Need help? Have a question about a convertor? Wish to offer a convertor package? Post here.
Anti-Spam Guide
Locked
vacz
Registered User
Posts: 3
Joined: Wed Feb 24, 2010 7:42 pm

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by vacz »

Thank You very much :) You give me some hope...

Settings (time, lang) was done in my profile and in forum config.
I've checked other things and found that the subSilver theme was deleted from admin panel. Now I installed it and set in my profile and in config.
Also changed table prefix in settings.py cose "forum_" looks to be not good idea ;)
Should I login to admin panel before cookie extraction? I took cookie always after login to forum not to admin panel.

Our forum stats are like this:

Code: Select all

Number of posts:  	1500 (~about)
Number of topics: 	50 (~about) 
Number of users: 	50 (~about)
We had about 3200 posts, but before we started conversion we deleted many posts to clean unneeded messages. Resync'ed all forums after delete.

Now python tells me (after members and topics downloading):

Code: Select all

Renumbering Posts
Elapsed Conversion Time 31.55 seconds
Converted 12 posts and 3 topics.
Complete! Press <enter> to exit.
It is better now :) but not perfect ;)
- still critical_error.log empty
- categories_forums_phpBB.sql - now i see all categories and forums :)
- members looks fine, just like passwords
- topics_posts_phpBB.sql has:
-- INSERT INTO `_topics` only 3 topics
-- INSERT of 12 posts into "_posts" table

After investigating, I found what happen: subforums was not converted!
Our forum has: categories, forums, subforums - only when forum has no subforums, all messages was converted from that forum. All 12 messages in 3 topics was posted in forums witout subforums :) :!:
Also when we use subforums - we dont put any messages into forum that has some subforums.

Am I right about subforums conversion? Is there any simple solution? Or should I switch subforums to forums for convertion time, then switch it back after sql import?

----------
2nd question (less priority, I can fight with this):
All converted posts have this texts on very end:

Code: Select all

The post has been approved 0 times (If you\'d like to approve this post, click [url=http://www.(myforumname).fora.pl/notice.php?m=approve&u=100&p=1000]here[/url])
[url=http://www.(myforumname).fora.pl/posting.php?mode=replace&p=1000&t=500]Move this post[/url]
Is there a way to remove this text globally from posts content? I think python can do that, but You know, I can't tell python how to do that :(
If not, I will try to remove it with some software text editor. It is just cosmetics, nothing important compared to subforums problem ;)
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by nneonneo »

phpBB2 doesn't support subforums by default, so even if I added the feature to convert them, they would still have to be flat forums. So, for now I think it is easier to just change the subforums to forums before conversion.

For the other (minor) problem, I think you can make it disappear by disabling "Enable post moving" in the admin panel (general configuration).
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
vacz
Registered User
Posts: 3
Joined: Wed Feb 24, 2010 7:42 pm

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by vacz »

Nneonneo , thank you very much 4 Your converter! Now I have copy of my forum :) and I can move it.

For the record: I had little problem, file convert_error.log has been created:

Code: Select all

Topic ID 100
Traceback (most recent call last):
  File "C:\Converters\Fora.pl\members_topics_posts_phpBB.py", line 360, in <module>
    post[0]=parsetime(ln[4])
  File "C:\Converters\Fora.pl\members_topics_posts_phpBB.py", line 91, in parsetime
    return int(tm) # if you followed the instructions, all dates should now be UNIX timestamps, so we can CHEAT.
ValueError: invalid literal for int() with base 10: '1266918477<br />MOVED<br />1266919589'
so I changed line 360 in members_topics_posts_phpBB.py from:

Code: Select all

post[0]=parsetime(ln[4])
changed to:

Code: Select all

post[0]=parsetime(ln[4].split('<')[0])
I think, it's in language of pythons ;) cose it worked :D (i know little javascript, so managed how it should be, but all those regular expression seems to be magic...)

And :) now The Python said to me:

Code: Select all

Renumbering Posts
Elapsed Conversion Time 151.56 seconds
Converted 1495 posts and 42 topics.
Complete! Press <enter> to exit.
Haha I've got it!
Going to import it now to mysql.
Thanx again nneonneo
MissPear
Registered User
Posts: 2
Joined: Mon Mar 01, 2010 1:19 am

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by MissPear »

Hello!

Firstly, thank you so much for making these converters - I can see they're making life easier for a fair few people who want to escape the trappings of various forum software. It's much appreciated!

However, I'm having a problem that seems to have come up before: the "Error: Topic is empty!" message. I'm trying to convert Proboards to phpbb3, have checked about fifty times that all the settings are correct in both my profile on the board (date/time, skin, avatars, images etc) and in settings.py. The members convert with no problems at all, as do the forum categories - but in the main, the topics and posts just will NOT convert!

The converter is downloading the page data, but on 99% of topics gets to "stage 5 of 36" and then returns the dreaded "Topic is empty" error. Something I HAVE noticed: the few posts that DO get converted are by someone who was a former member, but deleted their account. Could this have something to do with it?

Thanks in advance for any advice!
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by nneonneo »

Proboards modified some code on their forums; a new converter will be coming out soon, but in the meantime, you can make the following fix (from http://www.phpbb.com/community/viewtopi ... #p12185975):

In members_topics_posts_phpBB3.py, change

Code: Select all

("""<script language="JavaScript" type="text/javascript"><!--\\ndocument.write\\('<a href="mail'\\+'to:([^']+?)'\\+'@'\\+'([^"]+?)">'\\);// --></script>""","""<a href="mailto:\\1@\\2">"""),
to

Code: Select all

("""<script language="JavaScript" type="text/javascript"><!--\\ndocument.+\\'<a href="mail'\\+'to:([^']+?)'\\+'@'\\+'([^"]+?)">.+?';// --></script>""","""<a href="mailto:\\1@\\2">"""),
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
MissPear
Registered User
Posts: 2
Joined: Mon Mar 01, 2010 1:19 am

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by MissPear »

This worked perfectly; thank you so much!
sergiomiranda
Registered User
Posts: 11
Joined: Mon Jan 12, 2009 5:16 pm

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by sergiomiranda »

I used the Forumotion script but had a few problems with importing the SQL files to the new phpBB3 installation I had made.

I solved them by importing the files with Latin 1 encoding. Using UTF8 (the default, i believe) would ruin the import.

Just thought I would share in case anyone runs into the same problem.

Thank you for the scripts, nneonneo! Amazing work!
Formerly known as DarkAngelPT
khanater
Registered User
Posts: 1
Joined: Tue Mar 09, 2010 5:45 pm

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Post by khanater »

Hi!
Somebody succesfully exported a phpbb2 forumer account?

regards

ps: sorry my previous msg, tryied to be a joke :(
terrifiedofcarrots
Registered User
Posts: 7
Joined: Sun Mar 21, 2010 5:10 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by terrifiedofcarrots »

phpBB3 forum on Forumer?
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by nneonneo »

terrifiedofcarrots wrote:phpBB3 forum on Forumer?
Attached is a converter which I've created for Forumer phpBB3. Try it and let me know how it goes.
Attachments
Forumer_phpBB3.zip
Forumer phpBB3 converter (early alpha version)
(17.72 KiB) Downloaded 113 times
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
terrifiedofcarrots
Registered User
Posts: 7
Joined: Sun Mar 21, 2010 5:10 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by terrifiedofcarrots »

Thanks. It seems to be working. I don't have access to an admin account right now, and instead of using my moderator account on the forum I was crawling I created a new regular user account, so it didn't grab any private forums or email addresses, but everything else seemed to be fine.

One thing I noticed is that the last topic it downloaded was index 32767, so either it's a weird coincidence that the last public thread has that index, or you might have used a 16-bit integer in your script and it maxed out.
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by nneonneo »

The script is written in Python, so it uses the unbounded (big) integer type. Thus, 32767 probably comes from Forumer. Can you see if this is the case?
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
terrifiedofcarrots
Registered User
Posts: 7
Joined: Sun Mar 21, 2010 5:10 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by terrifiedofcarrots »

nneonneo wrote:The script is written in Python, so it uses the unbounded (big) integer type. Thus, 32767 probably comes from Forumer. Can you see if this is the case?
I don't know any Python, so I trust you. When it comes time for the real move, I'll have it use an admin account and if it still stops at 32767 I'll come yell at you again ;)

Wait. It didn't grab any forums that had subforums.
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by nneonneo »

The subforum thing is my fault. I should've checked that case more thoroughly.

Change

Code: Select all

</a><br />\\s*(.*?)\\s*</dt>'''
to

Code: Select all

</a><br />\\n\\t*(.*)'''
and forums with subforums should be detected and converted properly.
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
nneonneo
Registered User
Posts: 549
Joined: Sun Apr 30, 2006 1:42 am

Re: [CONVERT] Crawler Converters (Forumer, ZetaBoards, etc.)

Post by nneonneo »

Well, I'll be damned. I just ran the Forumer phpBB3 converter against this phpBB community forum (well, for a few seconds, anyway) and it started working :shock:

So, I suppose the Forumer phpBB3 converter is really a "generic" phpBB3 converter at this point. That's kind of useful. Perhaps the same is true of ForumerIPB and Forumer_phpBB2, in which case I think this can be a very important set of converters.
Need a conversion from another forum, but they won't give you the database? Try a crawler converter. If your converter isn't listed, feel free to post in that thread to ask for one.
Locked

Return to “[3.0.x] Convertors”