Page 1 of 14

Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Jan 27, 2008 9:54 pm
by nneonneo
EDIT: The converters are posted a few posts down.
Hello:

I've been told that making a screen-scraper type converter like the exBoard converter isn't legal for Proboards, as it is against the TOS. However, that script is also technically against the ezBoard TOS for the same reason (TOS 6h).

I have Proboards, Invisionfree and ActiveBoards converters for 2.0.x. They are far from perfect, but they convert members, topics and posts which is usually what is most needed. However, I have not released them here for fear of a legal repercussion or similar effect.

Presently, I've already been able to do a lot of conversions for people who specifically ask for it; however, I would like to release the software for public use (free of charge, and I will GPL it if necessary or desired).

Can someone help me understand this legal situation?

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Fri Feb 08, 2008 7:30 pm
by MjrNuT
I sent you a PM.

Thanks in advance!

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Thu Feb 14, 2008 10:27 pm
by nneonneo
As it happens, nobody has brought on any complaints, so I am releasing this set of converters.

Please note: I provide no warranty on this software, but I will be happy to provide support in this thread.

Converters for Invisionfree, ActiveBoards, MSN Groups and several other forums are provided. Please read the Instructions.txt file in the corresponding folder. I've tried my best to make these understandable, but any comments or suggestions on any part of the process are welcome.

The phpBB2 converter produces SQL which should be compatible with phpBB 2.0.22.
The phpBB3 converter produces SQL which should be compatible with phpBB 3.0.0.

The converters are NOT written in PHP, they are written in Python. This means that for Windows users (Linux and Mac OS X should come with Python interpreters) you will need a current distribution of the Python programming framework, available at http://python.org (the Windows Installer link on the left is the most direct way to get it).

Note that this converter has been re-uploaded several times. Thus, the download count may not be accurate. As of time of writing, it has been downloaded about 576 times.

EDIT: I should probably also add that the ActiveBoards converter is SMF only, since no request has ever been made for a phpBB version; if you need to convert, go through SMF 1.1 and then convert to phpBB.
EDIT 2: Yuku converter removed by request from Yuku.
EDIT 3: Added board conversion feature to Proboards converter, fixed a bunch of minor bugs.
EDIT 4: Fixed moderators bug, fixed missing pages bug in Proboards converter.
EDIT 5: Added script to create bot/crawler accounts, fixed members bug (anonymous had the wrong ID!)
EDIT 6: Fixed members conversion in Invisionfree
EDIT 7: Fixed Invisionfree email links and single-quote handling (thanks pjf123 for reporting it!)
EDIT 8: Updated instructions
EDIT 9: Invisionfree fixes: fix for usernames with strange characters (< and >), fix for missing group entries. Version bump to 0.1.3.
EDIT 10: Several minor fixes for the Proboards -> phpBB3 converter, including correct code, quote and list tags, corrected font sizes, basic bbcode_bitfield handling (it should soon be possible to leave out the resync_bbcode.php step), fix for subforum topics, minor fix for internal profile links and correct date, time and user stamps on topics (thanks Kymmy for pointing this out)
EDIT 11: Added one more missing field (thanks again, Kymmy!) to Proboards, and fixed a minor username bug in the Invisionfree converter
EDIT 12: Put the _right_ version of pm_passwords in the archive...
EDIT 13: Minor fixes to make it work with the new Proboards.
EDIT 14: Improved compatibility with Proboards v4.5. Fixed split_sql to properly split phpBB3 SQL files (thanks to gonefishin for reporting this one).
EDIT 15: Oops, fixed major bug in Proboards guest post conversion -- thanks to anyways for reporting it.
EDIT 16: Minor updates to all converters to bring them up-to-date with the latest forum code. Removed PB converter (by request) and added several converters.
EDIT 17: Updates to Forumotion to compensate for new code, user_group bug off-by-one fix for nearly all phpBB2 converters, updated SuddenLaunch to correctly deal with extra pages of topics, better error handling in SuddenLaunch PM module and better error handling from common.py (affects all converters) -- all errors are now caught and retried, and major errors produce a critical_error.log file which can be used to determine the cause. Also added fora.pl -> phpBB2 converter.
The package now contains ActiveBoards, fora.pl, Forumotion, Invisionfree, iPBfree, SuddenLaunch and Zetaboards converters.
EDIT 18: Update to Forumotion to compensate for new code, iPBfree member conversion update
EDIT 19: Serious update to ZetaBoards converter, minor updates to Forumotion converter, added MSN Groups
EDIT 20: More updates to ZetaBoards (fixed serious bug in post conversion and a few minor things).

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Wed Feb 20, 2008 6:06 pm
by pjf123
Hi nneonneo,

I've tried the Invisionfree to PHPBB scripts. I think these are a great idea and match my current migration route from Invisionfree - thanks for sharing these.

I hit a couple of small issues and have detailed these below. I'm running against a clean install of the latest build of PHPBB2 running locally on WAMPServer. If you need any more info or a better description please let me know.

The following relate to the SQL script generated by members_phpBB.py
  • 1. The Truncate SQL statement fails as it refers to a members table rather than the users table. I can't see a members table on the database so I think this is a typo and the code should match the Insert SQL and refer to the users table.

    Code is currently : out.write('TRUNCATE TABLE `'+table_prefix+'members`;')
  • 2. If the Truncate statement above is changed to refer to the users table the script runs, but I get the following issues with PHPBB:

    a: The Truncate SQL statement deletes the -1 ID user that seems to be used internally in PHPBB.
    This initially showed up as an error when logging out - I suspect it may well also break PHPBB elsewhere.
    Possibly replacing the Truncate with a Delete where ID > -1 would work around this.

    b: The administrator user from InvisionFree didn't seem to get administrator rights on the PHPBB board. They got a custom title of "Site Admin" but didn't have access to the Admin Panel via the link at the bottom of the forum.
    As the administrator account created during the PHPBB install was also removed by the above Truncate I ended up with no administrator access.

    (I'll try a dig a bit more on this one, but I think I got it to work by doing a fresh install, removing the truncate from the SQL script, renumbering the administrator login created when installing PHPBB to a high ID and then running the script in. However I currently have a stinking cold so will try an recreate the problem from scratch another day.)

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Wed Feb 20, 2008 7:22 pm
by nneonneo
Oops.

That would all be my fault. I uploaded a not-so-new version of the Invisionfree converters.

I'll re-upload them. Sorry for the trouble. I've also fixed the admin problem.

Also, I think only members_phpbb.py was affected; are the topics and posts OK?

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Feb 24, 2008 12:41 pm
by Kymmy
I'm currently trying this out migrating a proboards to either phpBB3 or phpBB2

On the phpBB3 I came along 2 problems, the first was when doing the posts query that bbcode_uid needed it's limit extending from 8 characters (i put it briefly at 20 and that was OK but will try to give you the exact figure needed later). When doing the members query it came up with a duplicte user_id at 53 (which is what the install set the admin user to be) I redid the user_ID to a lot higher but then it gave me the same errer at 54. I'll look into that later as currently I'm trying the phpBB2 migrate.

Thanks though for a great little script, if you can just get the final bugs out then I think it'll be a godsend to those stuck on places like proboards and don't want to lose thier history when migrating away.

Kymmy

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Feb 24, 2008 4:21 pm
by Kymmy
OK, The phpbb3 install still keeps giving me some sort of IDENTICAL KEY error, starting at 53, if you delete that user_id then it it gives the same error for 54 and so on adding one all the time.

phpbb2 gave one error and that was that 100 length wasn;t quite enough for the user_avatar field, so I upped it to 150 and after that it was fine. It also seemed to miss out one of the forums (lucky for me it's the least used "suggestions" forum.) I'll take a good look just in case I messed up the board short names in settings.py

I did a convert from phpbb2 to phpbb3 which worked well so I'm a very happy girl.

Thanks again.

Kymmy :D

(UPDATE} The missing forum was one I missed out the board short name in setting.py so no problem there

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Feb 24, 2008 4:26 pm
by nneonneo
Sorry about the BBCode thing; I'd actually fixed that bug already (for phpBB3) but neglected to upload the updated package to this topic (it was up at the phpBB3 converter topic already). phpBB2 used 10-digit UIDs, while phpBB3 uses 8, a detail that I overlooked when upgrading the converter. So, that's been fixed.

As for the users -- I haven't run into that myself, so I can't tell what the problem could be (I've converted a few tests with about 20 users, and nothing like that has come up). The script is set to delete all entries from the user table (caveat: this includes bots, though it should not be too difficult to write some SQL to put them back). If the user table isn't emptied, the script will probably cause tons of duplicate key errors.

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Feb 24, 2008 4:38 pm
by Kymmy
nneonneo wrote:As for the users -- I haven't run into that myself, so I can't tell what the problem could be (I've converted a few tests with about 20 users, and nothing like that has come up). The script is set to delete all entries from the user table (caveat: this includes bots, though it should not be too difficult to write some SQL to put them back). If the user table isn't emptied, the script will probably cause tons of duplicate key errors.
After the error the only entry in user_id is entry 1 with a key of 53. As you said all the bots and installation users (admin/anonymous) have gone.

If I can't figure it out I could send you a link to the members SQL file though now I know I can go the long way round (phpbb2 and then convert to phpbb3) it's not really a major issue though I'm sure that you yourself would like it bug free :)

Kymmy

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Feb 24, 2008 4:59 pm
by nneonneo
Yeah, members.sql would be useful. As far as I know, the very first user inserted (ID 1) should be anonymous, then the usual members...

EDIT: Oops, anonymous *wasn't* ID 1. That's now fixed.

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Mon Feb 25, 2008 12:18 pm
by Kymmy
As I wanted it for phpbb3 (yes I know that this is the phpbb2 convertor forum :p) I first migrated it to phpbb2 and then ran the phpbb3 convertor. This then added all the bots/users that phpbb3 needs to run. Might be a long way round, but it worked :)

Kymmy

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Mon Feb 25, 2008 2:02 pm
by nneonneo
That'll work fine. The main problem is that some inaccuracy is produced by having a double conversion, so I've been trying to make sure the phpBB3 installation is as accurate as possible.

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Mon Feb 25, 2008 5:49 pm
by pjf123
Hi nneonneo,

I've tried the updated InvisionFree convertor to see if it got rid of the previous problems I had, but unfortunately I now get the following for each member id when running members_phpBB.py :

Traceback (most recent call last):
File " ... \members_phpBB.py", line 82, in <module>
out.write("INSERT INTO `"+table_prefix+"groups` (group_id, group_type, group_name, group_description, group_moderator, group_single_user) VALUES ('%i', '1', '%s', 'Personal User', '0', '1');\n"%(dat[2]+1,'Admin'*dat[19]))
TypeError: cannot concatenate 'str' and 'int' objects

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Mon Feb 25, 2008 11:20 pm
by nneonneo
Whoops! Sorry about that, my testing copy was the wrong version so I didn't test my code before uploading it :oops:

It's now fixed. Sorry for the trouble.

Re: Proboards/Invisionfree/ActiveBoards converters

Posted: Thu Feb 28, 2008 6:02 pm
by pjf123
Hi nneonneo

I've found a small issue in the topics handling from InvisionFree. The python script runs fine against my forum, but the SQL won't run into the database.

I've located the bit of SQL that's failing. It's been generated from a topic post in which the member used a mailto link. Looks like when these are converted it ends up with a quote mark hanging that breaks SQL.

The format in the original post's HTML source is :
Lots of text
<a href='mailto:nnn@nnn.co.uk'>nnn@nnn.co.uk</a>
More text
<a href='http://www.nnn.co.uk' target='_blank'>Link to site</a>
This gets converted to the following in the SQL file:
Lots of text
[url=mailto:nnn@nnn.co.uk\\'>nnn@nnn.co.uk</a>
More text
<a href=\\'http://www.nnn.co.uk]Link to site[/url]
I'm guessing the quote mark just after the first instance of the email address is causing the SQL failure.

Not a biggie as I think that's the only post using a mailto in the whole forum so I can happily remove it before running the convertor, but thought you may want to know.