Page 2 of 37

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Nov 14, 2009 7:02 am
by nneonneo
@eatingmashin: Try the resync_bbcode.php: http://www.startrekguide.com/community/ ... 648#p47648

@Arrnea: You could also have e-mailed me: nneonneo [at] gmail <dot> com.

EDIT: Ok, Arrnea, I just ran a test conversion. Since I don't have many permissions on the forum, I can only see a few topics, but of those topics I can see, the converter converted all of them.

The result is a large number of "Topic is empty!" messages with the occasional "Completed! Added <n> posts". However, it seems to me that it is working. Remember, NUMTOPICS should be greater than the highest topic ID, not the total number of topics.

EDIT2: So it apparently isn't just my good luck: there's a minor change that needs to be made. iPBfree apparently started checking the user agent of the browser for security reasons, and my script has been logging on with a different user agent than the one it actually uses to crawl the forum.

This is easy to fix: in common.py, change

Code: Select all

req=urllib2.Request(url,headers={'COOKIE':cookiedata,'User-Agent':'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5'})
to

Code: Select all

req=urllib2.Request(url,headers={'COOKIE':cookiedata})
and run the get_cookie script again. This should allow all the threads to be properly downloaded.

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Nov 14, 2009 8:16 am
by Arrnea
Thanks for the quick response. I'll try that soon and let you know if it works.

Edit: It seems to be working fine now. The odd topic returns an empty topic error, but those are topics that have been deleted at one point or another. ;)

Edit again: Using relative timestamps (eg: Yesterday, 2:10pm, 53 minutes ago etc) also seems to cause an error. I've turned them off, should take care of the error, but it might be something worth adding to the instructions of things to turn off.

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Nov 14, 2009 9:01 am
by Arrnea
Ok, getting this when I try and import it into my MySQL database:

Code: Select all

SQL query:

INSERT INTO `phpbb_forums` ( forum_id, cat_id, forum_order, forum_name, forum_desc )
VALUES ( 2, 3, 20, 'SOS Brigade Archives', '' ) ;

MySQL said: Documentation
#1054 - Unknown column 'cat_id' in 'field list' 
Edit: Oh, I know why this is happening. It's because I'm trying to import into phpBB3 and this script is designed for phpBB2. Curses.
I'll manually edit the references to categories out of the generated sql file and see if it works then.

Edit again: Meh, I'll just install phpbb2 then upgrade to phpbb3 from the existing installation.

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Nov 14, 2009 4:12 pm
by nneonneo
Thanks for the tip about relative timestamps. I'll add that to the instructions.

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Tue Dec 08, 2009 1:08 am
by Shomiku
I downloaded the converter and python and read the instructions. . .
But i cant figure out how i start it,
do you think you could help. . .?
Icase it helps i am trying to convert Poboards to invisionfree.



Also, when i get it working, do all the threads/posts/etc stay on the old one as well?

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Tue Dec 08, 2009 1:52 am
by nneonneo
Did you say you wanted to convert Proboards to Invisionfree? I'm afraid that's not possible...the converter can only take Proboards to phpBB (or SMF), or Invisionfree to SMF.

Posts, etc. will remain on the old forum after conversion.

Can you state more specifically what your problem is?

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Tue Dec 08, 2009 2:26 am
by Shomiku
The problem is i dont knw how to run it, like, do i put in the site URL somwhere?


nneonneo wrote:Did you say you wanted to convert Proboards to Invisionfree? I'm afraid that's not possible...the converter can only take Proboards to phpBB (or SMF), or Invisionfree to SMF.

Posts, etc. will remain on the old forum after conversion.

Can you state more specifically what your problem is?
Oh. . . well, can phpBB be converted into anything other then proboards?

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Tue Dec 08, 2009 3:52 am
by nneonneo
I believe the instructions clearly state that you put the site URL and other settings in the settings.py file. If you are confused, please read them again, carefully.

Erm, are you asking if Proboards can be converted into something other than phpBB, or the other way around? I don't believe it is possible to import data into Proboards or Invisionfree, so there is no way to convert anything to these two, but with my crawler, you can convert *from* them *to* other software, like phpBB.

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Dec 12, 2009 5:41 am
by fantanoice
nneonneo wrote:You're in luck, then! It turns out that the iPBfree/Invisionfree converter, with minor changes, can convert Forumer boards. Attached is a converter package that should handle Forumer forums in IPB mode. Let me know how it goes.
Now, this seems to be working alright, but I have a topic with 3,000+ posts which tends to devastatingly slow down at about the 70-page mark (out of 178). Is there a way to speed this up?

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Dec 12, 2009 6:12 am
by nneonneo
I don't know of any way to speed up the conversion if it is slow on a download. If it consistently slows down at the same spot, it's possible that Forumer is applying rate-limiting (throttling), or that this particular thread (at that spot) just takes a long time to load. Does the slowness persist after the thread is completed?

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Dec 12, 2009 7:27 am
by fantanoice
It just seems to gradually slow down after each page. It's still downloading (page 112/178), so I'm not sure whether it's still slow after it has been completed.

Every other thread up to this point converted pretty quickly, but they maybe had, at max, 50 posts to convert (about 2-3 pages). 178 pages is, well, a LOT more than that. xD

Though I still have 1,500+ topics to convert after this one, though this is the forum's largest topic to it should be 'okay' to convert the others if it gets through this.

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sat Dec 12, 2009 8:00 am
by nneonneo
How much is being downloaded each time? It's possible that the total size of the topic is so large that it is causing problems for the converter.

To check, see if the "python" process on your computer is taking a lot of CPU time. If it is, then it is almost certainly the aforementioned efficiency problem; if not, then it's a slow download.

The fix is as follows:
Change the lines

Code: Select all

        pagedata=download_page(URL+'?showtopic='+str(topicid),progressline, COOKIEDATA)
        pages=re.findall(re_pages,pagedata)
to

Code: Select all

        pagedata=[download_page(URL+'?showtopic='+str(topicid),progressline, COOKIEDATA)]
        pages=re.findall(re_pages,pagedata[0])
(be sure to preserve the indentation)
and change

Code: Select all

                pagedata+=download_page(URL+'?showtopic='+str(topicid)+'&st='+str(offset*perpage),progressline, COOKIEDATA)
to

Code: Select all

                pagedata.append(download_page(URL+'?showtopic='+str(topicid)+'&st='+str(offset*perpage),progressline, COOKIEDATA))
        pagedata = ''.join(pagedata)

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Dec 27, 2009 11:39 am
by damienlucifera
Hi,
I'm having trouble converting posts and threads from Proboards. Everything else seems fine, but whenever it starts converting posts and threads, it comes up with error messages telling me to refer to the convert_error.txt

Here's one of the errors:

Topic ID 257 in [BOARD NAME]
Traceback (most recent call last):
File "[FILE DIRECTORY]\Proboards\members_topics_posts_smf.py", line 496, in <module>
post[0]=parsetime(ln[4])
File "[FILE DIRECTORY]\Proboards\members_topics_posts_smf.py", line 154, in parsetime
,fmt)
File "[FILE DIRECTORY]\Python\lib\_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "[FILE DIRECTORY]\Python\lib\_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '11/11/09 at 3:25am' does not match format '%m/%d/%Y at %I:%M%p'

Obviously I've taken out certain info on the board name and file directory, but I'm hoping whatever I've provided is enough for someone to let me know where I'm going wrong?

Please help!

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Sun Dec 27, 2009 8:31 pm
by nneonneo
You selected the wrong date format in your preferences; it should be the one that looks like "12/28/2004 at 11:11pm", not "12/28/04 at 11:11pm" ("2004" instead of "04").

Re: [CONVERT] Proboards/Invisionfree/ActiveBoards converters

Posted: Thu Dec 31, 2009 2:10 pm
by mbird_2005
Arrnea wrote:Hi, I am attempting to use your IPBfree crawler converter to move my IPBfree forums to phpBB.
However, when I attempt to do so, I find the following error upon executing the topics_posts_phpBB.py file:

Code: Select all

Topic ID 1 ... Error: Topic is empty!                                          
Topic ID 2 ... Error: Topic is empty!                                          
Topic ID 3 ... Error: Topic is empty!                                          
Topic ID 4 ... Error: Topic is empty!                                          
Topic ID 5 ... Error: Topic is empty!                                          
Topic ID 6 ... Error: Topic is empty!                                          
Topic ID 7 ... Error: Topic is empty!                                          
Topic ID 8 ... Error: Topic is empty!                                          
Topic ID 9 ... Error: Topic is empty!                                          
Topic ID 10 ... Error: Topic is empty!                                         
Topic ID 11 ... Error: Topic is empty!                                         
Topic ID 12 ... Error: Topic is empty!                                         
Topic ID 13 ... Error: Topic is empty!                                         
Topic ID 14 ... Error: Topic is empty!                                         
Topic ID 15 ... Error: Topic is empty!                                         
Topic ID 16 ... Error: Topic is empty!                                         
Topic ID 17 ... Error: Topic is empty!                                         
Topic ID 18 ... Error: Topic is empty!                                         
Topic ID 19 ... Error: Topic is empty!
This continues through all of the Topic IDs until the cap specified in settings.py is reached.
Any ideas what is causing this?

I'm running python2.6 on Ubuntu 9.10 'Karmic', by the way.

i have the same problem here

any one can help me plz ??