The SQL Import goes fine, It is really python that is giving these errors. It only chokes on those threads (the 45 000 + posts ones). It downloads everything fine, it's in the processing that the errors appears. I'll post the error log a bit later. This as been tried on two different computers, with differents providers. One system had 2 gigs mem and a 4gig swap, just to be on the safe side. We use the latest stable Python builds, 2.5.2 and 2.6nneonneo wrote:Is Python giving memory errors? Or is it the SQL import?
If it is the SQL import, then it is not hard to fix; use the split_sql.py program (put it in the same place as the .sql files) to split the file into smaller chunks.
If it is Python, then I would be surprised, since I've seen the converter handle 900,000 post forums.
It fails ine the middle of processing.Topic ID 163
Traceback (most recent call last):
File "C:\Converters\InvisionFree\topics_posts_phpBB.py", line 260, in <module>
processed_data=re.sub(search,repl,processed_data)
File "C:\Python25\lib\re.py", line 150, in sub
return _compile(pattern, 0).sub(repl, string, count)
MemoryError
Code: Select all
pagedata=download_page(URL+'?showtopic='+str(topicid),progressline, COOKIEDATA)
pages=re.findall(re_pages,pagedata)
if pages:
perpage=int(pages[0][0])
np=int(pages[0][1])
for offset in range(1,np):
progressline=statusline+"Downloading Extra Page %i of %i - "%(offset,np-1)
pagedata+=download_page(URL+'?showtopic='+str(topicid)+'&st='+str(offset*perpage),progressline, COOKIEDATA)
processed_data=pagedata
cn=0
for search,repl in re_repstrings:
cn+=1
printline(statusline+"Processing Step %i of %i"%(cn,len(re_repstrings)))
# file("%i_%i.txt"%(topicid,cn),"w").write(processed_data)
processed_data=re.sub(search,repl,processed_data)
Code: Select all
pagedata=download_page(URL+'?showtopic='+str(topicid),progressline, COOKIEDATA)
processed_data=pagedata
cn=0
for search,repl in re_repstrings:
cn+=1
printline(statusline+"Processing Step %i of %i"%(cn,len(re_repstrings)))
processed_data=re.sub(search,repl,processed_data)
pages=re.findall(re_pages,pagedata)
if pages:
perpage=int(pages[0][0])
np=int(pages[0][1])
for offset in range(1,np):
progressline=statusline+"Downloading Extra Page %i of %i - "%(offset,np-1)
pagedata=download_page(URL+'?showtopic='+str(topicid)+'&st='+str(offset*perpage),progressline, COOKIEDATA)
temp=pagedata
cn=0
for search,repl in re_repstrings:
cn+=1
printline(statusline+"Processing Step %i of %i"%(cn,len(re_repstrings)))
temp=re.sub(search,repl,temp)
processed_data+=temp
We don't allow solicitation on these forums, as per our forum rules. If you have any specific question or queries, we are happy to help youSkipper29 wrote:Hi,
i have a proboards account, and am wanting it to go on to a PHPbb 2 forum. it all looks rather complicated for me. anyone want to earn some $$/££ to do it for me?
Code: Select all
("""<title>%s - ([^<]+?)</title>"""%BOARDNAME,"""<^>TITLE<|>\\1</^>"""),
Code: Select all
("""<title>%s - ([^<]+?)</title>"""%re.escape(BOARDNAME),"""<^>TITLE<|>\\1</^>"""),