400,000 posts ... conversion will take hours

Converting from other board software? Good decision! Need help? Have a question about a convertor? Wish to offer a convertor package? Post here.
Suggested Hosts
whoopyYay
Registered User
Posts: 75
Joined: Thu May 24, 2007 10:20 pm

400,000 posts ... conversion will take hours

Post by whoopyYay »

Am running the conversion process from 2.0 to 3.1 and it seems to be taking around 3 minutes for each group of 6000 posts being converted. My posts table has over 400,000. Surely this can't be right, otherwise this is going to take hours. I read the conversion process is 10 minutes. :(
Luchtzak
Registered User
Posts: 273
Joined: Sat Jul 20, 2002 7:13 pm

Re: 400,000 posts ... conversion will take hours

Post by Luchtzak »

Then you haven't tried the search index yet, for my 5K message board it took already a few minutes! Just let the server do it's work, leave your pc on for a few hours and do a walk around the block. Helps for me all the time ;)
whoopyYay
Registered User
Posts: 75
Joined: Thu May 24, 2007 10:20 pm

Re: 400,000 posts ... conversion will take hours

Post by whoopyYay »

Ah so there is a search engine rebuild in phpbb3?
I'll wait, just frustrating as the time is fine if it would work, but I'm having problems with posts / users etc
missing after the conversion.
splitsch
Registered User
Posts: 31
Joined: Mon May 21, 2007 11:00 am
Location: Belgium :)

Re: 400,000 posts ... conversion will take hours

Post by splitsch »

Hello...maybe you could try to convert in local: your computer maybe have more ressources than the server.

For the index, you could empty all the tables, then convert, then rebuild the index (again, in local). It might be faster...I'm not sure rebuilding the index is possible, but I know it is un punbb (even though it would also take minutes, maybe hours!)

Bye
gesh13
Registered User
Posts: 45
Joined: Sat Dec 10, 2005 10:26 pm

Re: 400,000 posts ... conversion will take hours

Post by gesh13 »

The search index isn't rebuilt after converting. You must do it later via ACP
Visit my web site: Harry Potter 7
Amezis
Registered User
Posts: 116
Joined: Sun Jul 24, 2005 9:10 am
Location: Oslo, Norway

Re: 400,000 posts ... conversion will take hours

Post by Amezis »

In install/install_convert.php, scroll down to around line 60, where the variable $batch_size is set. You can change its value to 6000 or even higher (such as 10K if you believe your server is good enough). You can also increase the value in $num_wait_rows, but remember that increasing either of these values will put a higher load on your server and require more resources.

If it takes too long to rebuild the search index, it might be a good idea to increase the batch size there too. I do NOT know if this is the correct variable, and haven't tested this myself, but I'm quite sure that it should be correct anyway: includes/acp/acp_search.php - the variable is on line 20 and is also called $batch_size.

Hope that helps :)
Amezis

Wise men talk because they have something to say; fools, because they have to say something. -Plato

Return to “[3.0.x] Convertors”