Upgrade of v2 to v3 of a very large forum

Get help with installation and running phpBB 3.2.x here. Please do not post bug reports, feature requests, or extension related questions here.
rockedge
Registered User
Posts: 7
Joined: Sat Nov 30, 2019 9:10 pm

Upgrade of v2 to v3 of a very large forum

Post by rockedge » Sat Nov 30, 2019 9:18 pm

Hello all,

I am interested in opinions of the chance of success in upgrading a version 2+ to the latest v3.2 of a large to very large forum.

There are registration problems appearing where the registration emails are not sent or delivered. This is at this time only an inquiry before any actual action is taken. There may be around a million posts in the database.

thanks for any advice !

User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 50926
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Upgrade of v2 to v3 of a very large forum

Post by stevemaury » Sat Nov 30, 2019 9:50 pm

The chances of success are 100% if it is done right. Without further information that is all I can say.

Please fill out the Support Request Template and post it back here to enable us to assist you better.
For REALLY good and VERY inexpensive hosting CLICK HERE

I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)

rockedge
Registered User
Posts: 7
Joined: Sat Nov 30, 2019 9:10 pm

Re: Upgrade of v2 to v3 of a very large forum

Post by rockedge » Sat Nov 30, 2019 10:42 pm

Thank you! I will fill out a support request form. At this time the discussion has begun as how to move forward.
The forum in question is the Puppy Linux Forum with 1031023 articles and 41284 registered users here:
http://www.murga-linux.com/puppy/index.php

once again thank you, and for the quick response!

User avatar
janus_zonstraal
Registered User
Posts: 3805
Joined: Sat Aug 30, 2014 1:30 pm

Re: Upgrade of v2 to v3 of a very large forum

Post by janus_zonstraal » Sun Dec 01, 2019 7:35 am

Start with a copy of your board to test the conversion.
And than follow these steps.
https://www.phpbb.com/support/docs/en/3 ... onversion/
Sorry! My English is bat ;) !!!

rockedge
Registered User
Posts: 7
Joined: Sat Nov 30, 2019 9:10 pm

Re: Upgrade of v2 to v3 of a very large forum

Post by rockedge » Tue Dec 03, 2019 5:02 pm

Hello janus_zonstraal,

thanks for the tip! We will be migrating from one host server to another. I have a test site up and running using the latest version of phpBB on a completely different host server than the original forum.

I will read the docs!

rockedge
Registered User
Posts: 7
Joined: Sat Nov 30, 2019 9:10 pm

Re: Upgrade of v2 to v3 of a very large forum

Post by rockedge » Tue Dec 03, 2019 5:11 pm

Main question now is how to convert with the 2.+ to the 3.2 forum on different host servers?

with a over a million posts and 44000 or more users the sql export may be extensive. The new server runs PHP 7.2+ so I am not sure we can copy and install the original forum from here -> http://murga-linux.com/puppy/ over to this server -> https://puppylinux.rockedge.org/index.php

to run the conversion scripts.

User avatar
warmweer
Registered User
Posts: 3216
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Belt ... well actually Belgium

Re: Upgrade of v2 to v3 of a very large forum

Post by warmweer » Tue Dec 03, 2019 6:28 pm

rockedge wrote:
Tue Dec 03, 2019 5:11 pm
Main question now is how to convert with the 2.+ to the 3.2 forum on different host servers?

with a over a million posts and 44000 or more users the sql export may be extensive. The new server runs PHP 7.2+ so I am not sure we can copy and install the original forum from here -> http://murga-linux.com/puppy/ over to this server -> https://puppylinux.rockedge.org/index.php

to run the conversion scripts.
Split the database export in manageable parts. (e.g., for the search tables you only need the table definitions, the data can be rebuilt after the upgrade, backup the posts_table seperately, topics_track_table also, just see which are the large table, a table can also be split e.g. records 1-500.000 and records 500.001 to the end).)
Do it on a personal webserver so that you can really increase memory limit, max_execution time, etc... yourself.
Don't use something like USBwebserver (too weak) but WAMP XXAMP LAMP or DEVServer (EAsyphp) with which you can also use CLI.
The year is 2192. The British Prime Minister visits Brussels to ask for an extension of the Brexit deadline. No one remembers where this tradition originated, but every year it attracts many tourists from all over the world.

User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 3405
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Upgrade of v2 to v3 of a very large forum

Post by thecoalman » Tue Dec 03, 2019 7:24 pm

rockedge wrote:
Tue Dec 03, 2019 5:11 pm
Main question now is how to convert with the 2.+ to the 3.2 forum on different host servers?
I'm assuming you are using SSH for the basic database operations? e.g. dumping it on one server and populating it on the other. That's going to eliminate most issues people encounter with large conversions.

The issues you may encounter are going to be php versions and possibly php limitations. The php version only needs to support what you are upgrading too so it only becomes a concern if you encounter problems converting directly to 3.2. The php limitations are what they are and with forum that large I'm going to assume you'll be able to temporarily override those.

One big question is do you have the attachment mod installed? That is supported for the conversion from 2.X to 3.0 but if I recall correctly there was some specifics involved with that. I don't know how the converter in 3.2 handles it.

In any event swing for the fences.... populate the database on the new server with copy of the old forum databse. Open up config.php on your test install, change the prefix for the database if applicable. Upload the install folder and point your browser to the install folder. See what happens.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison

rockedge
Registered User
Posts: 7
Joined: Sat Nov 30, 2019 9:10 pm

Re: Upgrade of v2 to v3 of a very large forum

Post by rockedge » Tue Dec 03, 2019 9:08 pm

Thanks for the info guys!
Fortunately I have access to several pretty robust personal web servers I can run the test conversion on. In any case I would be splitting the sql export dump into several pieces seeing as the host server for the Puppy Linux forum is currently in Australia and the test servers are in the eastern United States with the final phpBB forum once a successful conversion is done on a personal server that I can max out the limits and tweak what needs to be tweaked, will be in the UK.

So data transfers will have to be done directly to the personal servers I have in Connecticut, USA from Australia

I really appreciate the help. And also from the entire Puppy Linux community, a thank you, and who are watching with great interest!

User avatar
warmweer
Registered User
Posts: 3216
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Belt ... well actually Belgium

Re: Upgrade of v2 to v3 of a very large forum

Post by warmweer » Tue Dec 03, 2019 9:26 pm

rockedge wrote:
Tue Dec 03, 2019 9:08 pm
I really appreciate the help. And also from the entire Puppy Linux community, a thank you, and who are watching with great interest!
Watching this topic. Do let us know how you fared.
The year is 2192. The British Prime Minister visits Brussels to ask for an extension of the Brexit deadline. No one remembers where this tradition originated, but every year it attracts many tourists from all over the world.

KYPREO
Registered User
Posts: 162
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Upgrade of v2 to v3 of a very large forum

Post by KYPREO » Tue Dec 03, 2019 9:29 pm

Is that really necessary to split up the SQL dump?

I am in Australia too, hosting in the US, with a forum database of 2 million posts and 45,000 users the dumps are only 2.5GB uncompressed (6GB once restored into the database). A GZip compressed backup is only around 700kb. This is including the search index, which is approximately half the database. This can be easily moved between servers.

I did the v3 to v3.2 conversion with minimal issues (just needed to tweak the execution time and memory limits of PHP). The database restore and phpBB upgrade was all done on the US based server directly via SSH. We did the v2 to v3 jump 10 years ago but that was done directly on the server as well (which at the time was a fairly low spec dedicated server).

Restoring a mySQL database of that size can be done in a single dump, provided you use the mysql CLI. The longest part of the process is rebuilding the search index. It takes a long time when using phpBB Fulltext native. However, to be honest, after switching my test server to Sphinx search engine, I would never use the fulltext native search on a forum with a million plus posts. The Sphinx search index takes 5 minutes to build, compared to several days for phpBB native when we first migrated from phpBB v2 to v3.
phpBB user since 2002
www.AusRotary.com

rockedge
Registered User
Posts: 7
Joined: Sat Nov 30, 2019 9:10 pm

Re: Upgrade of v2 to v3 of a very large forum

Post by rockedge » Wed Dec 04, 2019 1:32 am

Hello KYPREO,

I think that is what I am going to do and transfer the sql dump in one file. On the local server I can use mysql cli to import the sql file.
What about the conversion to the v3 database from the v2? Also the attachments and images need to be transferred.

It is good to hear your ideas, I've migrated other sites across servers but this a first with phpBB but your words are encouraging.

KYPREO
Registered User
Posts: 162
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Upgrade of v2 to v3 of a very large forum

Post by KYPREO » Wed Dec 04, 2019 2:07 am

It's been 10 years since the v2 to v3 upgrade was done so I can't tell you much about that part, except to suggest doing a test run on localhost first.

I didn't think phpBB v2 had an attachment system? If yours does have attachments, then the biggest trap is transferring the files via FTP using ASCII mode (which is the default in Filezilla, for example, when transferring files that do not have an extension, such as phpBB attachment files). Personally, I zip up the entire contents of the website and transfer that as a single file and it solves any binary vs ASCII mode issues.

I just visited your board (it was great to see a phpBB v2 board in action - it brought back many memories!) and it does have attachments. I suppose that was done through a modification?

One of the challenges for a heavily modified board will likely be ensuring that functionality is carried over through to the current version.
phpBB user since 2002
www.AusRotary.com

User avatar
3Di
Former Team Member
Posts: 14487
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: Upgrade of v2 to v3 of a very large forum

Post by 3Di » Wed Dec 04, 2019 2:22 am

KYPREO wrote:
Wed Dec 04, 2019 2:07 am
and it does have attachments. I suppose that was done through a modification?
Ofcourse, by the "attachments MOD of Acyd Burn", and the converter takes care of them automatically.
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity º PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
✒️ Black Friday 2019 @ The Studio ▪️◾️

User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 3405
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Upgrade of v2 to v3 of a very large forum

Post by thecoalman » Wed Dec 04, 2019 9:13 am

rockedge wrote:
Wed Dec 04, 2019 1:32 am
Also the attachments and images need to be transferred.
If it's large amount of files the easiest thing to do is zip them on the old server in public directory and use wget on new server to retrieve the file. There is of course more secure ways to do this if that is a concern. Unzip or upload into the files folders, e.g. individual files should be in that directory. Make sure the owner and file permissions are correct after unzipping. Use them on your test conversions, you can get any stray attachments uploaded afterward along with avatars using FTP.

As noted attachments is a mod and it is supported for conversion however I have no idea how or even if 3.2 handles it. Perhaps someone can chime in if they know.(I see 3di mention the converter still supports it). The only other mod that was supported for conversion that I'm aware of was the birthday mod and they only included that because of COPPA. Worse case is you would need to convert to 3.0 first. Here's link if you need it.

https://download.phpbb.com/pub/release/3.0/3.0.0/
KYPREO wrote:
Wed Dec 04, 2019 2:07 am
If yours does have attachments, then the biggest trap is transferring the files via FTP using ASCII mode (which is the default in Filezilla, for example, when transferring files that do not have an extension, such as phpBB attachment files). Personally, I zip up the entire contents of the website and transfer that as a single file and it solves any binary vs ASCII mode issues.
This actually should not be problem because the files were named exactly as uploaded including the extension. :D The oldest attachments I have are still this way.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison

Post Reply

Return to “[3.2.x] Support Forum”