Larger database issues?

Converting from other board software? Good decision! Need help? Have a question about a convertor? Wish to offer a convertor package? Post here.
Locked
User avatar
WildestDrake
Registered User
Posts: 13
Joined: Sun Feb 07, 2016 11:49 pm
Location: Denmark

Larger database issues?

Post by WildestDrake » Mon May 30, 2016 3:35 am

Hi.
So I have a larger community.
We're currently on 3.0.14

Our Database is about 1.1 GB

I'm well aware of this guide: https://www.phpbb.com/support/docs/en/3 ... /upgrade3/

But. Before I commit to the update process, and risk screwing up the database (and naturally just throwing back up the backup that would've been made before any changes would have been committed.)

Note:
I do not have access to any commands, as the guide explains for larger boards as I use One.com :
For large boards, you may wish to update via the command line. From your board's root, execute the following command: php ./bin/phpbbcli.php db:migrate --safe-mode
Is there any risks, or results I can expect from going through the regular process (visiting /install/database_update.php)
Will it work? Or is it a fruitless endevor?

Is there another way to do it with a large database?

I've been attempting to back up the database, and import it onto a different database (I have a few) and copy over the site files. In order to test it out myself. But so far the entire process on that project to test it out has been fruitless (yes I updated the config file to read the other database name and password, etc. Although I'll admit I unsure what I did wrong, and will continue to test it out. Hopefully finding a solution sometime soon..)
Last edited by WildestDrake on Mon May 30, 2016 9:58 am, edited 1 time in total.
Image

janus_zonstraal
Registered User
Posts: 2610
Joined: Sat Aug 30, 2014 1:30 pm

Re: Larger database issues?

Post by janus_zonstraal » Mon May 30, 2016 9:20 am

But so far the entire process on that project to test it out has been fruitless
Tell us where it go wrong and errors you see will help us.
Sorry! My English is bat ;) !!!

User avatar
WildestDrake
Registered User
Posts: 13
Joined: Sun Feb 07, 2016 11:49 pm
Location: Denmark

Re: Larger database issues?

Post by WildestDrake » Mon May 30, 2016 9:54 am

janus_zonstraal wrote:
But so far the entire process on that project to test it out has been fruitless
Tell us where it go wrong and errors you see will help us.
That's just not the main subject though.
But sure.

So far I've exported the entire database through phpmyadmin.
Downloaded all site files. - deleted unimportant large files (which were apparently some old backups)
Dropped the old database on the test site
Imported the new database after I split it up via SQL dump splitter
Uploaded the site files.
Replaced the config.php, with relevant database information.

The first set of errors I got were connection resets.
But now I'm getting errors on missing tables. So I'll look into trying again on the import to the database.
Image

User avatar
Ger
Recognised Extension Developer
Posts: 1665
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Larger database issues?

Post by Ger » Mon May 30, 2016 10:16 am

With databases that size, I would recommend against phpMyAdmin. Instead I suggest a tool like MySQL Workbench (my personal favorit) or Heidi SQL (also very powerful).
You can export and import you databases without the need to split them to make phpMyAdmin happy.

Splitting SQL files and uploading them as several files can work, but it's just an extra step with potential errors. Maybe with MySQL Workbench you will get a proper export and import, thereby preventing missing tables.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

User avatar
WildestDrake
Registered User
Posts: 13
Joined: Sun Feb 07, 2016 11:49 pm
Location: Denmark

Re: Larger database issues?

Post by WildestDrake » Mon May 30, 2016 10:19 am

Ger wrote:With databases that size, I would recommend against phpMyAdmin. Instead I suggest a tool like MySQL Workbench (my personal favorit) or Heidi SQL (also very powerful).
You can export and import you databases without the need to split them to make phpMyAdmin happy.

Splitting SQL files and uploading them as several files can work, but it's just an extra step with potential errors. Maybe with MySQL Workbench you will get a proper export and import, thereby preventing missing tables.
Cool. Thanks a lot.
I'll look into it.
Image

User avatar
WildestDrake
Registered User
Posts: 13
Joined: Sun Feb 07, 2016 11:49 pm
Location: Denmark

Re: Larger database issues?

Post by WildestDrake » Wed Jun 01, 2016 1:57 am

Successfully transferred the whole database, got the forum working.
I then began the process.

I got about 3 errors, of certain tables going kaput.
I was too hasty and in the excitement ignored the errors and just assumed it was a temporary thing.
But as it turns out, once the whole database update went through, a lot of content was missing. ergo tables that had nothing in them.

Threads that read that there were 6 replies to, there were 2.
Or there were 6 replies and the entire thread was gone.

So I'm gonna try and go back. Revert the whole database and site files and retry.
This time go through it all step by step.
Screenshot each error, and follow the instructions in them.

We'll see how it goes after that.

But except for the errors, it actually turned out a-okay.
A lot of the content were converted successfully.
But obviously it's the posts (the biggest table) that gets all the errors.
Image

User avatar
Ger
Recognised Extension Developer
Posts: 1665
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Larger database issues?

Post by Ger » Wed Jun 01, 2016 9:06 am

WildestDrake wrote:Successfully transferred the whole database, got the forum working.
Threads that read that there were 6 replies to, there were 2.
Or there were 6 replies and the entire thread was gone.
Have you checked those topics before the upgrade process? Were they OK at that time?
Also: are you sure it's in the backup at all?

If it is OK before the update and not after the update, than we know where to search. You should probably find a way to use de command line interface (CLI) to make sure all the posts get converted properly. Are you sure you don't have command line access? Have you checked the control panel at your hosting (DirectAdmin/cPanel/Plesk/..)
I'm not really familiar with One.com but if CLI is completely disabled maybe you can get their support people to execute the given command.

Edit:
I find this on One.com: https://www.one.com/en/support/faq/how- ... -web-space
Looks like you can get shell access, but need to activate it first. After that, you can use Putty (on Windows) or the terminal (Mac/Linux) to login.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

User avatar
WildestDrake
Registered User
Posts: 13
Joined: Sun Feb 07, 2016 11:49 pm
Location: Denmark

Re: Larger database issues?

Post by WildestDrake » Wed Jun 01, 2016 10:37 am

I'll be honest.
I had no idea what SSH was when I saw it on the control panel.
But knowing that's shell access, enabling commands and such, really gives comfort.

Now I just figure out where to put in the commands, and such.
Thanks a lot.
Image

User avatar
Ger
Recognised Extension Developer
Posts: 1665
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Larger database issues?

Post by Ger » Wed Jun 01, 2016 11:13 am

You're welcome. :)

1 tip though: Shell access gives you great power. And with great power comes great responsibility, as a wise man once said ;)
Don't do anything you're not comfortable with. When in doubt, search the internet for documentation or ask. And try it on you testforum first.

Good luck!
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

Locked

Return to “[3.1.x] Convertors”

Who is online

Users browsing this forum: No registered users and 2 guests