Trransferring Files To New Host

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Scam Warning
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Trransferring Files To New Host

Post by aeneas1 »

so i backed up all of my site files to my pc, then uploaded them to my new host, but during the upload process i keep getting messages that the file exists (in what was an empty folder before the upload began) along with options to overwrite, overwrite if source is newer than target, etc., etc..... so my question is how can these files already exist? thanks!
User avatar
EA117
Registered User
Posts: 2158
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Trransferring Files To New Host

Post by EA117 »

I haven't "proven" that it's true, but when I've seen that happen (upload to empty directory, and one or more files inexplicably prompt for overwrite) I've assumed it's because the FTP connection timed out & the automatic retry then finds the previously aborted/empty file.

When you're transferring from a case-sensitive file system to a non-case-sensitive file system (e.g. when downloading from a Linux server to a local Windows hard drive), technically you could encounter a "real issue" in that case. For example if someone named files "Test", "test" and "TEST", since when trying to download those to a case-insensitive Windows file system, they all overwrite the same file on the destination.

But I'm not aware of anything in phpBB that tries to use same-name-different-case filenames, and don't expect to encounter that issue backing up or making a local copy of a phpBB site.
User avatar
2600
I've Been Banned!
Posts: 2567
Joined: Fri Nov 14, 2014 5:14 pm
Location: Area-51

Re: Trransferring Files To New Host

Post by 2600 »

Are you just uploading what's in the public_html folder?
Morpheus: Unfortunately, no one can be told what The Matrix is. You'll have to see it for yourself.
Hack me.
Consider a canary token.
The nature of my chosen username
:ugeek:
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Re: Trransferring Files To New Host

Post by aeneas1 »

EA117 wrote: Fri Dec 07, 2018 5:10 am I haven't "proven" that it's true, but when I've seen that happen (upload to empty directory, and one or more files inexplicably prompt for overwrite) I've assumed it's because the FTP connection timed out & the automatic retry then finds the previously aborted/empty file.

When you're transferring from a case-sensitive file system to a non-case-sensitive file system (e.g. when downloading from a Linux server to a local Windows hard drive), technically you could encounter a "real issue" in that case. For example if someone named files "Test", "test" and "TEST", since when trying to download those to a case-insensitive Windows file system, they all overwrite the same file on the destination.

But I'm not aware of anything in phpBB that tries to use same-name-different-case filenames, and don't expect to encounter that issue backing up or making a local copy of a phpBB site.
this would make sense but the file name is always exactly the same, the target/source dates are different (but always the same difference), and the file sizes are often different too, sometimes the "existing" target file has 0 bytes whereas the source file size is correct... to be clear, we're not talking about this happening to 20% or so of the uploaded files, it's only a tiny amount, only a couple of dozen max or so....
John connor wrote: Are you just uploading what's in the public_html folder?
yes.
Holger
Registered User
Posts: 1883
Joined: Tue Mar 12, 2002 3:54 pm
Location: Hannover

Re: Trransferring Files To New Host

Post by Holger »

Happens me also sometimes. Just click on "Yes for all" and let the FTP continue the upload.
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Re: Trransferring Files To New Host

Post by aeneas1 »

Holger wrote: Fri Dec 07, 2018 7:34 am Happens me also sometimes. Just click on "Yes for all" and let the FTP continue the upload.
before setting "yes for all" you have to decide what action you want taken, i.e. overwrite, overwrite if source is newer, overwrite if source is larger, etc., etc., there's at least 10 options iirc... this can be a prob because the message pops up for different scenarios - for example, do i really want to check "overwrite if source is newer" and "yes for all" when the first message to pop up shows a newer source file with 0 bytes and an older target file with 2,863 kb?
User avatar
Lumpy Burgertushie
Registered User
Posts: 69223
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: Trransferring Files To New Host

Post by Lumpy Burgertushie »

try a different ftp program.

also, since the new files do not really already exist on the server, just say yes to overwrite everything all the time.

then it is up to you to be sure that is what you want everytime you upload a file.

remember, the overwrite will only overwrite files of the exact same name. the date does not matter.
uploading a folder of the same name will not overwrite the existing folder, it will just merge the contents of the new folder with the old.

that means that any files of the same name inside the folder will be overwritten but any files that do not have a duplicate will just be uploaded/left alone.


robert
Premium phpBB 3.3 Styles by PlanetStyles.net

I am pleased to announce that I have completed the first item on my bucket list. I have the bucket.
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Re: Trransferring Files To New Host

Post by aeneas1 »

Lumpy Burgertushie wrote: Fri Dec 07, 2018 4:02 pm > also, since the new files do not really already exist on the server, just say yes to overwrite everything all the time.
> remember, the overwrite will only overwrite files of the exact same name. the date does not matter.
thanks robert, that's what i ended up doing, except for a couple of instances when the target file had content and the source file had 0 bytes, just didn't seem right to overwrite a file with content with one with no content, know what i mean? also, every instance of the popup message showed identical file names, with same case, just different dates and, on some occasions, different files size.... whatever the case, everything seems fine.
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Re: Trransferring Files To New Host

Post by aeneas1 »

well didn't take long for the issue to rear its ugly head.... tried to look at a pm in my inbox and received a fatal error when clicking the pm link... turns out the /includes/ucp/ucp_pm_viewmessage.php file contained 0 bytes, opened it in editor and sure enough it was blank... looked at the source ucp_pm_viewmessage.php file in my backup folder (which i used to upload to my new host) and the file was 15.2kb in size... so i deleted the 0 byte file and re-uploaded the other file, all is fine now... of course it makes me wonder how many other files were effected during the upload process, during the "overwrite?" process.
User avatar
EA117
Registered User
Posts: 2158
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Trransferring Files To New Host

Post by EA117 »

I don't have anything that lets me do it "on the server itself", but I frequently download the files currently on the web site to a local directory, and then use a diff tool to compare the entire subdirectory structure against some previous backup and/or a known-source extraction like a phpBB release. (Any diff tool you prefer is fine; my preference is Scooter Software's BeyondCompare.)

So while it won't solve or answer any mysteries about why the overwrite issue is happening, this approach would reveal for you now "which file(s) on my site are zero length or wrong content" as compared to your backup. So that you could correct by uploading & intentionally overwriting just those files.
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Re: Trransferring Files To New Host

Post by aeneas1 »

EA117 wrote: Sat Dec 08, 2018 2:28 pm I don't have anything that lets me do it "on the server itself", but I frequently download the files currently on the web site to a local directory, and then use a diff tool to compare the entire subdirectory structure against some previous backup and/or a known-source extraction like a phpBB release. (Any diff tool you prefer is fine; my preference is Scooter Software's BeyondCompare.)

So while it won't solve or answer any mysteries about why the overwrite issue is happening, this approach would reveal for you now "which file(s) on my site are zero length or wrong content" as compared to your backup. So that you could correct by uploading & intentionally overwriting just those files.
unfortunately that doesn't really help with he prob i'm describing because the "file already exists > overwrite? > overwrite options" happens in both directions, when downloading (backing up) your phpbb files to a local disk, and again when uploading them to a server.

seems that the best was to ensure you're getting the exact files on your server, without having to guess whether you should overwrite an "existing file" or not, is to zip the phpbb directory at the server and download 1 (zipped) file, and then do the same when uploading, is that even possible? can you unzip on the server?

anyway i use winmerge to compare directories / files but, again, who's to say that the downloaded files you backed up, the ones that forced you to choose an overwrite option, are even the right files when they show as different during the file comparison process?
Nick225
Registered User
Posts: 131
Joined: Sat Nov 24, 2018 7:48 pm

Re: Trransferring Files To New Host

Post by Nick225 »

aeneas1 wrote: Fri Dec 07, 2018 3:41 am so i backed up all of my site files to my pc, then uploaded them to my new host, but during the upload process i keep getting messages that the file exists (in what was an empty folder before the upload began) along with options to overwrite, overwrite if source is newer than target, etc., etc..... so my question is how can these files already exist? thanks!
Those files exists probably because you have not changed your DNS to your new host. So you are still pointing to your old forum.
If you are using Cpanel, click on "Manage", right by your domain name specified at the top of the "Account Setting" page.
You will see the IP address for your site. Make sure that's the IP of your new host.
I bet that's your problem.
User avatar
EA117
Registered User
Posts: 2158
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Trransferring Files To New Host

Post by EA117 »

aeneas1 wrote: Sat Dec 08, 2018 3:25 pm unfortunately that doesn't really help with he prob i'm describing because the "file already exists > overwrite? > overwrite options" happens in both directions, when downloading (backing up) your phpbb files to a local disk, and again when uploading them to a server.
Okay, understood. The original post here said "during the upload process", and that's when I've encountered this kind of issue too, so I was expecting that your download would represent what was currently on the server.

What is the FTP transfer program being used currently?
aeneas1 wrote: Sat Dec 08, 2018 3:25 pm seems that the best was to ensure you're getting the exact files on your server, without having to guess whether you should overwrite an "existing file" or not, is to zip the phpbb directory at the server and download 1 (zipped) file, and then do the same when uploading, is that even possible? can you unzip on the server?
Sure, if you have console access to the server, you could use the Linux-provided compression and extract tools. (Or Windows-compatible, if your hosting is Windows-based.) Even if you don't have console access to the server, the CPanel, Plex or other online tools provided by your hosting provider may still give you access to an option. For example, in my CPanel the "File Manager" provided for my shared hosting provides access to the "Compress" and "Extract" functions.

We're "ignoring the problem" of course, and a "random failure of FTP transfer" is going to bite you again sooner or later for something else. But transferring only a single pre-compressed file might at least help solve problems in the short term.

Perhaps once we know what current FTP programs you're using & have tried, we'll figure out some kind of logging that can be enabled in the FTP client to confirm exactly what it might have already done with the file in question, and encountered on the FTP connection of that original file, before then ultimately making the attempt which results in an overwrite prompt.
aeneas1
Registered User
Posts: 451
Joined: Wed Jan 27, 2016 5:58 pm

Re: Trransferring Files To New Host

Post by aeneas1 »

Nick225 wrote: Sun Dec 09, 2018 12:59 amThose files exists probably because you have not changed your DNS to your new host. So you are still pointing to your old forum. If you are using Cpanel, click on "Manage", right by your domain name specified at the top of the "Account Setting" page. You will see the IP address for your site. Make sure that's the IP of your new host. I bet that's your problem.
thanks for the reply but that was one of the first thing i looked at, ip is the new host.
EA117 wrote:Sure, if you have console access to the server, you could use the Linux-provided compression and extract tools. Even if you don't have console access to the server, the CPanel or other online tools provided by your hosting provider may still give you this option. For example, in my CPanel the "File Manager" provided for my shared hosting provides access to the "Compress" and "Extract" functions.

We're "ignoring the problem" of course, and a "random failure of FTP transfer" is going to bite you again sooner or later for something else. But transferring only a single pre-compressed file might at least help solve problems in the short term.

Perhaps once we know what current FTP programs you're using & have tried, we'll figure out some kind of logging that can be enabled in the FTP client to confirm exactly what it might have already done with the file in question, and encountered on the FTP connection of that original file, before then ultimately making the attempt which results in an overwrite prompt.
yes, discovered how to gz zip and uncompress forum files (everything in the public_html folder) at the server level, will probably re-do everything using this process... i'm wondering how it will handle attachment files? when i backed things up the first time around, i didn't uncheck "treat files without extensions as ascii files" in filezilla's transfer settings, as a result they were corrupted and many of the attachments images didn't display after the migration, re-downloaded them after making the filezilla change, reuploaded the, and everything was fine.

and, yes, it's certainly doesn't fix the transfer prob, but it seems to be a solid work around for re-locations, i.e. big transfer, of course when you want to just edit a few files, or a folder of files, and don't want to zip them, it will continue to be a prob... btw i use filezilla and coffecup direct ftp, both have issues with large transfers, but coffeecup is an excellent live editor/updater, much better than notepad++ imo.
Nick225
Registered User
Posts: 131
Joined: Sat Nov 24, 2018 7:48 pm

Re: Trransferring Files To New Host

Post by Nick225 »

Ok.. got it. I don't see how the files could otherwise already exist on the server of your new host.
Mystery. I just moved to a different plan on godaddy and transferred everything to the new server.
My database is 400 Gig. I was able to download it and upload it through Cpanel Download/Upload utility. Pretty darn fast relatively to FTP.
Post Reply

Return to “phpBB Discussion”