Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Get help with installation and running phpBB 3.2.x here. Please do not post bug reports, feature requests, or extension related questions here.
BeBop
Registered User
Posts: 74
Joined: Tue Sep 05, 2017 1:07 pm

Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by BeBop »

Dear support and KevC,

Two old giffers back here again. We followed your advice KevC, rather than getting our hosting provider to roll back a PHP 7.2 update or two we upgraded phpBB board from version 3.2.2 to 3.2.7 yesterday.

http://www.973-eht-namuh-973.com/forum/ ... mode=login

While everything seemed to be working, one problem we have uncovered is that we appear to have lost every inline attachment image posted since the last upgrade from version 3.2.1 to 3.2.2 completed last year during the first week of September 2018.

We still have a copy of the system before the upgrade, is there a file that contains those attached images in the pre-uugrade copy mentioned that we failed to include and use in the upgrade to 3.2.7. If so can which get it back and install it in 3.2.7 or would that require doing the whole upgrade again.

Please advise

Thank you in advance and
kind regards
BeBop
Last edited by BeBop on Fri Jul 12, 2019 8:52 am, edited 1 time in total.
User avatar
janus_zonstraal
Registered User
Posts: 6418
Joined: Sat Aug 30, 2014 1:30 pm

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by janus_zonstraal »

What update method did you use?

Normally you can't loose the files but the are in the files folder.
Sorry! My English is bat ;) !!!
User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 52768
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by stevemaury »

Make sure the /files folder is CHMOD 777. I get a permission error when I try to browse to it.
I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)
BeBop
Registered User
Posts: 74
Joined: Tue Sep 05, 2017 1:07 pm

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by BeBop »

Thanks Janus and Steve.

We will check

Regards BeBop
robbell
Registered User
Posts: 20
Joined: Sun Feb 27, 2011 12:33 am

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by robbell »

Hi,

I'm the guy who updated the forum. I did it using the Automatic Update package and manual file uploads. I did the uploading from Dreamweaver CS3

The files folder is definitely 0777, I just had a look.

Any help you can give us will be greatly appreciated, as I'm completely stumped as to why the images have disappeared
User avatar
EA117
Registered User
Posts: 2158
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by EA117 »

stevemaury wrote: Fri Jul 12, 2019 2:32 pm Make sure the /files folder is CHMOD 777. I get a permission error when I try to browse to it.
Same is true on https://www.phpbb.com/community/files too, for what it's worth.
robbell wrote: Sat Jul 13, 2019 1:05 am Any help you can give us will be greatly appreciated, as I'm completely stumped as to why the images have disappeared
Since we don't know what's going on yet, can you expound on what "disappeared" actually means.

e.g. Is there just literally "nothing", or maybe a broken image artifact displayed, or maybe [img][/img] BBCodes being displayed instead of interpreted, etc.

If there is a publicly-accessible post that demonstrates this, maybe you can link to that. Or at least show a screen shot of what the failure looks like, if it can't be a public link.
User avatar
Lumpy Burgertushie
Registered User
Posts: 69223
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by Lumpy Burgertushie »

robbell wrote: Sat Jul 13, 2019 1:05 am Hi,

I'm the guy who updated the forum. I did it using the Automatic Update package and manual file uploads. I did the uploading from Dreamweaver CS3
either one or both of those things could be the cause of your problems.

unless you have been editing the core code of the php files there is no reason to use the auto update. it has its problems.

you should use the standard file replacement method mentioned in the instructions for updating/upgrading.

also, dreamweaver is not a good program to use for anything to do with phpbb. it is known to cause problems with phpbb.

use a good normal ftp program like filezilla to upload/download files keeping in mind that you MUST set it to transfer files with no extension in binary and not ascii. set it to binary for everything. it will not hurt any files like ascii will if they are images or pdfs etc.

use only a good text editor like notepad++ or editpad lite to open/edit the phpbb files.


for now, you need to check the actual images in the files folder to see if they are corrupted. add a jpg extension to a few of them and see if they open as images ok. then you can remove the jpg extension after you test.


robert
Premium phpBB 3.3 Styles by PlanetStyles.net

I am pleased to announce that I have completed the first item on my bucket list. I have the bucket.
User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 52768
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by stevemaury »

The files in the /files folder are not necessarily .jpg files. I you look in the database attachments table, you can correlate real filenames to physical filenames to determine which, if any, are .jpg files and test there viability by adding that extension and browsing to them.
I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)
BeBop
Registered User
Posts: 74
Joined: Tue Sep 05, 2017 1:07 pm

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by BeBop »

EA117 wrote: Sat Jul 13, 2019 1:38 am
stevemaury wrote: Fri Jul 12, 2019 2:32 pm Make sure the /files folder is CHMOD 777. I get a permission error when I try to browse to it.
Same is true on https://www.phpbb.com/community/files too, for what it's worth.
robbell wrote: Sat Jul 13, 2019 1:05 am Any help you can give us will be greatly appreciated, as I'm completely stumped as to why the images have disappeared
Since we don't know what's going on yet, can you expound on what "disappeared" actually means.

e.g. Is there just literally "nothing", or maybe a broken image artifact displayed, or maybe [img][/img] BBCodes being displayed instead of interpreted, etc.

If there is a publicly-accessible post that demonstrates this, maybe you can link to that. Or at least show a screen shot of what the failure looks like, if it can't be a public link.
Good evening EA117 further to your question please find below publically accessible post links to demonstrate the problem:

http://www.973-eht-namuh-973.com/forum/ ... =1&t=14820
http://www.973-eht-namuh-973.com/forum/ ... =1&t=14849

Small oblong window displayed in the post which contains the picture icon and the words “image.jpeg” or “image.png” which if you click on it goes instantly to a system message display “The selected attachment does not exist anymore.”

Regards BeBop
User avatar
EA117
Registered User
Posts: 2158
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by EA117 »

"In general" the behavior does seem consistent with "there are attachments in the database which aren't being found in the actual /files directory." What is actually seen when loading one of the problem pages is that trying to pull a thumbnail for one of the images will actually return status 500 (web server unable to complete request), while the others return a more normal 404 (not found) status. But it's not consistent which one will return 500 versus 404.

For example in the http://www.973-eht-namuh-973.com/forum/ ... =1&t=14820 thread, I can get 500 for file.php?id=4095 and 404 for file.php?id=4096 and all the others; but on a reload of the page, file.php?id=4096 may return 500 and file.php?id=4095 and all the others are the ones that return 404.

So maybe there is a clue in the web server and/or PHP error logs, which is being recorded during the 500 status event. I'm thinking of things like maybe there will be an error or call stack that implicates one of the additional extensions that are in use, or similar.

Aside from that, I agree with what Steve suggested too, regarding tracking down one of the "file ID to real file name in /files directory" mappings out of the database, and determining whether that file exists, whether that file has some kind of unique permission applied to it, whether it's content seems corrupt or maybe zero-length now, etc.
stevemaury wrote: The files in the /files folder are not necessarily .jpg files. I you look in the database attachments table, you can correlate real filenames to physical filenames to determine which, if any, are .jpg files and test there viability by adding that extension and browsing to them.
For example, in this screen shot here, I'm looking at attachment ID 1 out of the phpbb_attachments table, and can see that this attachment ID has a filename of "2_47ca94c7e0d7f7dd631a1a1b0b45c296", and the original filename was "sometext.png". So I would go verify the existence and security on /files/2_47ca94c7e0d7f7dd631a1a1b0b45c296, and then make a copy of that file and rename it to "sometext.png" and try to view the file to make sure it's not corrupt or empty.

attachment.png

In your case, you would have @robbell do that for at least attachment IDs 4095 and 4096 since we saw those as being affected, as well as any other affected attachment IDs you wanted to check.

What happens next probably depends on whether you find that the files are simply missing, or are all wrongly secured and preventing access, or are all corrupted, or all seem to be 100% fine, etc.
User avatar
canonknipser
Registered User
Posts: 2096
Joined: Thu Sep 08, 2011 4:16 am
Location: Germany
Name: Frank Jakobs
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by canonknipser »

For validating the attachment files, you can use this script to generate a copy having the original file names:
Knowledge Base - Backing up attachments with their original filenames
Even if the script was created for phpBB 3.0, it should still work with 3.2.
Greetings, Frank
phpbb.de support team member
English is not my native language - no support via PM or mail
New arrival - Extensions and scripts for phpBB
robbell
Registered User
Posts: 20
Joined: Sun Feb 27, 2011 12:33 am

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by robbell »

Hi,

I've tried to restore the database from our backup, but initially ran into problems where the backup couldn't be restored and an error showed up - fortunately someone else on this board had the same problem, and a fix was found and applied, which then allowed me to restore the backup, and we now have our users and posts back, but not the images.

I've also reuploaded the files directory, ensuring Filezilla was set to binary. This doesn't appear to have made any difference.

A question I find myself asking is why is there an Automatic Update package that phpbb recommend you use if it's not a good way to update the forum. It seems like they are asking for problems to arise... I quote the Auto Update page: "This is the package that most users should download to update their forums."

Anyway, can't do much about that.

The files in the files directory are not 0 length, they appear to be fine, although in phpadmin in the attachments table there are 2966 rows, whereas the files directory contains 5319 items... could this be relevant to the problems we're having?

There also appear to be a large number of orphaned attachments in the attachments table, could this be something to do with the problem?
User avatar
EA117
Registered User
Posts: 2158
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by EA117 »

robbell wrote: Tue Jul 16, 2019 1:45 pm I've also reuploaded the files directory, ensuring Filezilla was set to binary. This doesn't appear to have made any difference.
...
The files in the files directory are not 0 length, they appear to be fine...
I don't seem to be able to reproduce the "status 500 occurrs for one of the files" condition which was consistent before. Now all the failing images are consistently 404 to me, for what it's worth.

So to take a specific example, file.php?id=4095 again, you saw in the database what the "physical filename" value was set to. And you went to the /files directory and were able to find that specific physical filename. And that file was not zero length. And copying that file and renaming it to the correct original image name shows a valid and non-corrupt image.

What were the permissions on that specific file? (e.g. 666, 644, 777, etc.) Is the alphabetic case of the "physical filename" value in the database an exact match? (e.g. Not "2_47ca94c7e0d7f7dd631a1a1b0b45c296" versus "2_47CA94c7e0D7f7dd631a1a1B0B45C296" or similar.)

Again, we're just looking for any reason or excuse that "asking phpBB about attachment ID 4095" might not have been able to find or use the file described in the database for attachment ID 4095.

robbell wrote: Tue Jul 16, 2019 1:45 pm ...although in phpadmin in the attachments table there are 2966 rows, whereas the files directory contains 5319 items... could this be relevant to the problems we're having?
I think that's normal, because you're getting thumbnails generated (when the attachments are images) in addition to the actual original files.
robbell wrote: Tue Jul 16, 2019 1:45 pm There also appear to be a large number of orphaned attachments in the attachments table, could this be something to do with the problem?
Orphaned attachments aren't any kind of "problem" other than they're taking up space, and aren't "knowingly" linked to any message. e.g. Someone uploads a file during post creation and preview, but never actually submits the post.

Note its possible some savvy users take advantage of this, and actually reference these "orphaned" files from a different message "even though they're not considered attached to any post." So it's possible (but probably not common) that deleting orphaned attachments will actually delete in-use images. I personally would leave them there, but I expect board operators typically delete orphaned attachments to reclaim the space they represent.

But from phpBB's perspective that's "just another attachment and file ID", and is tracked no differently than any other attachment. i.e. No inherent technical problem that they exist, same and no different than any other in-use attachment.
User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 52768
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by stevemaury »

robbell wrote: Tue Jul 16, 2019 1:45 pm A question I find myself asking is why is there an Automatic Update package that phpbb recommend you use if it's not a good way to update the forum. It seems like they are asking for problems to arise... I quote the Auto Update page: "This is the package that most users should download to update their forums."
We stopped recommending the Auto Update packages as the preferred method several versions ago,
I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)
User avatar
warmweer
Jr. Extension Validator
Posts: 11234
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Bel ... gium
Contact:

Re: Done 3.2.2 to 3.2.7 upgrade but inline attachments missing

Post by warmweer »

robbell wrote: Tue Jul 16, 2019 1:45 pm A question I find myself asking is why is there an Automatic Update package that phpbb recommend you use if it's not a good way to update the forum. It seems like they are asking for problems to arise... I quote the Auto Update page: "This is the package that most users should download to update their forums."

https://www.phpbb.com/support/documents ... n=3#update

4.i. Full package
Updating using the full package is the recommended update method for boards without modifications to core phpBB files.
Considering that modifications (as they were used in 3.0) requires editing the affected files with each update, this won't work out of the box. If your board has no modifications then THIS is the one to use.

4.ii. Changed files
This package is meant for those wanting to only replace the files that were changed between a previous version and the latest version.
The same applies to this package and it works out of the box with an unmodified board. The advantage of this package is that the donwload is smaller and it allows immediate identification of files that need to be edited (and don't need to be edited) in case your board IS modified.

4.iii. Patch file
The patch file package is for those wanting to update through the patch application, and should only be used by those who are comfortable with it.
As mentioned: only to be used by those who are comfortable with it. If your board is unmodified, there's no point in using this package since the 2 packages above are already "patched" correctly. A patch application may work but one has to realise that the more a board is modified, the higher the risk that the patch application will have difficulties in finding the correct lines to delete, modify, add - this is because phpBB has no control over what was edited and cannot guarantee that user modifications will or even can work with a new version. ONLY USE THIS if you can sort out problems yourself, and if you use it, DO NOT use it on your host install but only on a localhost installation where you can easily test and debug if necessary.

4.iv. Automatic update package
This update method is only recommended for installations with modifications to core phpBB files. This package detects changed files automatically and merges in changes if needed.
The same remarks as for 4.iii. apply here, with the added possible hickup that in some cases it can go seriously haywire (example: files with length 0) because of the way the host FTP is set up (transfer mode). In fact almost all of the boards I've had to fix, were boards that were updated using autoupdate.

If 4.iii. or 4.iv. is used, better have a full backup (files and database) because the results are not always pleasing. ONLY DO THIS IF YOU ARE ARE CAPABLE OF SORTING OUT PROBLEMS YOURSELF, in which case I would choose to do the file modifications by hand anyway.
Having said this, these methods can work but the risk of something going wrong is implicitly a lot higher that with 4.i. and 4.ii. (where basically nothing can go wrong).

Should methods 4.i. and 4.ii. continue to be available? YES, the responsibility lays with the user. phpBB provides the tool but cannot guarantee it works for all situations, simply because the number of situations increases each day, and is dependent on the host setup. There is no way to tell beforehand whether it will work, even with all the testing phpBB does, on various setups.

Are the warnings given by phpBB sufficient?
That's a difficult one. In my opinion yes, but perhaps for some they aren't. A balance has to be found between the amount of positive reading material given (e.g. the simplicity of setting up and updating phpBB) and the amount of negative reading material (e.g. all the possible problems). IMHO for 4.iii. and 4.iv. more emphasis could be put on the requirement that the user can identify and fix the (possible) problems AND that if the problem is unsolveable, the user can resort to using the other methods of updating with his backup.

Apologies for the long read but I've spent too much time on fixing boards which were autoupdated by people who never bothered to read the manual (or had the bad luck that the host autoupdated their installation).

Added: Modifications are seldom needed anyway. Quite a few extensions are available and if not available, a request can always be made. And quite a few modifications are easily manually reintroduced by hand - I tried patch file once on 3.0.x and it took longer than doing it by hand (at least I know exactly what I'm editing and why).

added: this text was autopatched so I had to correct all the typo's editing mistakes by hand.
Spelling is freeware, which means you can use it for free.
On the other hand, it is not open source, which means you cannot change it or publish it in a modified form.


Time flies like an arrow, but fruit flies like a banana.
Post Reply

Return to “[3.2.x] Support Forum”