Uploading attachments issues

Get help with installation and running phpBB 3.0.x here. Please do not post bug reports, feature requests, or MOD-related questions here.
Ideas Centre
Forum rules
END OF SUPPORT: 1 January 2017 (announcement)
User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 9:58 am

Support Request Template
What version of phpBB are you using? phpBB 3.0.11
What is your board's URL? http://www.nordisk-forum.dk
Who do you host your board with? fusioned.net
How did you install your board? I used the download package from phpBB.com
What is the most recent action performed on your board? Update from a previous version of phpBB3
Is registration required to reproduce this issue? No
Do you have any MODs installed? Yes
What version of phpBB3 did you update from? phpBB 3.0.10
What MODs do you have installed? https://www.phpbb.com/customise/db/mod/thanks_for_posts
https://www.phpbb.com/customise/db/mod/nv_who_was_here
https://www.phpbb.com/customise/db/mod/prime_links
Tapatalk
What styles do you currently have installed? subsilver2_new_colors
What language(s) is your board currently using? Danish
Which database type/version are you using? MySQL 5
What is your level of experience? Experienced with PHP and phpBB
When did your problem begin? After the move to a new host
Please describe your problem. 1: On the old hosts, all the attachments works fine.

2: moved to new hoss. 9998 attachments downloaed and uploaded in binary mode.
rest normally. database extracted via phpmyadmin.

3: uploaded both to a local lamp setup and tested everythig prior to remote upload. Both setups seems to work just fine. exepct that only about half of the attachments (all images) does not show.resyncing the attachments in stk didnt do any difference.

Example of problem:

http://www.nordisk-forum.dk/viewtopic.php?f=49&t=43847

Short: only half of images (attachments) show up after a move. triple checked that it was done by the book. What should i try next ?
Generated by SRT Generator
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

User avatar
Oyabun1
Former Team Member
Posts: 23162
Joined: Sun May 17, 2009 1:05 pm
Location: Australia
Name: Bill

Re: Uploading attachments issues

Post by Oyabun1 » Fri Mar 22, 2013 11:17 am

You could use the script in this article, Backing up attachments with their original filenames, to backup the files and then you could see if the images open directly from the server and when downloaded. That should give an indication of whether the problems is with the files.
                      Support Request Template
3.0.x: Knowledge Base Styles Support MOD Requests
3.1.x: Knowledge BaseStyles SupportExtension Requests

User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Re: Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 12:07 pm

Thx. here is how it went.

This was run on my localhost with a copy of the board. Ran rename.php

It told me that there was 30000+ attachments to fetch, but there is only 9998 files in /files/. Mmm onwards. changed the interval to 1500. when done i had 9998 attachments in filebackup all read able. So thats good.

Now for the questions:

1: All the files are valid, no corrupt after this script extracted them. But why did it belive there was 30K images and not 10K ?

2: What now ? The attachments on the board is still not showing in most post. about 50-60%. anyway to fix this besides adding each image manually to the posts again ?
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

User avatar
Oyabun1
Former Team Member
Posts: 23162
Joined: Sun May 17, 2009 1:05 pm
Location: Australia
Name: Bill

Re: Uploading attachments issues

Post by Oyabun1 » Fri Mar 22, 2013 12:38 pm

DoYouSpeakWak wrote:Only very few files in /files/ are non cryptical in name. most are just called TSGR35RETE45364 etc. no names.jpg or anything.
All the files in there should have hashed file names and no file extensions (except for .htaccess and index.htm). It's a security feature, some of the reasons are covered in this Blog article.

If the files were transferred in binary mode it is strange that they have become corrupted.
                      Support Request Template
3.0.x: Knowledge Base Styles Support MOD Requests
3.1.x: Knowledge BaseStyles SupportExtension Requests

User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Re: Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 12:51 pm

Indeed. I can confirm that the extracted images are 100% match and valid. found 25 images that shows on the forum. copied them to HD and compared that dir with filebackup. all 25 had succesfully been extracted.

In files/ there is about 30 pics with a .jpg extension, rest is standard encoded.

Whats strange is that about half actually show fine after the transfer of the site. During which no files were changed and the db wasnt touched in anyway. not even the config table.

These guys are serious about their attachments, DIY forums etc going years back. I have redownloaded the /files/ files 4 times now in binary, reupped them locally and remotely. gone of the proccess many times on local. all the time its the same results.

Only part of this move that im not 100% about is the db extraction. It was done via phpmyadmin and the guide here

https://www.phpbb.com/kb/article/transf ... or-domain/

was followed. Since the org host had a older version of phpmyadmin i had to do this as the guide said
In the right-hand frame, select the Export tab. In the Export section of this page, be sure to choose Select All as well as the SQL radio button. In the Options section of this page, ensure the Add DROP TABLE checkbox is checked.

Below this setting in the Data section, ensure Complete inserts is checked, and Extended inserts is unchecked.
I had to stray a bit on that. If i didnt include "Extended inserts" the database would only be restored with 50 tables (tried it 3 times) when included "Extended inserts" then it worked fie. All other steps of the guide was followed besides this one. And every aspect of the board functions fine. (besides permission tracing that gives a module not know, but that shouldnt matter)
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

User avatar
T0ny
Registered User
Posts: 1383
Joined: Sun Jan 29, 2006 8:42 pm
Location: Lancashire
Name: Tony

Re: Uploading attachments issues

Post by T0ny » Fri Mar 22, 2013 1:40 pm

How many rows are there in your phpbb_attachments table?

User avatar
Brf
Support Team Member
Support Team Member
Posts: 51758
Joined: Tue May 10, 2005 7:47 pm
Location: {postrow.POSTER_FROM}
Contact:

Re: Uploading attachments issues

Post by Brf » Fri Mar 22, 2013 1:47 pm

DoYouSpeakWak wrote: It told me that there was 30000+ attachments to fetch, but there is only 9998 files in /files/.
Likely, whatever software you, or your host, were using to access the folder ran into a file-limit.

User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Re: Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 1:54 pm

T0ny wrote:How many rows are there in your phpbb_attachments table?
33,991

acp. number of attachments. 33,980

I didnt even think that there could be more than 9998 since thats whats in files.
Brf wrote:
DoYouSpeakWak wrote: It told me that there was 30000+ attachments to fetch, but there is only 9998 files in /files/.
Likely, whatever software you, or your host, were using to access the folder ran into a file-limit.
that would mean that my ftp program, filezilla. when loading /files/ are limited to 9998 files ?

Edit: seems to be a server issue. filezilla has no such limits.
http://forum.filezilla-project.org/view ... f=1#p58014

Image
Last edited by DoYouSpeakWak on Fri Mar 22, 2013 2:05 pm, edited 1 time in total.
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

User avatar
T0ny
Registered User
Posts: 1383
Joined: Sun Jan 29, 2006 8:42 pm
Location: Lancashire
Name: Tony

Re: Uploading attachments issues

Post by T0ny » Fri Mar 22, 2013 2:04 pm

I had a problem a few years ago where I could only see 1,998 files in a directory by FTP. It turned out that the host's FTP server was limited to showing 2,000 files ('.' & '..' being the other 2). I had to download the files that I could see, then delete them so the others would become visible.

Maybe your hosts FTP server is limited to 10,000 ?

User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Re: Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 2:07 pm

T0ny wrote:I had a problem a few years ago where I could only see 1,998 files in a directory by FTP. It turned out that the host's FTP server was limited to showing 2,000 files ('.' & '..' being the other 2). I had to download the files that I could see, then delete them so the others would become visible.

Maybe your hosts FTP server is limited to 10,000 ?
Maybe. that would be an option but i hate to delete them from the only place were they are backed up. The move was done due to the old host only pay bills. no support. Couldnt even get ssh access to do a mysql dump.

But i have contacted the old host again. hopefull that they can change the limits or give me ssh access. (its a dedicated)

Would be nice to confirm that this really is the issue, even though it looks like it.
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

User avatar
Brf
Support Team Member
Support Team Member
Posts: 51758
Joined: Tue May 10, 2005 7:47 pm
Location: {postrow.POSTER_FROM}
Contact:

Re: Uploading attachments issues

Post by Brf » Fri Mar 22, 2013 2:46 pm

from what I am reading, Filezilla itself does not have a limit, but your hosting might limit folder listings to 10000.

User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Re: Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 3:43 pm

Brf wrote:from what I am reading, Filezilla itself does not have a limit, but your hosting might limit folder listings to 10000.
I agree. i have contacted him. he refuses to give me ssh access for 2 hours or change the ftp settings. but did offer to zip that directory and put it up for download to me.

But that worries med with binary mode. If he goes in via ssh and zip the directory and makes it readable so i can download it. Then im not sure the files will be in the right mode even if i download the zip in binary. Any ideas on that ?
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

User avatar
Brf
Support Team Member
Support Team Member
Posts: 51758
Joined: Tue May 10, 2005 7:47 pm
Location: {postrow.POSTER_FROM}
Contact:

Re: Uploading attachments issues

Post by Brf » Fri Mar 22, 2013 3:50 pm

A zip file, by definition, is a binary file. If the folder is zipped on the server, it would contain the files in binary.

User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 50558
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Uploading attachments issues

Post by stevemaury » Fri Mar 22, 2013 3:53 pm

All you can do is try it.
For REALLY good and VERY inexpensive hosting CLICK HERE

I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)

User avatar
DoYouSpeakWak
Registered User
Posts: 2307
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Name: Hans Lassen
Contact:

Re: Uploading attachments issues

Post by DoYouSpeakWak » Fri Mar 22, 2013 3:54 pm

Thx very much. what i though. should be fine as long as he doesn use ssh, ftp to transfer them before zipping. direct in the terminal should work fine.

To be honest its the first time i have encountered such a limit on a server. In the future i will always check amount of files, and those listed in acp and the database before starting the transfer. And not just think that the number shown in /files/ are correct.
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid phpBB help and System administrator services.

Locked

Return to “[3.0.x] Support Forum”