Handling gateway timeouts installing an extension

Discussion forum for Extension Writers regarding Extension Development.
Post Reply
User avatar
MarkDHamill
Registered User
Posts: 4362
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Handling gateway timeouts installing an extension

Post by MarkDHamill »

Testing my own extension, I am getting a gateway timeout after 30 seconds or so when installing an extension I am working on. I know the reason: it's taking "too long" to install. I thought logic to handle this was built into the extension architecture, but maybe there is something I should be doing in addition, like adding a set_time_limit(0) statement.

It's happening because the spam remover extension I am working on adds two columns to the posts and private messages tables to flag if a post is spam and when it was checked. Since the posts table tends to be big, it's not surprising if you have lots of posts (about 30,000 in this case) some sort of web server timeout issue might occur.

Is there a preferred way to handle issues like this installing extensions?
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
rxu
Extensions Development Team
Posts: 3467
Joined: Wed Oct 25, 2006 12:46 pm
Location: Siberia, Russian Federation
Name: Ruslan
Contact:

Re: Handling gateway timeouts installing an extension

Post by rxu »

I had similar case for my "Posts merging" extension and ended up getting rid of adding posts table column in favor to own extension table. Actually there's nothing can be done with that as on big posts table users will get timeouts regardless of any settings because adding column will waste too much server resources -> server time (at least on most popular MySQL/MariaDB DBMS).
User avatar
MarkDHamill
Registered User
Posts: 4362
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: Handling gateway timeouts installing an extension

Post by MarkDHamill »

I was thinking I might do that. It's just that a separate table seems overkill to address a problem which at its root is due to a lack of sufficient server resources.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
User avatar
ViolaF
Registered User
Posts: 1557
Joined: Tue Aug 14, 2012 11:52 pm

Re: Handling gateway timeouts installing an extension

Post by ViolaF »

May you take a look at the MySQLDumper-Mechanism (PHP/SQL). It works even "under timeout".
The script remembers, on which point it was and starts again from this point (and again, ... till it finished)..
User avatar
EA117
Registered User
Posts: 2035
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Handling gateway timeouts installing an extension

Post by EA117 »

ViolaF wrote:
Sun Jan 03, 2021 7:23 pm
The script remembers, on which point it was and starts again from this point (and again, ... till it finished)..
Agreed, and had wondered if somehow that could be done "even during extension enablement", despite that this context doesn't seem to provide an obvious "navigate to a new URL" opportunity for re-issuing the request.

But the action currently at issue here is just a single "atomic" operation, from the PHP code's point of view: "Please Mr. SQL server, add a column to this existing huge table." It's not a multi-step request you could break up or "continue", even if we assume the URL request could be re-issued upon timeout.

It also made me ask "wait, doesn't the phpBB database migration have to do this every so often?" But I could not find an example of a column being added to the POSTS_TABLE during any of the migration modules. So no precedent to follow there, either.

edit: That was wrong, of course. Started bugging me that it would be "impossible" no changes to the phpbb_posts table would have been needed over the years, and indeed I was searching for it incorrectly.
User avatar
MarkDHamill
Registered User
Posts: 4362
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: Handling gateway timeouts installing an extension

Post by MarkDHamill »

Argh! Sounds like I need to create a separate table.

I was wondering if the still_on_time() function could be used in the migration. I use it elsewhere in the extension (although it never seems to get triggered, so I use a user-defined batch size to set a number of items to go to Akismet for a spam check before doing the next batch) but it looks impractical to integrate it into a migration program in any event.

All this amounts to constraints on shared hosting. These place implicit limitations on extensions as you have to work around these resource limitations, timeouts, etc.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
Post Reply

Return to “Extension Writers Discussion”