[ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Get help with installation and running phpBB 3.2.x here. Please do not post bug reports, feature requests, or extension related questions here.
Post Reply
harris1
Registered User
Posts: 42
Joined: Thu Mar 13, 2014 11:31 am

[ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by harris1 »

Hello,

I got those errors after I purge cache.

phpbb 3.2.9
PHP 7.3.14

extentions
Advanced Active Topics 2.0.5
Advanced BBCode Box 3.2.2
Ajax Base 1.1.0
Auto database backup (3.2) 2.1.0-rc2
Button menu 1.0.0
Database Optimize & Repair Tool 1.1.2
Disable all extensions 2.1.0
Exif Image Rotation 1.0.0-alpha
External Links Open in New Window 1.0.6
Fancybox 1.0.1
Moon Phases 1.0.7
Nivo Slider 1.0.6
Notify Admin on Registration 1.0.1
OneAll Social Login 4.5.2
Page Scrolling 1.1.0
PayPal Donation Extension 2.1.0-dev
Posts merging 2.1.0
Precise Similar Topics 1.5.0
Prime Login Return 1.0.2
Quick Login 1.1.1
Recent Topics 2.2.8
s9e/mediaembed 20170724
Scroll To Top 1.0.1
Search user topics 1.0.2
Simple mentions 1.0.2
Simple WYSIWYG 1.0.2
Site logo (3.2) 2.1.0-rc6
Tapatalk Plugin for phpBB 3.1/3.2/3.3 2.1.5
Thanks for posts 3.2.10
Topic Author 1.0.2

[phpBB Debug] PHP Warning: in file [ROOT]/vendor/s9e/text-formatter/src/Configurator.php on line 2508: DOMElement::getAttribute(): Couldn't fetch DOMElement
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/s9e/text-formatter/src/Configurator.php on line 2510: DOMXPath::query(): Couldn't fetch DOMElement
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/s9e/text-formatter/src/Configurator.php on line 2510: Invalid argument supplied for foreach()
Thank you in advance
User avatar
warmweer
Jr. Extension Validator
Posts: 6227
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Bel ... gium

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by warmweer »

harris1 wrote:
Tue Feb 25, 2020 8:34 pm
phpbb 3.2.9
PHP 7.3.14
phpBB 3.2.9 is not compatible with php 7.3 and higher.
You'll need to upgrade to 3.3.0 if you can't downgrade the php version.
Also some of those extensions won't function with php 7.3
Spelling is freeware, which means you can use it for free.
On the other hand, it is not open source, which means you cannot change it or publish it in a modified form.
harris1
Registered User
Posts: 42
Joined: Thu Mar 13, 2014 11:31 am

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by harris1 »

warmweer wrote:
Tue Feb 25, 2020 9:07 pm
harris1 wrote:
Tue Feb 25, 2020 8:34 pm
phpbb 3.2.9
PHP 7.3.14
phpBB 3.2.9 is not compatible with php 7.3 and higher.
You'll need to upgrade to 3.3.0 if you can't downgrade the php version.
Also some of those extensions won't function with php 7.3
Hi,
PHPBB 3.2.9
Downgrade to php 7.2.28


no extentions


The same error.
User avatar
mrgoldy
Development Team Member
Development Team Member
Posts: 1394
Joined: Tue Oct 06, 2009 7:34 pm
Location: The Netherlands
Name: Gijs
Contact:

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by mrgoldy »

Do you have any Custom BBCodes created through your ACP?
phpBB Studio / Member of the Studio

Contributing: You can do it too! Including testing Pull Requests (PR).
phpBB Development and Testing made easy.
harris1
Registered User
Posts: 42
Joined: Thu Mar 13, 2014 11:31 am

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php (Solved)

Post by harris1 »

mrgoldy wrote:
Wed Feb 26, 2020 4:19 pm
Do you have any Custom BBCodes created through your ACP?

You are the best! I removed some bbcodes and now I do not have errors!
Thank you!
User avatar
mrgoldy
Development Team Member
Development Team Member
Posts: 1394
Joined: Tue Oct 06, 2009 7:34 pm
Location: The Netherlands
Name: Gijs
Contact:

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by mrgoldy »

You’re welcome. You had some incorrect HTML (xsl) replacement.
phpBB Studio / Member of the Studio

Contributing: You can do it too! Including testing Pull Requests (PR).
phpBB Development and Testing made easy.
User avatar
JoshyPHP
Code Contributor
Posts: 1198
Joined: Mon Jul 11, 2011 12:28 am

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by JoshyPHP »

If there's any way to reproduce that error on a recent version of phpBB I'd be interested in that template.

That said, the most recent version of phpBB that contains those instructions at the lines referenced in those error messages is phpBB 3.2.5, from 2018. This is not phpBB 3.2.9.
I wrote the thing that does BBCodes in 3.2+.
User avatar
gn#36
Translator
Posts: 205
Joined: Fri Oct 13, 2006 1:16 pm
Contact:

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by gn#36 »

JoshyPHP wrote:
Wed Feb 26, 2020 8:20 pm
If there's any way to reproduce that error on a recent version of phpBB I'd be interested in that template.

That said, the most recent version of phpBB that contains those instructions at the lines referenced in those error messages is phpBB 3.2.5, from 2018. This is not phpBB 3.2.9.
This post is a while ago, but I have come across this problem as well now in phpBB 3.3.2 on PHP 7.4 when updating my forum from phpBB 3.1.x. This is only an educated guess, but to me, the error seems to be related to the number of smilies in the board and ocurrs when parsing the forum description texts, based on the huge PHP string it tries to optimize when it crashes and the resulting index.php html when I replace it by an empty string.

I put an echo of the input string into the optimize function where it crashed. As a result, it dumped a bunch of strings, the relevant long one starting with:

Code: Select all

if($this->params['S_VIEWSMILIES']===''){$this->out.=htmlspecialchars($node->textContent,0);}else{switch($node->textContent){case'8)':$this->out.='<'.htmlspecialchars('img',3);$this->out.=' '
This is followed by an extremely long (>620.000 characters!) switch case with I assume all smilies of the board. This is quite a huge number on mine (19 pages worth in the ACP), and therefore, it crashed with:
Allowed memory size of 67108864 bytes exhausted (tried to allocate 8388616 bytes) in [ROOT]vendor/s9e/text-formatter/src/Configurator/RendererGenerators/PHP/Optimizer.php on line 57
Since the amount of memory requested is not exactly small, I am guessing this needs some sort of optimization.

Now I temporarily "solved" this by having the function return an empty string as result (increasing the memory limit is right now not really an option and returning the original string shifts the problem to a different function), but obviously that leads to the index.php and viewforum.php having some empty spots where actually there should be forum descriptions, which use BBcodes & smilies.

Maybe it would be better to not put all smilies in a big switch case to be parsed or have it optimized in two steps using placeholders, so each case doesn't need so much code with all these htmlspecialchars elements.

In case you actually do need the whole string somehow, you can find that here (it has all the output I got before crashing when viewing my index.php directly after cache was deleted, the line it crashed on is the 5th): https://www.phpbb.de/pastebin/?mode=view&s=69. Be careful when you open this, it is a huge page, the code to optimize has more than 620.000 characters.

EDIT: Since you asked for a sample: The only code on the index.php I found using BBcodes & smilies was one Forum description which had a coloring in it, it didn't even have smilies actually.
German Support Team Member • http://www.phpbb.deMy Extensions in the CDBMy Extensions on Github • Contact with caos is inavoidable but no catastrophy if you keep an overview.
User avatar
JoshyPHP
Code Contributor
Posts: 1198
Joined: Mon Jul 11, 2011 12:28 am

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by JoshyPHP »

That's a different error. The one from the first post can't be reproduced because it was probably caused by an unrelated software issue. Yours is simply PHP running out of memory while compiling a template into PHP. You have ~931 smilies and with that amount of smilies, regenerating phpBB's cache after it's been cleared takes about 68 MB on my local test board running PHP 8.0.1. With a 64 MB limit, the compilation fails partway. Reloading the page after it failed lets it complete successfully because at that point some of the cache has already been recreated and it frees a few MB. It may be different on yours, depending on your PHP version, OS, whether OPCache is enabled and other factors.

I periodically review resource usage (CPU and memory) and there's no simple "optimization" that I know of that will make things use less memory. If you can't raise the memory limit (PHP uses 128 MB by default) then you can try disabling PHP's tokenizer extension to avoid executing the specific line of code from that PHP error, although on my test board it increases the amount of memory required for some reason.
I wrote the thing that does BBCodes in 3.2+.
User avatar
gn#36
Translator
Posts: 205
Joined: Fri Oct 13, 2006 1:16 pm
Contact:

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by gn#36 »

Hm sorry, I was sure I had read my exact error message here in some followup post and thought you were referring to that one.

But on my side, the error is non-recoverable. It doesn't build up the cache partially (I mean, it does, but apparently not enough) and then use less RAM, it permanently fails even with multiple reloads. Might also be due to my very custom BBcode definitions - they are very old, still from phpBB 3.0.x essentially, and some of them were calling special functions in my extension on phpBB 3.1.x to get some data, others were using regular expressions. I haven't been able to get them to work again, even in 3.1.x it was somewhat difficult, but there they were working.

I know optimizing RAM is hard - I am working on microcontroller SW for a living, these don't even have remotely that much RAM total typically...

I think the only way this can be optimized is on the input side, really. What is killing it is the huge amount of concatenated htmlspecialchars applied to single characters or even empty quotes.
German Support Team Member • http://www.phpbb.deMy Extensions in the CDBMy Extensions on Github • Contact with caos is inavoidable but no catastrophy if you keep an overview.
User avatar
JoshyPHP
Code Contributor
Posts: 1198
Joined: Mon Jul 11, 2011 12:28 am

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by JoshyPHP »

The relative complexity of your BBCode won't have that kind of effect, it's really only about the volume of data generated when compiling the smilies' template. Specifically, the data from PHP's tokenizer which has the most overhead.

If you disable PHP's ext/tokenizer extension, it will skip that code. You'll end up with unoptimized PHP code which is slightly slower but not by much. Alternatively, you can hack some files to cut down on the amount of code the tokenizer will have to handle. The code is below. It's fairly untested so obviously that's only if you're into that kind of things. You could also implement that as an extension so you don't have to modify any existing file, but I'm not sure it's worth the trouble.

Code: Select all

diff --git a/src/Configurator/RendererGenerators/PHP/Optimizer.php b/src/Configurator/RendererGenerators/PHP/Optimizer.php
index f5e23328b..743e1a853 100644
--- a/src/Configurator/RendererGenerators/PHP/Optimizer.php
+++ b/src/Configurator/RendererGenerators/PHP/Optimizer.php
@@ -54,6 +54,18 @@ public function __construct()
 	*/
 	public function optimize($php)
 	{
+		$php = preg_replace(
+			"(\\\$this->out\\.='([^']++)'\\.htmlspecialchars\\('([-\\w]++)',\\d\\))",
+			'$this->out.=\'$1$2\'',
+			$php
+		);
+		do
+		{
+			$regexp = "(\\\$this->out\\.='[^']++\\K'(?:\\.'|;\\\$this->out\\.='))";
+			$php    = preg_replace($regexp, '', $php, -1, $cnt);
+		}
+		while ($cnt);
+
 		$this->tokens = token_get_all('<?php ' . $php);
 		$this->cnt    = count($this->tokens);
 		$this->i      = 0;
FWIW, I've been working on a different design that won't use PHP's tokenizer for a future version but I have no date for it.
I wrote the thing that does BBCodes in 3.2+.
User avatar
gn#36
Translator
Posts: 205
Joined: Fri Oct 13, 2006 1:16 pm
Contact:

Re: [ROOT]/vendor/s9e/text-formatter/src/Configurator.php

Post by gn#36 »

Thank you, I will try it out. I already hacked the function myself to get rid of the problem, so taking a bit of code from someone who knows it better than I do is probably a much better solution.
German Support Team Member • http://www.phpbb.deMy Extensions in the CDBMy Extensions on Github • Contact with caos is inavoidable but no catastrophy if you keep an overview.
Post Reply

Return to “[3.2.x] Support Forum”