tables too large?

Get help developing custom BBCodes or request one.
Post Reply
jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

tables too large?

Post by jonas3344 » Thu Nov 16, 2017 12:15 pm

Hey guys,

I'm a member of a board still running on 2.0.x mainly because there are a lot of tables there which were (and still are) created with a mod which couldn't be converted to 3.2.x
I now jumped into it and wrote a script which would make those tables convertable and phpbb3 could display them through customized BBCodes.
I tried that out, it works, but I have one major problem:

Some of the table are fairly large and phpbb3 stops to parse the table at a certain point. Means: The table gets rendered fine until that point where I just get to see the BBCode and the text, it just doesn't get parsed.

Does anyone have an idea what the problem there could be and if there is a bugfix or workaround? Any help is appreciated.

phpBB 3.2.1, installed last week. Data converted from a board running phpbb 2.0.22.

Code: Select all

INSERT INTO `phpbb_bbcodes` (`bbcode_id`, `bbcode_tag`, `bbcode_helpline`, `display_on_posting`, `bbcode_match`, `bbcode_tpl`, `first_pass_match`, `first_pass_replace`, `second_pass_match`, `second_pass_replace`)
VALUES
	(13,'table','',0,'[table fontsize={NUMBER;optional} $tagLimit=50000]{TEXT}[/table]','<table fontsize={NUMBER} cellspacing="2" cellpadding="2" border="1" bgcolor="#FFFFFF" align="top">{TEXT}</table>','!\[table fontsize\=\{NUMBER;optional\} \$tagLimit\=50000\](.*?)\[/table\]!ies',''[table fontsize={NUMBER;optional} $tagLimit=50000:$uid]'.str_replace(array("\r\n", '\"', '\'', '(', ')'), array("\n", '"', '&#39;', '&#40;', '&#41;'), trim('${1}')).'[/table:$uid]'','!\[table fontsize\=\{NUMBER;optional\} \$tagLimit\=50000:$uid\](.*?)\[/table:$uid\]!s','<table fontsize={NUMBER} cellspacing="2" cellpadding="2" border="1" bgcolor="#FFFFFF" align="top">${1}</table>'),
	(14,'mcol','',0,'[mcol $tagLimit=50000 color={COLOR;optional}]{TEXT}[/mcol]','<td style="background-color: {COLOR}">{TEXT}</td>','!\[mcol \$tagLimit\=50000 color\=\{COLOR;optional\}\](.*?)\[/mcol\]!ies',''[mcol $tagLimit=50000 color={COLOR;optional}:$uid]'.str_replace(array("\r\n", '\"', '\'', '(', ')'), array("\n", '"', '&#39;', '&#40;', '&#41;'), trim('${1}')).'[/mcol:$uid]'','!\[mcol \$tagLimit\=50000 color\=\{COLOR;optional\}:$uid\](.*?)\[/mcol:$uid\]!s','<td style="background-color: {COLOR}">${1}</td>'),
	(15,'mrow','',0,'[mrow color={COLOR;optional} $tagLimit=50000]{TEXT}[/mrow]','<tr style="background-color: {COLOR}">{TEXT}</tr>','!\[mrow color\=\{COLOR;optional\} \$tagLimit\=50000\](.*?)\[/mrow\]!ies',''[mrow color={COLOR;optional} $tagLimit=50000:$uid]'.str_replace(array("\r\n", '\"', '\'', '(', ')'), array("\n", '"', '&#39;', '&#40;', '&#41;'), trim('${1}')).'[/mrow:$uid]'','!\[mrow color\=\{COLOR;optional\} \$tagLimit\=50000:$uid\](.*?)\[/mrow:$uid\]!s','<tr style="background-color: {COLOR}">${1}</tr>'),
	(16,'FARBE=','',0,'[FARBE={COLOR} $tagLimit=50000]{TEXT}[/FARBE]','<span style="color: {COLOR};">{TEXT}</span>','!\[farbe\=([a-z]+|#[0-9abcdef]+) \$tagLimit\=50000\](.*?)\[/farbe\]!ies',''[farbe=${1} $tagLimit=50000:$uid]'.str_replace(array("\r\n", '\"', '\'', '(', ')'), array("\n", '"', '&#39;', '&#40;', '&#41;'), trim('${2}')).'[/farbe:$uid]'','!\[farbe\=([a-zA-Z]+|#[0-9abcdefABCDEF]+) \$tagLimit\=50000:$uid\](.*?)\[/farbe:$uid\]!s','<span style="color: ${1};">${2}</span>');
Thanks,

jonas
Last edited by jonas3344 on Fri Nov 17, 2017 10:41 pm, edited 1 time in total.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Thu Nov 16, 2017 6:13 pm

I've tested some more:
Bildschirmfoto 2017-11-16 um 19.09.40.png
It really looks like the bbcode-parser stops parsing at a certain point. The table is correct and the post is also correct in the database.

There has to be a workaround for this. Why should there be a limit for tables when a post can face up to 60'000 characters (or even more).

User avatar
JoshyPHP
Code Contributor
Posts: 753
Joined: Mon Jul 11, 2011 12:28 am

Re: tables too large?

Post by JoshyPHP » Thu Nov 16, 2017 6:24 pm

There's a default limit to any BBCode that's set to 1000 or 5000 depending on your version. You can set a different limit in the BBCode definition, e.g.

Code: Select all

[td $tagLimit=10000]{TEXT}[/td]
I wrote the thing that does the BBCodes in 3.2. Unless it broke yours, in which case it was somebody else with a similar name.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Thu Nov 16, 2017 7:11 pm

That doesn't seem to make a difference for me.

I've made:

Code: Select all

[mcol color={COLOR;optional} $tagLimit=20000]{TEXT}[/mcol]
If I set the value to 2 it works (it just parses two [mcol]). If I try to raise the value it doesn't change anything.

$tagLimit in the Configurator.php of s9e was set to 5000, I tried to raise it to 10000 which also doesn't help.
And further, my test-table has approx. 1400 tags in it, which is far away from 5000.
That's really strange.

User avatar
JoshyPHP
Code Contributor
Posts: 753
Joined: Mon Jul 11, 2011 12:28 am

Re: tables too large?

Post by JoshyPHP » Thu Nov 16, 2017 7:30 pm

I don't know. You can try installing this extension and test it again in a new post. Always test markup-related things in a new post.
I wrote the thing that does the BBCodes in 3.2. Unless it broke yours, in which case it was somebody else with a similar name.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Thu Nov 16, 2017 7:59 pm

I did that, always created a new post.

Your extension doesn't show up. Which is folder name for the author? Putting it directly into the ext-folder doesn't work. Other extensions work.
Did a fresh reinstall of a forum, without any data in it. Didn't work either.

Edit: never mind, found it out! (s9e/nolimits)

Edit2: Doesn't change anything. Doesn't look like the tagLimit is the problem here.

Thanks anyway for your help!

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Thu Nov 16, 2017 8:31 pm

post_text in the phpbb_posts looks like:

Code: Select all

<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48</MCOL>	<MCOL color="lime"><s>[mcol color=lime]</s>	48</MCOL>	<MCOL><s>[mcol]</s>	48	[mcol]	48	[mcol]	48	[mcol]	48	[mcol]	48	[mcol]	48	[mcol]	48<br/>
[mrow color=grey]	[mcol]	49	[mcol]	49	[mcol]	49	[mcol]	gugus49	[mcol]
Looks like the posting is screwed.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Fri Nov 17, 2017 2:54 am

So I went throught the parser of s9e and found the function pushContext() and found the line:

Code: Select all

++$this->cntTotal[$tagName];
Without this line it seems to work. I cannot say why setting the limit was not possible.

Code: Select all

var_dump($tagConfig['tagLimit'])
showed:

int(9223372036854775807)

User avatar
JoshyPHP
Code Contributor
Posts: 753
Joined: Mon Jul 11, 2011 12:28 am

Re: tables too large?

Post by JoshyPHP » Fri Nov 17, 2017 11:24 am

Can you post a dump of your phpbb_bbcodes table and a copy of the original post's text? (in plain text form, something to be copy/pasted into the editor) I'd like to see if I can reproduce that behaviour locally. With $tagConfig['tagLimit'] being that high, there's no way it could ever be reached. The only explanation that comes to mind would be some kind of PHP bug, but that would be very surprising.
I wrote the thing that does the BBCodes in 3.2. Unless it broke yours, in which case it was somebody else with a similar name.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Fri Nov 17, 2017 10:41 pm

I did that. Thanks for looking over that.

User avatar
JoshyPHP
Code Contributor
Posts: 753
Joined: Mon Jul 11, 2011 12:28 am

Re: tables too large?

Post by JoshyPHP » Fri Nov 17, 2017 10:47 pm

I see that you edited your first post to include the SQL dump but I can't find the plain text content of the post. You can use a service like Pastebin, or GitHub's Gist if it's too big.

Actually nevermind, I found why it still ignores some of your BBCodes. Update the "nolimits" extension and you should be able to post those huge tables.
I wrote the thing that does the BBCodes in 3.2. Unless it broke yours, in which case it was somebody else with a similar name.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Sat Nov 18, 2017 8:59 pm

Thanks, that works. Great. :!:

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Sun Nov 19, 2017 2:45 am

My (hopefully) last problem: When I convert my data from my old phpbb 2 board the convertor does the same the phpbb3 did without your extension. Bevor I dive into the source code of the convertor and try to find that, do you have a plan where I could look or what I could change to convert my whole tables?

Edit: Ok, if I resubmit the post it works. So I've to do that via CLI.

User avatar
JoshyPHP
Code Contributor
Posts: 753
Joined: Mon Jul 11, 2011 12:28 am

Re: tables too large?

Post by JoshyPHP » Sun Nov 19, 2017 11:03 am

I don't know much about conversion. Recently I learned that the recommended procedure is to disable extensions before upgrading. Unfortunately, that means that any post that uses an extension will be converted incorrectly. Personally, I'd try upgrading a local copy with the extension enabled and see if it works.

Alternatively, you can upgrade using the recommended way then use the CLI reparser after you've re-enabled all the extensions you need.
I wrote the thing that does the BBCodes in 3.2. Unless it broke yours, in which case it was somebody else with a similar name.

jonas3344
Registered User
Posts: 10
Joined: Thu Nov 16, 2017 12:09 pm

Re: tables too large?

Post by jonas3344 » Sun Nov 19, 2017 12:05 pm

Yeah, I guess the second way is the one for me. Thanks for your help! :)

Post Reply

Return to “Custom BBCode Development and Requests”

Who is online

Users browsing this forum: No registered users and 5 guests

cron