phpBB2.0.20: Problems with unicode characters

This is an archive of the phpBB 2.0.x support forum. Support for phpBB2 has now ended.
Forum rules
Following phpBB2's EoL, this forum is now archived for reference purposes only.
Please see the following announcement for more information: viewtopic.php?f=14&t=1385785
Locked
Stoepsel
Registered User
Posts: 395
Joined: Sun Oct 23, 2005 11:23 am

phpBB2.0.20: Problems with unicode characters

Post by Stoepsel »

I'm admin on a forum, which I updated to phpBB 2.0.20 yesterday. I then continued to apply the patches provided here:
http://www.phpbb.com/phpBB/viewtopic.php?t=382680

Now to the problem itself...

On my forum, members regularly post ASCII art made up in part of unicode characters. A short example of such a post could look like this:

Code: Select all

▓▒▒░░░    My, doesn't this look nice?    ░░░▒▒▓
Ever since the upgrade to 2.0.20, such posts are no longer showing up as before, when the user doesn't disable HTML for the post. The post will now show up as:

Code: Select all

& #9619;& #9618;& #9618;& #9617;& #9617;& #9617;    My, doesn't this look nice?    & #9617;& #9617;& #9617;& #9618;& #9618;& #9619;
... except that there are no spaces between the ampersand and the hash symbol. I had to fiddle a little to give you an idea, because this board doesn't have this problem.

To recreate the problem, apply the following settings:

On forum level:
Allow HTML: On

On user level:
Always allow HTML: On

On post level:
Disable HTML in this post: Off

Then create a post with the example I posted above.

Next, edit the post and set the "Disable HTML in this post" option to On and submit. You should see the unicode characters as intended.

Next, edit the post again and set the "Disable HTML in this post" option to Off and submit. You should see the problem reoccur.

As stated before, I've already applied all the patches given here:
http://www.phpbb.com/phpBB/viewtopic.php?t=382680

I'm fairly certain that this has to do with the changes to prepare_message() in includes/functions_post-php, but unfortunately I'm not PHP guru enough to narrow down the problem more or even suggest a fix for it.

In essence, this problem is related to the "named quotes" problem reported numerous times before.

Some kind of feedback on this would be highly appreciated. A fix for this would be even more helpful, assuning that you also view this as a bug and not a feature. ;)
Last edited by Stoepsel on Sat Apr 15, 2006 3:42 pm, edited 1 time in total.
StarF1DotCom
Registered User
Posts: 37
Joined: Wed Apr 12, 2006 6:39 pm

Post by StarF1DotCom »

Pal I am with U.

The whole day today I tried to convince the members here something wrong with unicoded messages.

But the same unicoded are working pretty fine here.

U see u yourself has put a space between the unicode to avoid converted into the special charcter.
Stoepsel
Registered User
Posts: 395
Joined: Sun Oct 23, 2005 11:23 am

Post by Stoepsel »

StarF1DotCom wrote: But the same unicoded are working pretty fine here.

Yes. But 2 reasons come to my mind:

* This board has HTML turned off, either forum wide or on user level.
* This board is not yet running 2.0.20.

At least now you know that there is somewhat of a problem. I welcome anyone to take my little example above and test it on a 2.0.20 board, playing around with the HTML on/off settings.
sanejane
Registered User
Posts: 288
Joined: Mon Jan 09, 2006 6:05 pm

Post by sanejane »

I tried this on my 2.0.20 board (html disabled) and the looks fine.

have u tried the fix here:
http://www.phpbb.com/phpBB/viewtopic.php?t=382680

?
Stoepsel
Registered User
Posts: 395
Joined: Sun Oct 23, 2005 11:23 am

Post by Stoepsel »

Yes, I have installed the patches as stated in my first post.

And you're supposed to test this with HTML enabled to get the problem, and test with HTML disabled to see, that the problem does not show anymore. That is why I think that the problem lost likely lies in prepare_message().
sanejane
Registered User
Posts: 288
Joined: Mon Jan 09, 2006 6:05 pm

Post by sanejane »

actually, works also with html allowed. try the above fix, and mmaybe u messed something up during the upgrade.

i tested both html enbled/disabled and works fine for me.
sanejane
Registered User
Posts: 288
Joined: Mon Jan 09, 2006 6:05 pm

Post by sanejane »

I'm not php guru as well, actually much less then that, but I will try redoing bbcode.php again. as stated in my last reply, works fine here, with or without html enabled.
Stoepsel
Registered User
Posts: 395
Joined: Sun Oct 23, 2005 11:23 am

Post by Stoepsel »

And while I recheck everything, I ask you to make sure that you have:

On forum level:
Allow HTML: On

On user level:
Always allow HTML: On

On post level:
Disable HTML in this post: Off
sanejane
Registered User
Posts: 288
Joined: Mon Jan 09, 2006 6:05 pm

Post by sanejane »

ok, you are right, with the above settings it doesn't work.
but if i try Disable HTML in this post: On it does work.
00100100
Registered User
Posts: 16
Joined: Sat Dec 31, 2005 6:29 am

Post by 00100100 »

sanejane wrote: ok, you are right, with the above settings it doesn't work.
but if i try Disable HTML in this post: On it does work.


same for me ............. im having the same problem :?
Stoepsel
Registered User
Posts: 395
Joined: Sun Oct 23, 2005 11:23 am

Post by Stoepsel »

sanejane wrote: ok, you are right, with the above settings it doesn't work.
but if i try Disable HTML in this post: On it does work.

Yes, but I thought I explained that all in my first post. Either I wasn't clear enough or you missed it. :)

Anyway, I updated the first post to make it clearer, how this problem can be reproduced.
User avatar
T0ny
Registered User
Posts: 1383
Joined: Sun Jan 29, 2006 8:42 pm
Location: Lancashire
Name: Tony

Re: phpBB2.0.20: Problems with unicode characters

Post by T0ny »

Stoepsel wrote: I'm fairly certain that this has to do with the changes to prepare_message() in includes/functions_post.php


Back up includes/functions_post.php
open it and find

Code: Select all

$message .= htmlspecialchars($part) . clean_html($tag);
replace that line with

Code: Select all

$message .= preg_replace($html_entities_match, $html_entities_replace, $part) . clean_html($tag);
That fixes the problem, can anyone see any security issues with making that change?
TheComforter
Registered User
Posts: 19
Joined: Fri Mar 11, 2005 12:30 pm
Location: West (by God) Virginia
Contact:

Post by TheComforter »

Tony.

This fix in conjunction with another alteration we discussed in another thread seems to have repaired the problem and we will return and test it with and without html and with the quickReply mod as well.

Thanks and we knew it would be in the same general area as the quote rememdy previously played out here at the Forums. Just were not sure of the includes to place and the order.

Thanks again.

Related thread:

http://www.phpbb.com/phpBB/viewtopic.ph ... 83#2106083
TheComforter
Registered User
Posts: 19
Joined: Fri Mar 11, 2005 12:30 pm
Location: West (by God) Virginia
Contact:

Post by TheComforter »

After further testing the non breaking space is functioning correctly but after parsing two unicodes satifactory with no changes the script suddenly returned to malfunctioning and not displaying anything but the  

It was strang because with no changes and after several tests in a single post with three unicodes or escape codes two displayed and the last one did not. Something changed in the middle of parsing the same post.

Really strange. Afterwards the very same unicodes that worked no longer parsed and now no code will parse except the   that was included in the other thread meantioned above this post.
TheComforter
Registered User
Posts: 19
Joined: Fri Mar 11, 2005 12:30 pm
Location: West (by God) Virginia
Contact:

Post by TheComforter »

Just to update:

We now have some code parsing and some not. It is not what code it is just at random...

Kwazy...
Locked

Return to “2.0.x Support Forum”