New posted string in custom table loses foreign character

[Nick225]

No problem with foreign characters in the converted phpbb3. Old messages, new messages, all is fine.

But I have a phpbb2 external page with a form that writes to a custom table.
I imported the table, and rewrote the custom page to fit the phpbb3 requirement.

Code: Select all

<?php
define('IN_PHPBB', true);
$phpbb_root_path = (defined('PHPBB_ROOT_PATH')) ? PHPBB_ROOT_PATH : './';
$phpEx = substr(strrchr(__FILE__, '.'), 1);
include($phpbb_root_path . 'common.' . $phpEx)

All the data in the table looks great in the database and on the custom page when I run it. No problem displaying the existing accented characters.

Problem arises when I insert a new row via the form. The posted string loses all its foreign accented letters; for instance, "employés à" becomes "employ??s ??"

From phpadmin, I changed the database encoding from latin1_sweddish_ci to utf8-bin. That didn't help.

I have read dozens of topics about foreign characters here and tried all the suggestions. None work for my case: existing data is ok. New rows are not.
If I edit an existing row and POST, it gets messed up too.
Please help. Thanks.

[canonknipser]

rewrote the custom page to fit the phpbb3 requirement.

Did you use the correct functions for storing and reading data from inside phpBB?

• generate_text_for_display()
• generate_text_for_storage()
• generate_text_for_edit()

[StevieWonderer]

No problem with foreign characters in the converted phpbb3. Old messages, new messages, all is fine.

But I have a phpbb2 external page with a form that writes to a custom table.
I imported the table, and rewrote the custom page to fit the phpbb3 requirement.

Code: Select all

<?php
define('IN_PHPBB', true);
$phpbb_root_path = (defined('PHPBB_ROOT_PATH')) ? PHPBB_ROOT_PATH : './';
$phpEx = substr(strrchr(__FILE__, '.'), 1);
include($phpbb_root_path . 'common.' . $phpEx)

All the data in the table looks great in the database and on the custom page when I run it. No problem displaying the existing accented characters.

Problem arises when I insert a new row via the form. The posted string loses all its foreign accented letters; for instance, "employés à" becomes "employ??s ??"

From phpadmin, I changed the database encoding from latin1_sweddish_ci to utf8-bin. That didn't help.

I have read dozens of topics about foreign characters here and tried all the suggestions. None work for my case: existing data is ok. New rows are not.
If I edit an existing row and POST, it gets messed up too.
Please help. Thanks.

utf_8bin is a collation method, not a charset encoding. First you need to make sure the column charset is really utf8.

[Nick225]

rewrote the custom page to fit the phpbb3 requirement.

Did you use the correct functions for storing and reading data from inside phpBB?

• generate_text_for_display()
• generate_text_for_storage()
• generate_text_for_edit()

Thank everyone for the reply.. Reading on those functions... I didn't use them. Lots of info here.
I used the

$db->sql_build_array('INSERT', array(

but didn't call generate_text_for_storage beforehand.

My table doesn't need bbcode. But it looks like I still need to add the columns for uid and bitfield, etc...

[StevieWonderer]

rewrote the custom page to fit the phpbb3 requirement.

Did you use the correct functions for storing and reading data from inside phpBB?

• generate_text_for_display()
• generate_text_for_storage()
• generate_text_for_edit()

Thank everyone for the reply.. Reading on those functions... I didn't use them. Lots of info here.
I used the

$db->sql_build_array('INSERT', array(

but didn't call generate_text_for_storage before hand.

My table doesn't need bbcode. But it looks like I still need to add the columns for uid and bitfield, etc...

Even though you have to pass 4 variables by reference to generate_text_for_storage(), only the first one is important, which has the post text in XML format after the function call. Other 3 are not used anymore as far as I know.

[Nick225]

Thanks. I mentioned that because it's in every example the tutorial gives; As is they were mandatory fields.

$sql = 'SELECT text, bbcode_uid, bbcode_bitfield, bbcode_options
FROM ' . YOUR_TABLE . '
WHERE some_id = ' . (int) $example_id;
-----------------------------

$sql = 'INSERT INTO ' . YOUR_TABLE . ' ' . $db->sql_build_array('INSERT', array(
'text' => $some_text,
'uid' => $new_uid,
'bitfield' => $new_bitfield);
---------------------------

generate_text_for_storage($some_text, $new_uid, $new_bitfield, $bbcode_options, true, true, true);

I guess we can just set those arguments to null or 0 when calling this function, if we don't have those columns.

[Nick225]

Thank you all. Problem resolved with:

Code: Select all

$post_vars_array = $request->variable_names(\phpbb\request\request_interface::POST);
	$fl_text_raw = utf8_normalize_nfc(request_var('scrollingFlash', '', true));

	if(strlen($fl_text_raw)){
		$uid = $bitfield = $options = '';
		$allow_bbcode = $allow_urls = $allow_smilies = false;
		generate_text_for_storage($fl_text_raw, $uid, $bitfield, $options, $allow_bbcode, $allow_urls, $allow_smilies);
...
...

Hope that helps a newbie like me..
phpbb3 rocks !!!!!!!!!!!!!!!!
Now the only strings with accented characters issues are the table headers .. I applied the utf_normalize_ncf function to them but nope.. ... Have to research that function.