General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Converting from other board software? Good decision! Need help? Have a question about a convertor? Wish to offer a convertor package? Post here.
Post Reply
StringMeteo72
Registered User
Posts: 4
Joined: Sat Jul 27, 2019 8:33 pm

General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by StringMeteo72 » Sat Jul 27, 2019 9:52 pm

I was trying to convert my old phpBB2.0.20 forum to freshly installed phpBB3.2.7 forum and many times I was getting the same (or very similar) error soon after the beginning of the actual conversion (after the pre-conversion steps have successfully been completed):

General Error:
Unknown encoding:
in file /home/stringme/public_html/forum_new000/includes/utf/utf_tools.php on line 280

The BACKTRACE is as follows:
FILE: (not given by php)
LINE: (not given by php)
CALL: installer_msg_handler()

FILE: [ROOT]/includes/utf/utf_tools.php
LINE: 280
CALL: trigger_error()

FILE: [ROOT]/install/convertors/functions_phpbb20.php
LINE: 422
CALL: utf8_recode()

FILE: [ROOT]/includes/functions_convert.php(1289) : eval()'d code
LINE: 1
CALL: phpbb_set_encoding()

FILE: [ROOT]/includes/functions_convert.php
LINE: 1289
CALL: eval()

FILE: [ROOT]/install/convert/convertor.php
LINE: 449
CALL: restore_config()

FILE: [ROOT]/install/convert/controller/convertor.php
LINE: 370
CALL: phpbb\convert\convertor->convert_data()

FILE: (not given by php)
LINE: (not given by php)
CALL: phpbb\convert\controller\convertor->convert()

FILE: [ROOT]/vendor/symfony/http-kernel/HttpKernel.php
LINE: 135
CALL: call_user_func_array()

FILE: [ROOT]/vendor/symfony/http-kernel/HttpKernel.php
LINE: 57
CALL: Symfony\Component\HttpKernel\HttpKernel->handleRaw()

FILE: [ROOT]/install/app.php
LINE: 61
CALL: Symfony\Component\HttpKernel\HttpKernel->handle()
As far as I see the default html charset of the new forum is set to iso-8859-1. Actually about 6-7 years ago the hosting company changed forcibly the charset of all my pages on their server, including pages of my old phpBB2.0.20 forum, to utf-8. Could the problem originates from that fact? The collation of my old forum DB tables is cp1251_general_ci, excepting 2 columns from the table phpbb_posts_text of which I have changed the collation arbitrarily to utf8_general_ci in order to... well, I am not an ace of this matter :oops: but it looked useful for some special symbols to appear properly in the forum. At the other side, the collation of my new forum DB tables is utf8_bin.

Actually I tried also some variants like changing the html charset of some files, resetting the collation of the 2 changed old columns back to utf8_general_ci and removing the folder for Bulgarian language of the old forum. And only in some cases I saw some small changes in the BACKTRACE lines but the general error remained the same.

Some DB details:
Database server

Server: Localhost via UNIX socket
Server type: MariaDB
Server connection: SSL is not being used
Server version: 10.0.38-MariaDB - MariaDB Server
Protocol version: 10
User: xxxxxxxxxxxxxx
Server charset: UTF-8 Unicode (utf8)

Web server

cpsrvd 11.78.0.34
Database client version: libmysql - 5.1.73
PHP extension: mysqli curl mbstring
PHP version: 7.2.7
I would really appreciate any help or even some hints. :?

User avatar
EA117
Registered User
Posts: 1050
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by EA117 » Mon Jul 29, 2019 2:17 pm

Wish I had phpBB 2.0 conversion experience to offer, but I don't.

What does seem interesting is that the specific "unknown encoding" error that was thrown is indicating "the name of this encoding isn't alpha-numeric." Not that "it's unknown" per se, but more like "I was about to try and use the identifier as a filename, but when I tested the identifier name for validity, the name contains more than just a-z, 0-9, underscores, dashes, spaces or backslashes."

i.e. It's not a name like "ISO-8859-1" or "utf-8". It's "something else" which contains more unusual characters in the identifier name of the encoding. The way I read the code, a blank string would also trigger this error; i.e. if the identifier name did not contain any characters at all.

For this particular call stack, it appears to be a case in which the code intended to fall back to using "default_lang" of the phpBB 2.0 board. (As opposed to cases where it intends to use the "user_lang" of the user who actually posted a message or string that is being converted.)

Which means the converter reads the board's default_lang configuration set to "english" or similar, and then attempts to load /language/lang_english/lang_main.php from the phpBB 2.0 board, in order to read the $lang['ENCODING'] = value set in that file. This $lang['ENCODING'] = value is what's expected to be "ISO-8859-1", or "utf-8", etc.

So it seems somewhat unlikely "the phpBB 2.0 converter successfully read '/language/lang_<languagename>/lang_main.php' and ended up with an invalid encoding name." Although you could definitely look at the /language/ directory on the phpBB 2.0 board and check the lang_main.php file in each one of the language directories. To view whether their $lang['ENCODING'] = value sets a name which would fall outside of the "alphanumeric, underscores, dashes, spaces or backslashes" constraints.

It seems more likely that perhaps "default_lang" on this phpBB 2.0 board is set to a value for which the "/language/lang_<languagename>/" directory simply doesn't exist? And as a result we ended up with a blank encoding name for this "default_lang" configuration reference. Do you perhaps not have "the actual phpBB 2.0 board files", and just an English-only copy of the phpBB 2.0 files, without any of the language(s) that may be referenced in the actual phpBB 2.0 database for default board language and/or user-selected language? Or maybe it's just a case of one or more language directories missing from the phpBB 2.0 board, even though they're still referenced in the database.

Whether that ends up being of any help at all, to me this doesn't look like a "conversion error", where an attempt to convert actual characters from the database has failed or encountered non-mappable characters or similar. This is the phpBB 2.0 converter code declaring "I don't understand the name of the encoding I'm supposed to attempt converting." And the encoding in question seems to come from a phpBB language configuration; not the collation or other property of an actual SQL table or similar.

StringMeteo72
Registered User
Posts: 4
Joined: Sat Jul 27, 2019 8:33 pm

Re: General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by StringMeteo72 » Tue Jul 30, 2019 2:23 pm

EA117, thank you for your time! Actually you gave me a good lead. I checked what is going on with the languages in my old board and... I found with a big surprise that there was no "default_language" row in the phpbb_config table. The same time actually there is such setting through the ACP (General Admin->Configuration) and this setting works, at least apparently (I'm not sure where exactly the administrator preference is written/stored if not in this table...). So it is possible the convertor to try to read the value of this preference from somewhere (for example from the respective row in the phpbb_config table as it is logical), but the value is missing and... Also I remembered that this setting has never really worked. Although every user is able to set his own default language (for the case when he/she is logged in) and this setting works, the language for the other case is always english (regardless of my setting of "default_language" for the board).

OK, this seems to me as a good explanation of the error (аnd even it is in full agreement with your line of thought :) ) but even more interesting questions here are why is this row missing from phpbb_config table and is this just a bug in this specific version 2.0.20 or... something happened only in my installation? And also, if this is a real ("official") bug, why is it not being circumvented by the convertor? :?:

But anyway, if this is the real reason, even I have an idea how to circumvent it - it seems very easy :) , but... I will make some tests later today and will write here again :) ...

No... I checked more precisely and saw that this "default_language" setting actually does not work even only apparently which is the expected behavior when the respective row is missing in the phpbb_config table :) ...

User avatar
EA117
Registered User
Posts: 1050
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by EA117 » Tue Jul 30, 2019 3:45 pm

One thing I missed is that the error message in this case is actually attempting to show us the encoding string that is being declared "unknown":

trigger_error('Unknown encoding: ' . $encoding, E_USER_ERROR);

So the fact that the failure you're seeing says nothing where $encoding was appended:
General Error:
Unknown encoding:
in file /home/stringme/public_html/forum_new000/includes/utf/utf_tools.php on line 280
does confirm that ultimately phpbb_set_encoding() is ending up with a blank encoding value. That does still leave multiple possibilities, that perhaps the "'default_lang" value in the phpBB 2.0 configuration itself is blank; or it refers to a language name which isn't in the phpBB 2.0 /language directory; or the lang_main.php in that language directory for some reason doesn't define an encoding value (least likely).

A row named "default_lang" in the phpbb_config table indeed appears to be how phpBB 2.0 stored this configuration, same as phpBB 3.2.x. Although in phpBB 2.0 the name wasn't an ISO-defined value like "en" or "en-gb", and was instead "english" like you see used in the /language directory name. But simply looking at the "default_lang" value in the phpBB 2.0 database, and setting it to "english" if there isn't already a value there, may be a start.

I do note that although phpBB 2.0 itself appears to fall back to hard-coded "english" if the actual /language subdirectory corresponding to the "default_lang" value does not exist, the converter code (phpbb_set_encoding() specifically) is content to leave it blank under those circumstances. So indeed there could be a problem in conversion that wouldn't necessarily have existed on the phpBB 2.0 board itself.

StringMeteo72
Registered User
Posts: 4
Joined: Sat Jul 27, 2019 8:33 pm

Re: General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by StringMeteo72 » Tue Jul 30, 2019 5:29 pm

Yeeeeeeeeeee! It seems that your/my idea works! Firstly I fixed the bug with the missing "default_lang" (yes, firstly I checked that this is the exact name that the board 2.0.20 uses, not "default_language" :) ) in the phpbb_config table just by adding the missing row :) . Now the conversion is in progress and I hope it will be finished without any other problems. Many thanks once again!!! :D :D :D

User avatar
EA117
Registered User
Posts: 1050
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by EA117 » Tue Jul 30, 2019 6:08 pm

Excellent. Glad you figured out how to get the convert working; here's hoping that was the last issue before conversion success.

StringMeteo72
Registered User
Posts: 4
Joined: Sat Jul 27, 2019 8:33 pm

Re: General Error "Unknown encoding" during converting form phpBB2.0.20 to phpBB3.2.7

Post by StringMeteo72 » Tue Jul 30, 2019 6:33 pm

Well... it was only almost successful :oops: . Actually the new database looks full of all respective data (excepting indexing) but the new board is not working properly - it seems that any converted content is not being detected by the scripts... The reason is maybe because actually the converting stopped very shortly before the end without any error message...
Step 2 of 8

Processing last statements
update_folder_pm_count(); : Done
But this is a completely different case, I'll try to break it myself first before asking for help again :) ...

It is OK now! I needed just to use the "refresh" button to skip successfully through all the last steps :) ...

Post Reply

Return to “[3.2.x] Convertors”