[howto] Write a converter with the Unified Convertor System

This is an archive of the phpBB 2.0.x convertors forum. Support for phpBB2 has now ended.
Forum rules
Following phpBB2's EoL, this forum is now archived for reference purposes only.
Please see the following announcement for more information: viewtopic.php?f=14&t=1385785
Ashe
Former Team Member
Posts: 642
Joined: Sun Jul 08, 2001 11:38 am

[howto] Write a converter with the Unified Convertor System

Post by Ashe »

I initially wanted to post it with my release announce but it was very long so I decided to split it out.

The UCS can be found here. A converter for BlazeBoard can be found here.
If you have any question or if you need help post here or send me a pm.

Ludovic 'Ashe' Arnaud

_____________

A converter can be broken down in three parts:
  • Converter infos
  • Translation schema
  • Functions
Converter infos

It is composed of 3 arrays: $converter_infos, $tables, $config_schema.

$converter_infos

Code: Select all

$converter_infos = array(
	'forum_name'	=>	'XMB Forum 1.8',
	'version'	=>	'1.0.1',
	'phpbb_version'	=>	'2.0.1',
	'table_prefix'	=>	'xmb_',
	'author'	=>	'<a href="mailto:[email protected]">Ashe</a>'
);
where forum_name is the name of the bulletin board system to convert,
version is the version of the converter,
phpbb_version is the version of phpBB it has been written for,
table_prefix (optional) is the default table prefix, it will be used if the user does not know if his/her forum uses one.
author is the string that will be echo'ed in the Author column. You can leave your email or your website address if you want to.
pre_message (optional) contains a message that will be displayed before the conversion actually begins. Then the user has the choice to continue and convert or to return to the Converters panel.

$tables

Code: Select all

$tables = array(
	'announcements',
	'banned',
	'category',
...snipped...
	'smilies',
	'thread'
);
$tables must hold the names of all the tables used by the forum you're converting from. They will be used to destroy the forum.

$config_schema
Holds a small conversion table for config settings. Among all recent PHP forums, config tables are stored in three different ways:
  • file-based, where everything is stored in a PHP array. Example: the config file is named config.php, it is stored in the config/ folder of the forum and the array is $config.
    table_format => 'file',
    filename => 'config/config.php',
    array_name => 'config'
  • in a single-row table, having one field per setting (like phpBB 1.x). In this example the table is named [table_prefix]settings, like xmb_settings.
    table_format => 'fixed',
    table_name => 'settings'
  • in a table, with only two fields (config_name, config_value) and one row per setting. (like phpBB 2.x). In this example, the table is identical to phpBB2's config table phpbb_config:
    table_format => array('config_name'=>'config_value'),
    table_name => 'config'
Another field called "settings" is required, it's an array where each key is the name of the config setting used by phpBB and the corresponding value is the name of the config setting used by the forum you're converting from. You can also specify a function to apply to this settings. Example:

Code: Select all

$config_schema = array(
	'table_name'	=>	'config',
	'table_format'	=>	array('config_name' => 'config_value'),
	'settings'		=>	array(
		'board_disable'		=>	'not(board_enabled)',
		'hot_threshold'		=>	'hottopic',
		'topics_per_page'	=>	'topicsperpage'
	)
);
In this example, the table is name [table_prefix]config, it has two fields named 'config_name' and 'config_value'. The phpBB config setting 'board_disable' is set with the value returned by not() (a built-in function that works like the PHP function empty()) used with 'board_enabled'.


Translation schema

The core of your converter. It's an array that must be called $converter. Here are all the fields you can use:
  • execute_first (string) (optional)
    PHP code that will be eval'ued at the beggining of the conversion.
  • query_first (string|array) (optional)
    query_first can either be a string or an array of strings. It holds all the SQL queries you want to execute at the beggining of the conversion. They are processed after execute_first.
  • execute_last (string) (optional)
    PHP code that will be eval'ued at the end of the conversion.
  • query_last (string|array) (optional)
    Identical to query_first, the queries will be executed after execute_last.
  • test_file (string) (optional - required if you use any of the file management functions)
    Holds the filename of any of the forum file, it will be used to make sure the relative filepath is correct. Please do not use files like 'index.php' that are not specific enough. For phpBB2 you would use 'admin/pagestart.php'.
  • avatars_path (string) (optional - required if you use import_avatar() or import_avatar_gallery())
    Path to the avatars directory. Relative to the base directory of the forum you're converting from, if the dir is /mysite/vbulletin/avatars_storage/ then 'avatars_path' will be 'avatars_storage/'. If the target directory does not exist it will be created, if it can't be the user will be prompted to do so.
  • ranks_path (string) (optional - required if you use import_rank())
    Identical to avatar_path but for ranks.
  • smilies_path (string) (optional - required if you use import_smilie())
    Identical to avatar_path but for smilies.
  • schema (array) (required)
    The biggest part, translation tables.
$converter['schema']

$converter['schema'] is an array where each element describes how a table must be imported. Example:

Code: Select all

array(
	'target'		=>	CATEGORIES_TABLE,
	'query_first'	=>	'DELETE FROM ' . CATEGORIES_TABLE,

	array('cat_id',					'category.CID',					''),
	array('cat_title',				'category.category',			'stripslashes'),
	array('cat_order',				'category.displayorder',		'')
)
Here is a description of the all the available fields:
  • target (string) (required)
    The name of the target table. Always use constants for them.
  • execute_first (string) (optional)
    Identical to the higher level execute_first. There is no execute_last.
  • query_first (string|array) (optional)
    Identical to the higher level query_first. Since target tables aren't automatically emptied, most of the time you will use this field to do so. There is no query_last.
  • Array elements with no key
    Each of those will be an array with three fields. The first one holds the name of the target field, the second one is the names of source table and source field (more on this later in this document). The third one is the function to use. In the example, the field cat_id of table CATEGORIES_TABLE will be field with stripslash'ed result of field category, from the source table [table_prefix]category. Note that you never have to mention the table prefix of any table, they are automagically added.
  • left_join (string|array) (optional)
    Formats:
    tablename LEFT JOIN joined_table ON <clause>
    tablename LEFT JOIN joined_table USING (field)
    tablename LEFT JOIN joined_table AS alias ON <clause>

    You can specify one or several LEFT JOIN clauses.
  • where (string) (optional)
    Ex: 'where' => 'members.userid > 1 OR members.status = "Administrator"'
    Standard WHERE clause.
  • group_by (string) (optional)
    Ex: 'group_by' => 'ranks.titlename'
    Standard GROUP BY clause.
  • having (string) (optional)
    Standard HAVING clause. Note that only MySQL allows to use HAVING with aliases.
  • order_by (string) (optional - recommended)
    Standard ORDER BY clause.
  • distinct (boolean) (optional) new
    Set 'distinct' to TRUE to make the select query uses DISTINCT.
Basic rules: fields
Each element has three fields: target, source, function.
target is the target database field. (optional)
function the function to apply before values are inserted. (optional)
source is the source of data. It can use several formats, here is a quick description with the way it will be translated to SQL language:

table.field
SELECT table.field FROM [prefix]table table

table_alias.field (deprecated)
SELECT table_alias.field FROM ? table_alias

table.field AS alias
SELECT table.field AS alias FROM [prefix]table table

table.field AS table_alias.alias
SELECT table_alias.field AS alias FROM [prefix]table table_alias

table.field AS table_alias.field
SELECT table_alias.field FROM [prefix]table table_alias

The second form is deprecated, but can be used in conjunction with a LEFT JOIN using an alias, otherwise the script has no way to know what table is this alias for. Note that you can also specify an abolute value instead of a fieldname and that value will be treated like if it was returned from the database.

Basic rules: logic

For each element:
  • If target isn't empty, it will be filled with the result of the row.
  • If source is a fieldname, it will be queried.
  • If function isn't empty, the value of source will be passed.
i.e. you can leave target and function empty if you need source to be queried for any reason.


Functions

There are three types of functions:
  • global functions, that are used for the script itself or can be used in a converter. They can be found in functions_convert.php.
  • general functions to use directly in translation tables. For example, inc() will increment the passed var then return it. str_to_userlevel will translate english levels (Administrator, Moderator, etc...) into their numeric equivalence (ADMIN, MOD, etc...). They can be found in functions_convert.php.
  • specific functions. Functions specific to a converter, they are stored at the end of the converter file. Always prefix them with the converter tag or shortname to avoid conflicts.
I will detail available functions as soon as possible, here are a few tips to remember:
  • Know phpBB coding standards by heart ;)
  • Never assume that the entire table will be processed at once, it can be stopped at any time.
  • Results of the current row are available through $row. Remember to declare it with global at the beggining of the function.
  • If, for any reason you don't want the current row to be inserted set $sql_flag to FALSE. Remember to declare it with global at the beggining of the function.
  • ...more tips will come :D
Last edited by Ashe on Thu Jul 25, 2002 3:33 am, edited 2 times in total.
Ashe
Former Team Member
Posts: 642
Joined: Sun Jul 08, 2001 11:38 am

Post by Ashe »

Functions to use in table schema

These functions only require one parameter and can be used in table schema.


auto_id($pad=0)
It can be used to auto-increment a field. Used by itself it will return the offset of the current row (the same offset that you'd use in a LIMIT clause), if you pass a value it will add them up and return the result. Instead of using the row offset you can indicate to use another value by naming one of the result as max_id
For instance, some forums store announcements in a separate table. When you import them as topic you will use:

Code: Select all

array('topic_id',      'announcements.id',              'auto_id'),
array('',              'MAX(threads.id) AS max_id',     ''),
In this example, if the highest threads.id is 5, the lowest topic_id will be 6.

auto_group_id($group_id)
This function, derived from auto_id() can be used to restore custom groups. Every member is assigned to a personnal group, since Anonymous already takes group_id 1 the first user will take group_id 2 and so on. (user_id 4 will be assigned to group_id 5) When you import custom groups you will have to query the highest user id, this value will be added up to the current offset and incremented by one.

Code: Select all

array('group_id',      'members.userid',                'auto_group_id'),
array('',              'MAX(members.id) AS max_id',     ''),
In this example, if the highest members.userid is 4, the lowest group_id will be 6.

dec($var)
Returns ($var - 1)

days_to_seconds($group_id)
Converts days to seconds. (returns $days * 86400)

dec($var)
Returns ($var - 1).

encode_banip($ip)
Returns $ip formatted to be inserted in BANLIST_TABLE.

import_avatar($src, $trg='')
Can either be used in a table schema or in a custom function. If $trg is specified, it is relative to the avatars directory. $src is the path to the avatar, relative to the source forums directory or its avatars directory. In other words, pass whatever is stored in the database and it will sort it out ;)
This function will simply copy the avatar file from its original location to phpBB avatars directory. You must specify $converter['avatars_path'] to use this function.

import_rank($src)
Same as above, for ranks. You must specify $converter['ranks_path'] to use this function.

import_smilie($src)
Same as above, for smilies. You must specify $converter['smilies_path'] to use this function.

is_forum_locked($bool)
If $bool, it will return FORUM_LOCKED. Otherwise it will return FORUM_UNLOCKED.

is_pm_new($bool)
Returns PRIVMSGS_NEW_MAIL if $bool, PRIVMSGS_READ_MAIL otherwise.

is_pm_read($bool)
If $bool, it will return the constant value for a read mail (PRIVMSGS_READ_MAIL), otherwise it will return PRIVMSGS_UNREAD_MAIL.

is_smilie_new($code)
Will search SMILIES_TABLE for $code, if it is not already in the table it will return $code. Otherwise it will return FALSE and will prevent the current row from being inserted in the database.

is_topic_locked($bool)
If $bool, it will return TOPIC_LOCKED. Otherwise it will return TOPIC_UNLOCKED.

is_topic_sticky($bool)
If $bool, it will return POST_STICKY. Otherwise it will return POST_NORMAL.

make_uid($var='')
(If $var isn't specified it will use the current row offset)
Returns a bbcode_uid based on $var.

not($var)
Will return !$var.

str_to_bool($str)
Some forums store TRUE/FALSE (1/0) as "yes"/"no" or "on"/"off" (no comment ;)), this function will convert them into their numeric value.

str_to_userlevel($var)
Some forums store users levels as "Administrator", "Moderator", "Guest", etc... This function will convert them into their correct numeric value.

validate_icq($icq)
Returns $icq if it is a valid ICQ number, or return NULL otherwise.



General functions


convert_bbcode($message, $convert_size=TRUE, $extended_bbcodes=FALSE)
This function will convert bbcodes that aren't supported by phpBB. ([php] and [sql] will become

Code: Select all

, [font] will be stripped) It will also convert font size from pt to px. (proportional / pixel) If specified, it will also remove/convert extended bbcodes that can be found in some forums softwares like [glow], [shadow], [table] and a bunch of other stup... I mean [i]useful[/i] bbcodes that aren't supported by phpBB.

[b][color=blue]copy_file([/color][color=brown]$src[/color][color=blue], [/color][color=brown]$trg[/color][color=blue], [/color][color=brown]$overwrite[/color][color=blue]=FALSE, [/color][color=brown]$die_on_failure[/color][color=blue]=TRUE)[/color][/b]
A function masterpiece :D
It will copy [color=brown]$src[/color] to [color=brown]$trg[/color] where $src is the filename relative to the source forum and $trg is the filename relative to the phpBB installation. You [b]don't[/b] have to care about the absolute path of the forums or the relative path from phpBB to the source forum.

[u]Example:[/u] you want to copy '/site/sourceforum/storage/thing.zip' to '/site/phpBB2/files/thing.zip', you will use [b]copy('storage/thing.zip', 'files/thing.zip');[/b].

If [color=brown]$overwrite[/color], the destination will be overwritten.
If [color=brown]$die_on_failure[/color], the script will stop if it can't copy the file. (if the target directory isn't writable or if [color=brown]$overwrite[/color] has been set to FALSE and the file already exists)

[b][color=blue]copy_dir([/color][color=brown]$src[/color][color=blue], [/color][color=brown]$trg[/color][color=blue], [/color][color=brown]$copy_subdirs[/color][color=blue]=TRUE, [/color][color=brown]$overwrite[/color][color=blue]=FALSE, [/color][color=brown]$die_on_failure[/color][color=blue]=TRUE)[/color][/b]
Same as copy_file() but for dirs. If [color=brown]$copy_subdirs[/color], subdirectories will be created if necessary and copied as well, otherwise only files will be.

[b][color=blue]fix_ghost_forums()[/color][/b]
This function is executed within the main script at the end of the conversion, don't use it.

[b][color=blue]get_config()[/color][/b]
Returns an associative array containing the source forum config settings.
Use get_config_value() instead.

[b][color=blue]get_config([/color][color=brown]$config_name[/color])[/b]
Allow you to access to the config table of the source forum. Returns $config[$config_name] where $config has been created by get_config().

[b][color=blue]mass_auth([/color][color=brown]$forum_id[/color][color=blue], [/color][color=brown]$type[/color][color=blue], [/color][color=brown]$list[/color][color=blue], [/color][color=brown]$auth_fields[/color][color=blue], [/color][color=brown]$separator[/color][color=blue]=',')[/color][/b]
Allows you to quickly set auth option for multiple users.
[color=brown]$forum_id[/color] - self-explanatory ;) Can be a non-associative array.
[color=brown]$type[/color] - can be "user_id", "username", "group_id" or "group_name".
[color=brown]$list[/color] - the list of users you want to modify. It can be an array where each element is an user (or group id) or a string where all users/groups are concatened and separated by [color=brown]$separator[/color].
[color=brown]$auth_fields[/color] - must be a string containing all the auth fields you want to set to 1. They can be separated by spaces, commas or anything else (the function uses a regexp anyway :D).
[color=brown]$separator[/color] - if you have to use something else than commas to separate users/groups in [color=brown]$list[/color] pass it with [color=brown]$separator[/color].

[u]Examples:[/u]
[list][*]mass_auth(12, 'username', 'John, Paul, RfX99', 'auth_view, auth_read, auth_post');
[i]Will give users "John", "Paul" and "RfX99" access to the forum whose forum_id is 12.[/i]

[*]mass_auth(array(1,2,3,4,6), 'groupname', 'SuperModerators', 'auth_mod');
[i]Will make the group "SuperModerators" moderator of forums 1, 2, 3, 4, 6.[/i]

[*]mass_auth(1, 'user_id', array(1,2,5), 'auth_mod');
[i]Will make users whose user_id are 1, 2 and 5 moderators of forum 1.[/i][/list]Note that when users are set moderators their user_level is set to MOD if it was USER.

[b][color=blue]save_config([/color][color=brown]$config_name[/color][color=blue], [/color][color=brown]$config_value[/color][color=blue])[/color][/b]
Saves a phpBB config setting. If it doesn't exist a new row will be inserted so be careful and make sure that [color=brown]$config_name[/color] is an actual phpBB setting. If necesserary, [color=brown]$config_value[/color] will be slashed.
User avatar
cwcollector
Registered User
Posts: 353
Joined: Tue Nov 27, 2001 1:25 pm
Contact:

Converter

Post by cwcollector »

Does this converter system require a mysql database or could be used to convert flat file databases?

Chuck
Ashe
Former Team Member
Posts: 642
Joined: Sun Jul 08, 2001 11:38 am

Post by Ashe »

At present time it will only work with a database, preferably MySQL but it can work with about anything.
Chisoxfn19
Registered User
Posts: 5
Joined: Sun Nov 24, 2002 1:35 am

Post by Chisoxfn19 »

Is this the same converter that you can download on the main page?

Also, I notice the stuff you wrote up their had 1.8 which is what I'm currently using. Will the 1.6 one work to convert the 1.8 to phpbb?

I already uploaded and installed the newest version of phpbb at www.soxtalk.com/phpbb

My xmb board is located at www.soxtalk.com

Now I installed the converters into the phpbb section and then converters. The question is how do I convert it? Will it not work cause they are in different sections(Ie. xmb is strictly under www.soxtalk.com while phpbb is www.soxtalk.com/phpbb

Also, if thats the case, then how can I keep my xmb board up while converting to the php board?

Please lead me in the right direction. You seem to know what your talking about, and well, I am figuring things out, but I'm not expert.

So far have I done things right, ie install it first then run the converter?

Also, if you can help me, what exactly does the conveter convert? Is it the posts and all the members info, meaning they won't have to reregister?

Thanks so much

Once these questions are answered I can complete the switch, spend a couple days customizing it for my sites needs and then switch it over to www.soxtalk.com where it will become the main board.
Narga
Registered User
Posts: 2
Joined: Fri Dec 13, 2002 6:00 am

Post by Narga »

what's a way to convert database from ibf ??
omega13a
Registered User
Posts: 88
Joined: Wed Feb 20, 2002 11:53 pm
Location: SF Bay Area
Contact:

Post by omega13a »

I like to know that too... I'm trying to make one for Invision Board 1.1 and I'm blowing my brains out... :x

It transphered the stuff I wanted but I get some (I mean a lot! :x ) mySQL errors on my board now...
ReNeGaDeSoLjA
Registered User
Posts: 37
Joined: Fri Jun 21, 2002 8:23 pm
Contact:

Post by ReNeGaDeSoLjA »

i believe somone is close to releasing an ibf convertor. check the thread , its currently the longest going thread about an IBF convertor.
Seether
Registered User
Posts: 74
Joined: Thu Apr 10, 2003 8:09 pm

Post by Seether »

Any idea to convert from a rapidforum?


S.
HelloHi
Registered User
Posts: 12
Joined: Wed Jun 25, 2003 3:02 pm

Post by HelloHi »

yeah i'm interested in rapid forum
User avatar
SupremeWeapon
Registered User
Posts: 593
Joined: Mon Feb 17, 2003 12:48 pm
Location: Battle Creek, Mi
Contact:

Post by SupremeWeapon »

can this method be used to convert Suddenlaunch? i know there are conerters for others like ezboard... but have yet to find one for sudden launch....
Image
loomy
Registered User
Posts: 1
Joined: Fri Aug 22, 2003 12:19 am

Post by loomy »

Chisoxfn19 wrote: Also, I notice the stuff you wrote up their had 1.8 which is what I'm currently using. Will the 1.6 one work to convert the 1.8 to phpbb?
I've been trying to figure out the exact same thing.. does anyone know?
Dark Hedgehog
Registered User
Posts: 50
Joined: Sun Aug 24, 2003 6:43 am

Post by Dark Hedgehog »

My converter won't work at all, and here is the url for it:

Convert url

Thanks if anyone can help.
pkevin
Registered User
Posts: 1
Joined: Fri Sep 05, 2003 3:23 pm

is there any convertor for leoboard to PHPBB

Post by pkevin »

please help
I plan to change my leoboard to PHPBB
samy56
Registered User
Posts: 26
Joined: Sat Oct 04, 2003 2:19 pm

Post by samy56 »

ya nice is there any translator available
Locked

Return to “[2.0.x] Convertors”