[RFC] UMIL design proposal: Data providers

Discussion forum for MOD Writers regarding MOD Development.
User avatar
Former Team Member
Posts: 8024
Joined: Fri Dec 16, 2005 12:23 pm
Location: {postrow.POSTER_FROM}
Name: Igor Wiedler

[RFC] UMIL design proposal: Data providers

Post by igorw »

Status quo

UMIL in its current implementation makes it hard to call from an external application. This is because the data definition is tied to the execution. The UMIL install script (UMIL auto) contains an array of data and includes the UMIL auto library. This means an external application cannot re-use this file.

So what would use-cases for re-using it be? The most important one would of course be AutoMOD. Instead of forcing the user to call the script by himself manually it can be embedded into the MOD installation process. In fact a php-installer tag was recently added to MODX (1.2.4), which cannot fully taken advantage of though, due to the nature of UMIL.

A second use-case would be phpBB QuickInstall (or similar), a tool for automated installation of phpBB and MODs.

Proposed changes

In order to change this, the architecture of UMIL will need to be changed to some extent. By using OOP and abstraction it is possible to get a high amount of flexibility.

Data providers

The "data" are the "UMIL auto" instructions, currently held by the $versions array, in addition to the $mod_name, $version_config_name, $language_file. The idea is to put this data into a separate file. The file containing the data is the data provider.

By defining an interface it is possible to allow several formats to provide data. What I'm thinking of here is mainly PHP and XML, but things like YAML may be an interesting option too. In order to organise these data providers, there would be a directory within "/umil", possibly "/umil/data". The format can be detected using the file extension. The filename is basically just the MOD name. So AutoMOD would call its file "automod.(php|xml|yml)".

Model and data reader

The actual data should also be represented in PHP by a class. It will contain the data defined above. The $versions array can stay the same as it currently is, it would just be encapsulated inside a class. This is basically the "model".

To read the data provider into the model there are data readers. There's one of these readers for each format and they all implement a common interface. All the reader does is take a filename, create the model and then return it.

The rest of the UMIL system (as it currently exists) will then access the $versions array by using $model->versions or $model->get_versions().

Example design


Example YAML

Here is what such a YAML file could look like.

Code: Select all

mod_name: GitHub profile field
version_config_name: github_version
language_file: mods/github
            table: users
            column: user_github
            type: ['VCHAR_UNI', '']
    1.0.0-rc2: ~
    1.0.0: ~
Example PHP

And here a PHP example. I'll leave the XML one up to your imagination.

Code: Select all


return array(
	'mod_name'				=> 'GitHub profile field',
	'version_config_name'	=> 'github_version',
	'language_file'			=> 'mods/github',
	'versions'				=> array(
		'1.0.0-rc1'				=> array(
			'table_column_add'		=> array('users', 'user_github', array('VCHAR_UNI', '')),
		'1.0.0-rc2'				=> array(),
		'1.0.0'					=> array(),
Manually calling the installer

Of course the default use case of somebody manually installing the MOD also needs to be considered. Instead of the data provider including UMIL auto (status quo), UMIL auto should get the data from the data provider. Therefore the person installing the MOD should be directed to /umil/index.php?provider=github.yml. Or perhaps /umil/auto.php?....

Of course usage of all of this could be kept strictly optional. Especially if any of this is going to happen for phpBB 3.0.

Advanced considerations

With raw UMIL auto (-style) files you can already do quite a lot. But in certain cases it may be required to collect data from the user. This is a capability offered by UMIL and should also be possible in the future.

User data collection

In order to collect user data you will need some kind of form. The system should be designed in a way that an application can either directly provide data to it (think phpBB QuickInstall) or the form can be embedded within an other page (think AutoMOD). A bonus would be allowing a command-line data collection (for example) by extending the user data collection code.


There are several components, such as data readers, form elements, actual actions/commands (eg. table_column_add). It would be great if the system were flexible to allow several of these. for the data readers this is not important, but it should be possible for a mod author to include his own form elements and commands. Ideally these would be automatically loaded.

For example: I create a blog MOD for phpBB. I use all provided UMIL components for my install script. Then I notice that it lacks a command for adding users and a form element for selecting a specific user. These are both critical for my blog MOD. I could write a patch for them to be included in the next release of UMIL. But they are far too specific and only really needed for my MOD.

How it could/would work. Within /umil there is a place for these things. /umil/contrib/command and /umil/contrib/form are two folders that will contain custom commands and forms. I can use them in my data provider.

Data dumper

Like the data reader can read a data provider, a data dumper would do ther reverse. It would dump the model into various formats. This can be very flexible, as demonstrated by Symfony. Dumpers can be for the included XML/YAML/PHP formats, but it can also be exported into a PDF report or an HTML page, for example.


While there is a lot of room for improvement, there is an installer system I am building around the same concepts here [github.com]. Another project that also may be useful is zend frameworks' Zend_Form [framework.zend.com].
Igor Wiedler | area51 | GitHub | trashbin | Formerly known as evil less than three

User avatar
Former Team Member
Posts: 5666
Joined: Sun Aug 21, 2005 9:31 pm
Location: Wisconsin, U.S.
Name: Nathan

Re: [RFC] UMIL design proposal: Data providers

Post by EXreaction »

You can re-use it if you store that information in a separate file and then setup the main installer to just include it, that's what I've done in at least one of my modifications.

It was planned to have a system to manage installations for the files and just have files containing the information for the install process, but that was changed at some point.

The UMIL Auto system already supports optional form inputs using the acp board style array to set it up (not sure if there is much documentation, but it exists in the example file). Custom function calls are setup so you can do anything extra you may want/need.

I'd love to get UMIL rebuilt for these and many other reasons, but I don't have the time and nobody else has offered to take it up.

User avatar
Registered User
Posts: 482
Joined: Mon Apr 30, 2007 9:39 am
Location: Russia, Moscow, Khimki

Re: [RFC] UMIL design proposal: Data providers

Post by m157y »

2 evil<3
Khm, i thinks that php will be better here, we don't need spend time to parse it, just include the file, also most mod developers don't love xml :) what about yaml.. i think my opinion will be not objectively, because i don't like yaml. :mrgreen: And one more plus to PHP-schemas most authors also just moves their install-instructions(for UMIL, of course) from db_install.php(or something like this) to their schema file.
So, i don't sure that umil really need some another formats for schema. :roll:
2 EXreaction
If you need any help with UMIL, just pm :mrgreen: i don't like umil at this stage, so preferred own installer, but i thinks that umil can be better :) I thinks that evil<3 can update it, too.
m157y aka Misty
KarmaMOD for phpBB 3.0.x | bbAJAX
Follow me on twitter

User avatar
Former Team Member
Posts: 5666
Joined: Sun Aug 21, 2005 9:31 pm
Location: Wisconsin, U.S.
Name: Nathan

Re: [RFC] UMIL design proposal: Data providers

Post by EXreaction »

If you'd like to change some stuff please create a diff or patch and place it in the bug tracker.


Return to “[3.0.x] MOD Writers Discussion”