Building the transport package without installing MODX

Writing your packages for MODX is not easy for a beginner, and even an experienced developer sometimes has a hard time. But the novice is frightened, and the experienced understands :).

This note describes how to write and assemble a component package for MODX without installing and configuring MODX itself. The level is above average, so you may have to break your head on occasion, but it's worth it.

For details, I ask under the cat.

Once, when MODX Revolution had just appeared, it was still in the early beta version, developers still did not know how to work with it and how to write plugins for it. Well, except for the team that pored over the CMS. And the team, I must say, partially succeeded and provided in the system itself the ability to conveniently assemble packages, which can then be installed through a repository, which looks logical. But many years have passed since then and the requirements for packages and their assembly have changed a bit.

Copypasta is evil, though not always

For the past few months, I wondered why, in order to build a package for MODX, you need to install it, create a database, create an admin, etc. So much extra action. No, there is nothing wrong with that, if you set it up once and then use it. Many do. But what to do when you want to entrust the assembly to the script, and go yourself to drink some coffee?

It so happened that the creators of MODX got used to working with MODX itself and added classes directly to the kernel that provide package building. They wrote the first components, the first build-scripts, which were later used as examples by other developers who simply copied the solution, not always really delving into the essence of what is happening. And I did that.

But the task is to automate the assembly of the package, preferably on the server, necessarily with the minimum set of required software, with the minimum expenditure of resources and therefore with greater speed. The task was set and after examining the sources, Jason was hampered by a solution in the chat.

And what?

The first thing I found out is that the code responsible for building the package directly lies in the xPDO library, and in MODX only the wrapper classes provide a more convenient API and with which it is somewhat easier to work, but only if MODX is installed. Therefore, probably somehow you can use only xPDO, but in the code, the xPDO object constructor requires you to specify data for the connection to the database.

publicfunction__construct(
    $dsn, 
    $username = '', 
    $password = '', 
    $options = [], 
    $driverOptions= null
);

After Jason's inquiries, it became clear that although the parameters had to be set, the real physical connection to the database occurred exactly at the moment when it was necessary. Lazy load in all its glory. The second problem was solved.

The third problem was the issue of connecting xPDO to the project. Composer immediately came to mind, but the 2.x version, on which the current MODX runs, does not support Composer, and the 3.x branch uses namespaces and class names are written differently from 2.x, which leads to conflicts and errors. In general, incompatible. Then I had to use git tools and connect xPDO as a submodule.

How to use submodules

To get started is to read the documentation on them.

Then, if this is a new project, you need to add a submodule:

$ git submodule add https://github.com/username/reponame

This command will incline and install a submodule in your project. Then you will need to add the submodule folder to your repository with the git add command. It will not add the entire folder with the submodule, but will add to git only a link to the last commit from the submodule.

In order for another developer to clone a project with all dependencies, you need to create a .gitmodules config for submodules. In the Slackify project, he is:

[submodule "_build/xpdo"]
  path = _build/xpdo
  url = https://github.com/modxcms/xpdo.git
  branch = 2.x

After that, when cloning, just specify the recursive flag and git will download all dependent repositories.

As a result, we have xPDO, xPDO can be used without connecting to the database, if it is not necessary, xPDO can be connected to the component code as an external dependency (git submodule). Now the implementation of the build script.

Let's understand

I will describe the build script of the Slackify add-on I recently posted . This component is free and available on Github to make it easy to learn on your own.

Connecting xPDO

Omit the task of the constants with the package name and other necessary calls and connect the xPDO.

require_once'xpdo/xpdo/xpdo.class.php';
require_once'xpdo/xpdo/transport/xpdotransport.class.php';
$xpdo = xPDO::getInstance('db', [
    xPDO::OPT_CACHE_PATH => __DIR__ . '/../cache/',
    xPDO::OPT_HYDRATE_FIELDS => true,
    xPDO::OPT_HYDRATE_RELATED_OBJECTS => true,
    xPDO::OPT_HYDRATE_ADHOC_FIELDS => true,
    xPDO::OPT_CONNECTIONS => [
        [
            'dsn' => 'mysql:host=localhost;dbname=xpdotest;charset=utf8',
            'username' => 'test',
            'password' => 'test',
            'options' => [xPDO::OPT_CONN_MUTABLE => true],
            'driverOptions' => [],
        ]
    ]
]);

I added the xPDO submodule to the folder _buildthat we need only during the development and build phase of the package and which will not be included in the main component archive. The second copy of xPDO on the site with live MODX is not needed.

In the xPDO connection settings, I set the dsnname of the database, but it does not play any role. It is important that the cache folder inside the xPDO is writable. That's all, xPDO is initialized.

Making a tricky hack with classes

When creating a package uses the installed MODX, everything is simple, we take and create an object of the class we need. MODX actually finds the necessary class, finds the necessary implementation for this class (the class with the _mysql postfix), which depends on the database and then creates the necessary object (because of this peculiarity, you may encounter errors when building the package * _mysql not found, it's not scary). However, we have neither a base nor an implementation. We need to somehow replace the desired class, which we do.

classmodNamespaceextendsxPDOObject{}
classmodSystemSettingextendsxPDOObject{}

We create a dummy class (stub), which is needed to create the desired object. This would not have to be done if xPDO had not checked in a special way which class the object belongs to. But he checks.

But there are special cases when you need to do a little more than just define a class. These are cases of dependencies between classes. For example, we need to add a plugin to the category. The code is simple $category->addOne($plugin);, but in our case it will not work.

If you have ever looked at the MODX database schema , you probably have seen such elements as aggregate and composite. It is written about them in the documentation , but if in simple terms, they describe the interconnections between the classes.

In our case, there may be several plug-ins in a category, for which in the classmodCategoryresponds to the aggregate element. Consequently, since we have a class without a concrete implementation, we need to point this connection with our hands. It is easier to do this by overriding the method getFKDefinition:

classmodCategoryextendsxPDOObject{
    publicfunctiongetFKDefinition($alias){
        $aggregates = [
            'Plugins' => [
                'class' => 'modPlugin',
                'local' => 'id',
                'foreign' => 'category',
                'cardinality' => 'many',
                'owner' => 'local',
            ]
        ];
        returnisset($aggregates[$alias]) 
               ? $aggregates[$alias] 
               : [];
    }
}

In our component, only plugins are used, so we add links only for them. After this, the addMany method of the modCategory class will be able to add the necessary plugins to the category without any problems and then to the package.

Create a package

$package = new xPDOTransport($xpdo, $signature, $directory);

As you can see, everything is very, very simple. Here it was necessary for us to transfer by parameter $xpdowhich we initialized right at the beginning. If not for this moment, problem 2 would not exist. $signature- package name, including version, $directory- the place where the package will be carefully put. Where do these variables come from?

Create a namespace and add it to the package.

We need the namespace in order to attach lexicons and system settings to it. In our case, only for this, others are not yet considered.

$namespace = new modNamespace($xpdo);
$namespace->fromArray([
    'id' => PKG_NAME_LOWER,
    'name' => PKG_NAME_LOWER,
    'path' => '{core_path}components/' . PKG_NAME_LOWER . '/',
]);
$package->put($namespace, [
    xPDOTransport::UNIQUE_KEY => 'name',
    xPDOTransport::PRESERVE_KEYS => true,
    xPDOTransport::UPDATE_OBJECT => true,
    xPDOTransport::RESOLVE_FILES => true,
    xPDOTransport::RESOLVE_PHP => true,
    xPDOTransport::NATIVE_KEY => PKG_NAME_LOWER,
    'namespace' => PKG_NAME_LOWER,
    'package' => 'modx',
    'resolve' => null,
    'validate' => null
]);

The first part is clear to anyone who has ever written code for MODX. The second, with the addition of the package, a little more complicated. The method puttakes 2 parameters: the object itself and an array of parameters describing this object and its possible behavior at the time the package is installed. For example, it xPDOTransport::UNIQUE_KEY => 'name'says that for a namespace a field namewith the name of the namespace itself will be used as a value as a unique key in the database . More information about the parameters can be found in the documentation on xPDO , or better having studied the source code.

In the same way, you can add other objects, such as system settings.

$package->put($setting, [
    xPDOTransport::UNIQUE_KEY => 'key',
    xPDOTransport::PRESERVE_KEYS => true,
    xPDOTransport::UPDATE_OBJECT => true,
    'class' => 'modSystemSetting',
    'resolve' => null,
    'validate' => null,
    'package' => 'modx',
]);

Create a category

With the addition of a category, I had the biggest gag when I understood all this. The elements put into a category in the xPDO model must both belong to this category, i.e. to be nested in it, and only then the category itself should be nested in a package. And at the same time, it is necessary to take into account the interrelations between the classes that I have already described above. It took quite a lot of time to understand this, realize and apply it correctly.

$package->put($category, [
    xPDOTransport::UNIQUE_KEY => 'category',
    xPDOTransport::PRESERVE_KEYS => false,
    xPDOTransport::UPDATE_OBJECT => true,
    xPDOTransport::ABORT_INSTALL_ON_VEHICLE_FAIL => true,
    xPDOTransport::RELATED_OBJECTS => true,
    xPDOTransport::RELATED_OBJECT_ATTRIBUTES => [
        'Plugins' => [
            xPDOTransport::UNIQUE_KEY => 'name',
            xPDOTransport::PRESERVE_KEYS => false,
            xPDOTransport::UPDATE_OBJECT => false,
            xPDOTransport::RELATED_OBJECTS => true
        ],
        'PluginEvents' => [
            xPDOTransport::UNIQUE_KEY => ['pluginid', 'event'],
            xPDOTransport::PRESERVE_KEYS => true,
            xPDOTransport::UPDATE_OBJECT => false,
            xPDOTransport::RELATED_OBJECTS => true
        ]
    ],
    xPDOTransport::NATIVE_KEY => true,
    'package' => 'modx',
    'validate' => $validators,
    'resolve' => $resolvers
]);

It looks monstrous, but not so seen. An important parameter xPDOTransport::RELATED_OBJECTS => truethat says that the category has nested elements that also need to be packaged and then installed.

Since most modules contain various elements (chunks, snippets, plug-ins), the category with elements is the most important piece of the transport package. Therefore, it is here that validators and resolvers are set that are executed during the package installation process.

Validators are performed before installation, resolvers - after.

I almost forgot, before packing the category, we need to add our elements to it. Like this:

$plugins = include $sources['data'] . 'transport.plugins.php';
if (is_array($plugins)) {
    $category->addMany($plugins, 'Plugins');
}

Add other data to the package

In the package you need to add another file with the license, a file with a change log and a file with a description of the component. If necessary, you can add another special script through an attribute setup-optionsthat shows the window before installing the package. This is when instead of "Install" button "Installation Options". And since MODX 2.4, it has requiresbecome possible to specify dependencies between packages using an attribute , and it is also possible to specify the version of PHP and MODX in it.

$package->setAttribute('changelog', file_get_contents($sources['docs'] . 'changelog.txt'));
$package->setAttribute('license', file_get_contents($sources['docs'] . 'license.txt'));
$package->setAttribute('readme', file_get_contents($sources['docs'] . 'readme.txt'));
$package->setAttribute('requires', ['php' => '>=5.4']);
$package->setAttribute('setup-options', ['source' => $sources['build'] . 'setup.options.php']);

We pack

if ($package->pack()) {
    $xpdo->log(xPDO::LOG_LEVEL_INFO, "Package built");
}

Everything, we take away a ready package in _packages, well or from there where you configured the assembly.

What is the result?

The result exceeded my expectations, because such an approach, although it imposes some restrictions and sometimes adds some inconvenience, gains in application possibilities.

To build a package, just run 2 commands:

git clone --recursive git@github.com:Alroniks/modx-slackify.git
cd modx-slackify/_build && php build.transport.php

The first is the cloning of the repository and its submodules. An important parameter --recursive, thanks to it, git will download and install, in addition to the component code itself, all the dependencies described as submodules.

The second is building the package directly. After that, you can take the finished package-1.0.0-pl.transport.zipfrom the folder _packagesand download it, for example in the repository.

The prospects are broad. For example, you can set up a hook in GitHub, which, after a commit to a branch, will run a script on your server that will compile the package and put it in all sites that you have. Or you will download a new version in some repository, and at this time you will make coffee for yourself, as I said in the beginning. Or you can invent and write tests to the module and run the test run and build through Jenkins or Travis. Yes, a bunch of scenarios can be invented. With this approach, doing it is now much easier.

Ask questions, try to answer.

PS Don't pass by, put a Slackify star on GitHub , please.