Using Composer Packages in XenForo 2.1+ Addons Tutorial

Compatible XF 2.x versions

Additional requirements: You must have Composer installed

Composer is a tool for dependency management in PHP. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.

XenForo v2 uses Composer behind the scenes to include certain packages used by the core software. As addon developers, we can include Composer packages in our own addons which will be autoloaded alongside those provided by the core.

In XenForo 2.0, we had to use extension points to do our own autoloading - but as of XenForo 2.1, the core software makes it really easy for us to include Composer packages. This tutorial describes the process to do so and how to build your addons which utilise composer.

Note that there are some caveats here - dependency management can be complex, and it is entirely possible for you to introduce unexpected bugs in your addon, other addons, or even in the XenForo core by following this tutorial and including composer packages. Care must be taken and you would be well advised to avoid this approach if you are not already familiar with Composer dependency management.

Assumptions made in this tutorial:

you have Composer installed on your development server and you are familiar with how to use it
you understand how Composer packages work
all examples assume a Linux environment running bash, you will need to translate these for use on Windows or other platforms yourself

About this tutorial

This tutorial runs through a basic example of adding a Composer package to an addon. The addon itself is installable so you can see all of the code in operation and examine how it works. We will install the Carbon API extension for DateTime objects.

Note that the instructions in this tutorial are specific to XenForo v2.1 and later versions - please see my previous tutorial Using Composer Packages in XenForo 2.0 Addons Tutorial for instructions for v2.0, which requires quite a bit more effort compared to v2.1.

All code in this tutorial is licensed under the MIT license, which essentially allows you to use (or modify!) the code for any purpose (including commercial purposes) free of charge, with the only condition being that you include the relevant copyright notice and permission notice.

See the LICENSE file in the addon root for full license and copyright information. Alternatively, you can view the license file in the Git repository for the tutorial code: LICENSE.

Just FYI, Composer itself and the Carbon package we will install also use the MIT license.

Getting Started

I have created a basic addon called ComposerTutorial.

XenForo is installed on my dev server at /srv/www/xenforo21 - I will refer to this as the "XenForo root". My addon is installed at /srv/www/xenforo21/src/addons/ComposerTutorial - I will refer to this as my "addon root".

The first step to adding a Composer package is to create a composer.json file in the root of your addon. You can either do this by hand, or use the composer require command to do it for you.

Bash:

$ cd /srv/www/xenforo21/src/addons/ComposerTutorial
$ composer require nesbot/carbon
Using version ^2.25 for nesbot/carbon
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 4 installs, 0 updates, 0 removals
- Installing symfony/translation-contracts (v1.1.7): Loading from cache
- Installing symfony/polyfill-mbstring (v1.12.0): Loading from cache
- Installing symfony/translation (v4.3.5): Loading from cache
- Installing nesbot/carbon (2.25.1): Loading from cache
symfony/translation suggests installing symfony/config
symfony/translation suggests installing symfony/yaml
symfony/translation suggests installing psr/log-implementation (To use logging capability in translator)
Writing lock file
Generating autoload files

We now have a basic Composer file created at /srv/www/xenforo21/src/addons/ComposerTutorial/composer.json:

JSON:

{
"require": {
"nesbot/carbon": "^2.25"
}
}

While there are dozens of fields available in the composer.json Schema - you actually only need the require field to make Composer work.

If you created your composer.json file by hand, you would instead run the composer update command to install the package and its dependencies.

Bash:

$ cd /srv/www/xenforo21/src/addons/ComposerTutorial
$ composer update

The other thing we now have is a vendor folder containing the installed packages and all dependencies.

Bash:

$ ls -1p /srv/www/xenforo21/src/addons/ComposerTutorial/vendor/
autoload.php
bin/
composer/
nesbot/
symfony/

The autoload.php file is auto-generated by the composer commands and is for running the autoloader in a stand-alone project. This won't be suitable for using in XenForo, so we will ignore this file.
The bin directory is where executable files and scripts are kept. You don't typically need to use these.
The composer directory is where the generated autoloader files are - it's worth familiarising yourself with the contents of this directory to understand how Composer finds and loads classes. We will be using some of the files in this directory to identify the classes and files we need to pass to the XenForo autoloader.
The nesbot directory is where the Carbon package is installed.
The symfony directory is where some Carbon dependency packages are installed (symfony/polyfill-mbstring and symfony-translation)

Finally, we have a composer.lock file that was created in our addon root directory. This file is auto-generated by the Composer commands and contains a list of the dependencies (and their specific versions!) that were resolved from composer.json the first time the packages were installed. You should consider the composer.lock file a "version-specific point-in-time snapshot" of the dependencies.

While you are developing your addon, you may run the composer update command from your addon root to update the dependencies, which will cause the composer.lock file to also be updated if new versions have been released. It is important to test that new versions have not included any breaking changes - although if using semantic versioning, this should (in theory) not occur.

However, in a production environment, we don't ever run the composer update command - we want to ensure we are running with versions of packages we have tested. Instead, we run the composer install command which relies on the composer.lock file to install exactly the versions specified.

Either way, we'll cover installation in more detail later. For now, just recognise that the composer.lock file is important and should be checked into your source code control alongside your composer.json file.

So now that we have our package and dependencies installed, we need to get XenForo to autoload everything.

XenForo Autoloader

As mentioned previously, XenForo uses Composer behind the scenes - although they hide the composer.json file from us, because we don't need to install or update any core packages and dependencies ourselves - their upgrade process takes care of that for us. See the src/vendor directory under the XenForo root to see the packages used by XenForo.

XenForo 2.1 introduces a handy new addon.json parameter composer_autoload which we can set to ask the system to include our required packages in the autoloader.

Add the following line to your addon.json file:

JSON:

"composer_autoload": "vendor/composer"

This tells the XenForo autoloader to load the packages from our addon as well - the path is relative to our addon root directory (ie /srv/www/xenforo21/src/addons/ComposerTutorial/vendor/composer).

So my addon.json for this tutorial package now becomes:

JSON:

{
"legacy_addon_id": "",
"title": "Composer Tutorial",
"description": "Shows how to include Composer packages in XenForo 2.1 addons",
"version_id": 2,
"version_string": "2.0.0",
"dev": "Simon Hampel",
"dev_url": "https://bitbucket.org/hampel/",
"faq_url": "",
"support_url": "https://bitbucket.org/hampel/xenforo-composer-tutorial/issues",
"extra_urls": {
"Git Repository": "https://bitbucket.org/hampel/xenforo-composer-tutorial",
"Twitter": "https://twitter.com/SimonHampel"
 },
"require": [],
"icon": "composer.png",
"composer_autoload": "vendor/composer"
}

Order of Class Loading

There is a way of overriding the order that the classes are loaded in the situation where multiple addons require different versions of Composer packages and you need to have your version loaded first. However, that is beyond the scope of this tutorial and so I won't go into details here.

Source Control

When using source control, you generally don't want to check in the vendor directory - that can be rebuilt either at build time if packaging an addon .zip file, or at deployment time if using automated deployment tools.

As such, my .gitignore file contains the following directives:

Code:

_data/
_releases/
vendor/

the _data directory contains the files necessary to install the addon in production. This will be packaged into the build .zip file but isn't necessary for development purposes (we need the _output directory instead).
the _releases directory is where our addon build .zip files are put - we don't want those in source control
the vendor directory is where our required Composer packages are installed. This can be rebuilt at any time based on the contents of the composer.lock file and so isn't needed in source control

Here is a list of the important files I check in to source control - in addition to any source code:

.gitignore as above - tells us what not to check in to source control
addon.json - our addon definition file
build.json - instructions on how to build our addon package for deployment - see more on this below
composer.json - our Composer file defining the packages we use
composer.lock - the Composer file that tells us which versions of the packages we are using (and have tested!)
_output - our development output, required to install the addon on our dev server

Packaging the Addon

We have one additional step required before we can package the addon. We need to give the addon build process some instructions on how to install our required Composer packages.

We can use the build.json file for this - which contains a series of directives which are executed during the build process.

When using Composer packages, my build.json file looks like this:

JSON:

{
"exec": [
"composer install --working-dir=_build/upload/src/addons/{addon_id}/ --no-dev --optimize-autoloader"
]
}

The {addon_id} code is automatically substituted by the build process - no need to use your actual addon ID, just use the above string verbatim including the curly braces.

The core build process starts by creating a copy of your addon files in the temporary _build directory. We can manipulate these copied files to get them ready for production use.

Our build.json file tells the build process to execute the composer install command using the location of our temporary addon files in the _build directory.

We add two important parameters at this point:

--no-dev instructs Composer to remove any development dependencies we may have included during our development. For example, if we use a unit testing framework such as PHPUnit - we don't need that in production, we would include it via a require-dev directive in our composer.json file. This command deletes all require-dev packages.
--optimize-autoloader converts PSR-0/4 autoloading to classmap files to get a faster autoloader, which is strongly recommended for production use

So now, when we swap back to our XenForo root and run the addon build command, it will optimise our autoloader code for us ready for production use.

Bash:

$ cd /srv/www/xenforo21
$ php cmd.php xf-addon:build-release ComposerTutorial

Performing add-on export.

Exporting data for Composer Tutorial to /srv/www/xenforo21/src/addons/ComposerTutorial/_data.
26/26 [============================] 100%
Written successfully.
Attempting to validate addon.json file...
JSON file validates successfully!

Building release ZIP.
Loading composer repositories with package information
Installing dependencies from lock file
Nothing to install or update
Generating optimized autoload files

Writing release ZIP to /srv/www/xenforo21/src/addons/ComposerTutorial/_releases.

Release written successfully.

You can see the output of the composer commands along with the build output.

That's all we need to do - our addon zip file has now been built with the specific versions of the Composer packages we have tested and has been optimised for production use.

Testing This Tutorial Addon

I have attached the built addon to this tutorial so you can install it, look at the code and experiment with other packages.

Once installed, go to the Admin > Tools > Test Composer Tutorial page and you'll see some output generated by Carbon.

You may also clone the git repository for the tutorial code - XenForo Composer Tutorial

Optional Setup.php requirement checks

If your build process works correctly, your vendor folder should be created and all dependencies installed and included in the release file ready for deployment.

However, if you are using source control and cloning your repository - you must remember to install the vendor dependencies by running composer install (or composer update if you're still developing) before you attempt to install the addon.

If you try and install an addon which has composer dependencies but your vendor folder does not exist, you can break your forum in a rather unpleasant way (requires manually updating the database to fix it!).

As a safety check, you might want to perform a simple verification that the vendor folder exists when attempting to install your addon - we can do this by adding a Setup.php file to the root of your addon (you may already have one).

PHP:

<?php namespace ComposerTutorial;

use XF\AddOn\AbstractSetup;
use XF\Db\Schema\Create;

class Setup extends AbstractSetup
{
public function install(array $stepParams = [])
{
// Nothing to do yet
 }

public function upgrade(array $stepParams = [])
{
// Nothing to do yet
 }

public function uninstall(array $stepParams = [])
{
// Nothing to do yet
 }

public function checkRequirements(&$errors = [], &$warnings = [])
{
$vendorDirectory = sprintf("%s/vendor", $this->addOn->getAddOnDirectory());
if (!file_exists($vendorDirectory))
{
$errors[] = "vendor folder does not exist - cannot proceed with addon install";
}
}
}

The magic is in the checkRequirements function which simply validates that the vendor directory exists and prevents the addon installation from continuing if it doesn't.

This last step isn't really required - but it can be a useful thing to include, just in case!

Unfortunately, the compose dependencies themselves aren't available to the checkRequirements function - composer hasn't been initiated at this point, so you can't check that any specific classes are available. You could check for specific files though.

Reactions: usimaweb, emonb and rotmos

Using Composer Packages in XenForo 2.1+ Addons Tutorial 2.1.1

More resources from Admin

Share this resource