Blog

Drupal's configuration system has at its heart a key concept: the site owns the configuration. What that means is that after a module has provided install configuration, it no longer has any control over it. You, the site admin, can change that config or even delete it. The module is no longer involved.

This kept the configuration system simple (up to a point); after all, the development of this system was one of the harder parts of the lengthy and difficult Drupal 8 cycle ("The ConfigImporter class was written with a get this done and working attitude" wrote @alexpott on a core issue, and I think the same could be said of the configuration system as a whole).

Since then, the config transformation API was added, and various modules have been created that take advantage of this, such as Configuration Split and Config Merge. These still work in the paradigm of the site owning the configuration. But what if we could turn the clock back, just partially, to when modules could own config too?

The (different) old days

If you were around in the days of Drupal 7, you may remember the various default hooks. Modules such as Views and Flag exposed a hook which allowed modules to define configuration items. (These were not configuration in the modern sense, because there wasn't that concept at the time, but referring to them as such makes things simpler.)

These were thus a code-defined instance of a type of thing that could also be defined in the UI by site admins. A module could supply a default view, and rely on it always existing. A new release of a module could come with enhancements to a that view, and they would exist on the site as soon as the module code was updated to the new version.

Because these hooks were PHP code, it was also possible for configuration to dynamically depend on other aspects of the site. You could do something such as define a default flag for every node type.

The idea

For a long time, I've wondered if something similar could be possible with Drupal's configuration system: put simply, defining configuration in code. Or, as I'm calling it, fixed config.

Many modules, such as Drupal Commerce and LocalGov Drupal rely on configuration items which are essential to their functionality (what I call machinery). Drupal's paradigm of configuration being owned by the site means that a site admin can break or outright delete key parts of the module's functionality: a commerce system missing its cart view, or a directory system missing its node types and vocabularies.

There's a case for admins being able to enhance and tweak a module's machinery, but that leads to the problem of how to reconcile a site's changes with changes that come in a new version of a module. There are modules that aim to help with this, but configuration entities are complex data structures, and without in-depth knowledge of that structure, this is a difficult and painstaking task. Some modules take care of this in update hooks each time they want to change their machinery, but that also requires complicated work each time.

The final push to get me to work on this was the use case of Entity Pager: the maintainer of the Flippy module suggested merging the two modules. But while Entity Pager provides the means for site admins to create any pager for any entity type, Flippy's functionality is to automatically provide a pager for each node type on the site. It seemed to me that the way to achieve this feature on top of Entity Pager was to have a way for the Flippy module to define an entity pager view automatically for each node type. The same way that Drupal 7 default hooks could.

Could this be done within the modern Drupal config system? The module would be in charge, not the site. If the module's code updated and changed the definition of the config, then the site would pick up on that.

Developing the module

The basic requirement is to add config entity definitions that are coming from code.

I could see two ways of doing this: intercept the entity storage handlers, or intercept the config transformation API. I opted for the latter. Partly because working at the entity storage level would require decorating every config entity type storage handler, which seemed heavy-handed, and because working within the config system has its advantages, and finally because one general principle I have found over the years is that complexity should generally be pushed down in a system. The config system is beneath the entity system, so working there would mean that this would be completely invisible to the entity system which would just see some additional entities.

So, I experimented with the config storage transformation API. Could I make it see, for example, a hardcoded node type? The answer was yes: it's simple to add an additional config entity in the STORAGE_TRANSFORM_IMPORT event, and the site sees it just as if it were some other piece of config.

Then the real development work started: making the config system read the fixed config, but also ignore it when necessary. I decided on this simple rule:

• Fixed config is not exported to config sync. Instead, it is always re-created from files on config import.

On Drupal 7, some (but not all) default hooks had a concept of overriding. The site admin could choose to edit a default object, such as a view, and deviate from the version in code. From that point on, the site owned that object, not the module.

I've decided, for now at least, not to support overriding. I think a better, and simpler, pattern is to allow selection of config: the module provides machinery, but also provides a setting to select it as being in use. For our example of the commerce cart view, there would be the fixed config view, and also a config setting where you select which view is used for the cart. This means that the default view remains under the control of the commerce module, but a site admin can duplicate it, change the duplicate, and then use that one instead. The site then owns the duplicate view, but the default view remains available and functional at all times.

The Alpha-1 API

Having got a proof-of-concept, the next step was to design an API too. Rather than use hooks or an event, I decided fixed config should look the same as install config: YAML files. The big advantage here being that while developing it, you can move YAML files from sync or install folders to become fixed config. And it provides a system that is already familiar to developers: fixed config is simply a YAML file in a module's config/fixed folder instead of config/install. (I made a small change and a new release of Config Devel so that its module config export feature works with this folder too.)

But static YAML files doesn't cover all the use cases that the Drupal 7 era hooks used to satisfy. So I added the concept of derivers. We already have plugin derivers in Drupal (and maybe I should have tried to think of a different name); but these are config deriver plugins: they create multiple config items from a single template.

So while a static fixed config file looks identical to an install config file, a derived fixed config file has these additions:

third_party_settings:
  fixed_config:
    deriver: deriver_plugin_id
    dependee_config_patterns:
      - node.type.*

What this means is that when config whose name matches node.type.* is created or updated, the fixed config deriver plugin deriver_plugin_id reacts, and maintains the fixed config based on the YAML file that holds these properties.

The node type config entity is the dependee config; the fixed config that gets defined in response to that is the dependent fixed config.

This is how the Flippy module could then work: it defines a single fixed config YAML file for a view, and a deriver plugin which alters these config values:

• Sets the value of the node type filter.
• Sets the path for the page display.

Eating my own dogfood

I find that you never fully realise what an API needs to do or how it needs to work and how to best make it usable until you actually try to use it, and that therefore it's best to do that early on.

I started making the Entity Pager view, and I was soon struck with several things about what I'd made:

• Code that sets a view's entity bundle filters to an incoming bundle entity should be reusable.
• Code that adds the incoming entity's ID as a suffix to the view's path should be reusable, but not necessarily by the same things as the first.
• As well as deriving a single view for each node type, it might be useful to have a single view which adds a new display for each node type.

That reusability feature could be solved by using PHP traits, and requiring developers to composite them into their plugin classes. Or it could be solved by making the plugins themselves smaller, and allowing the deriving process to specify more than one. A pipeline, basically.

And the many-to-one feature could be solved by letting code alter the fixed config YAML file values, rather than deriving from them: the same config values get processed by the PHP code for every config entity it listens for, rather than making a new copy each time.

To me, this suggests a new type of plugin, fixed config alterers, which can do the work in both of these scenarios.

So what I am pondering now would look like this:

• For derived fixed config:
  • The fixed config deriver plugin does the bare minimum to the config values from the YAML file.
  • The config values are passed through multiple alterer plugins.
• For static fixed config:
  • If config patterns are specified, the config values are passed through alterer plugins for each config matching the pattern.
  • A better name is needed as it would no longer be completely static!

Am I over-engineering, I wonder? Also, the pipeline of alterers seems very much like the Migrate API's pipeline of processors, but I don't see how to reuse those as they operate on different sorts of values: entire config entity values for my case, and single content entity field values for Migrate.

I'd be interested in hearing ideas and use cases for the fixed config module. It would help me with the process of refining my initial ideas for the API.

I've made an alpha release, try it out and let me know what you think, either in the issue queue or on mastodon.

Do you have potential uses for fixed configuration? I'd love to work on this module with some real-world use cases, and I'm available for hire - contact me!

It's funny how big ideas start.

I am doing some maintenance work on the Entity Pager module, which has been fixing bugs, improving the tests, and so on. And because Computed Field is also a module I maintain, and because I have at the back of my mind the idea of finding more use cases for it, I had the thought that I could add support for Computed Fields to Entity Pager.

Specifically, this would mean that Entity Pager would allow you to add computed fields to your entity type, which would be computed entity reference fields to the previous and next entities in the pager. As well as allowing you to output the previous and next links with more flexibility, and within the rendered entity rather than in a block, it would open up having these links in JSON:API (though there's a bug to fix still).

So, then, quite a good use case!

It does, however, require a bit of re-plumbing inside the EntityPager class. Currently, EntityPager, expects to be instantiated within the theming for an executed view. Our computed field needs a new API which it would call with the basic data (the view ID, display ID, and current entity), and that would take care of executing the view, extracting the data from the result, and returning it.

I suppose I could make a whole new pathway, but a lot of the code that would need is in EntityPager so it makes more sense to me to change that to allow both cases. This means adding a new way of constructing it from the factory service, and then executing the view if necessary.

So then we'd have an API for getting the previous and next entities. And that's where the big idea suggests itself: what if we used this API for everything?

Currently, the rendered entity pager is a specially-themed view. We define a custom Views style plugin, and that uses our theme template 'entity_pager' for its theming. We use the Views block system to show the pager. By the time our code is involved, the view has already been executed, and all of our code is taking place within the Views theming. This makes it tricky to do things like hiding the pager completely.

But... once we have an API, we could totally invert this. We could define a custom render element which outputs the pager. This would take the view ID and display ID as properties, and the current entity if you have it (and continue to detect it from the current route if you don't). Like this:

$build['pager] = [
  '#type' => 'entity_pager',
  '#view_id' => 'my_pager_view',
  '#view_display_id' => 'my_display',
];

This render element would then be in charge of executing the view, and getting the data it needs from the view's result. You'd still store settings for the pager on the view's style plugin, but we'd no longer rely on the theming of that — the view would just be used as a data source. The render element would have similar theming to the Views style — it could pretty much use the same Twig template. The block we provide would change to being a completely custom block plugin, which would output the pager element.

To me, this seems like a cleaner structure. Our pager is a separate render element, and our code no longer runs inside Views rendering, which feels a little bit convoluted and fragile.

If you use Entity Pager, what do you think? Would this make your use of Entity Pager simpler, more complex, or not affect you at all? It would be a big change to the module, so I'd love to hear opinions on the issue for this, as I'm still undecided about it.

Do you need help with updating a contrib module, refactoring it, or expanding its capabilities? I'm available for hire - contact me!

It's often useful to let the machines do the work, and output something that's dynamically computed on an entity. By that I mean something that can't be hardcoded as a fixed value for all entities of a particular type, but that varies for each entity, in a way that allows it to be generated in code rather than laboriously entered into each entity form by humans.

For example, you might want a backlink for an entity reference, or a link to a view that has an argument for the entity's ID, or something that depends on field values on the entity.

There are several ways in Drupal of putting something dynamic on the entity's display output. You can of course add something to the build array yourself, in either the entity's view handler or hook_entity_view(). The extra fields system lets you then declare your additional build array item with hook_entity_extra_field_info() which allows it to be rearranged among normal fields in the field admin UI.

This is okay, but the extra fields system is Drupal 5-era stuff. Your piece of build array is just that, some render stuff; it can't participate in any data structures and nothing else will recognise it and work with it.

A better approach is a computed field. This involves a little more boilerplate code than the extra fields technique, but there are several benefits.

The first is that you are defining a field value, not a render array, and you get access to all the field formatters that apply to your type of data. So for example, if your computed data is a URL, you get all of the link field formatters at your disposal, in core and contrib.

The second is that anything that works with fields will be aware of your computed field. So you can add it to a view as a field (though not a sort or filter of course, since it has nothing in the database). You can add it to a SearchAPI index (and there, you actually can filter on it, because SearchAPI will index the computed value into its backend).

The code

Here's what you need to do. You need two things:

1. A FieldItemList class.
2. A declaration of your field in an entity class or field info hook.

Unlike declaring code fields, you don't need to declare a field storage: that's because a computed field doesn't store anything!

1. The FieldItemList class

Create a subclass of \Drupal\Core\Field\FieldItemList that uses \Drupal\Core\TypedData\ComputedItemListTrait. In this class, all you need to do is implement computeValue() to return your data.

<?php

// The namespace doesn't matter, but I like to put it under \Field.
namespace Drupal\my_module\Field;

use Drupal\Core\Field\FieldItemList;
use Drupal\Core\TypedData\ComputedItemListTrait;

/**
 * Field item list class for my computed field.
 */
class MyFieldFieldItemList extends FieldItemList {

  use ComputedItemListTrait;

  /**
   * {@inheritdoc}
   */
  protected function computeValue() {
    // You have access to the complete entity, so you can use other field
    // values.
    $entity = $this->parent->getValue();

    // Create a field item for your data. You can create more than one for a
    // multi-valued field.
    $this->list[] = $this->createItem(0, [
      'value' => 'cake',
    ]);
  }
}
?>

For most field types, the key to use in the item array is 'value', but some more specialised fields use something else. You can find this by looking in the field item class for the field type. For example, in \Drupal\link\Plugin\Field\FieldType\LinkItem::propertyDefinitions() you can see that for a link field, you need these array keys:

    $this->list[] = $this->createItem(0, [
      'title' => $this->t('My link title'),
      'uri' => $some_url_that_we-compute->toUriString(),
    ]);

2. Define the field

For the field definition, there are two things to consider:

• Is it on an entity type you control, or somebody else's?
• Do you want a base field or a bundle field?

If it's your own entity type, you define the field in the entity class, in either the baseFieldDefinitions() or bundleFieldDefinitions() method. If it's an existing entity type, you need to use hook_entity_base_field_info() or hook_entity_bundle_field_info().

In all cases, the code is broadly similar. For a base field, it looks like this:

/**
 * Implements hook_entity_base_field_info().
 */
#[Hook('entity_base_field_info')]
function entityBaseFieldInfo(EntityTypeInterface $entity_type) {
  if ($entity_type->id() == 'node') {
    $fields = [];
    $fields['my_computed_field'] = BaseFieldDefinition::create('link')
      ->setLabel($this->t('My computed field'))
      ->setDescription($this->t('My field is amazing.'))
      // This declares it as a computed field.
      ->setComputed(TRUE)
      // This is the class you created earlier, which provides the values.
      ->setClass(MyFieldFieldItemList::class)
      // Optional default view display options, which can be overriden in the admin UI.
      ->setDisplayOptions('view', [
        'label' => 'above',
        'type' => 'link',
        'weight' => '0',
      ]);

    return $fields;
  }
}

For a bundle field, you need the Drupal\entity\BundleFieldDefinition class from Entity module, and a few extra things need to be explicitly set on the definition because the field system doesn't handle them for you:

    $fields['my_computed_field'] = BundleFieldDefinition::create('link')
      ->setName('my_computed_field')
      ->setTargetEntityTypeId($entity_type_id)
      ->setTargetBundle($bundle)
      // Rest of the definition as above.

See my earlier blog post on bundle fields for more about their uses and their quirks.

The contrib module

If you want to do it all with less boilerplate, or your computed field is something that's reusable across different entity types, consider the Computed Field module as an alternative to the code examples above. Instead of a Field class, the computational code does in a plugin, which the module then makes available in an admin UI where you can create and edit computed fields alongside the usual stored fields.

And if your computed field is purely a render array rather than data, the Computed Field module also provides a computed_render_array field type for that, with an accompanying field formatter.

Do you need help with data structures, and their integration with Views or SearchAPI? I'm available for hire - contact me!

It's true, no April fools. You can make your Browser tests run much quicker. How? By deleting them!

You will of course need to add a corresponding Kernel test - and that's the trick. Kernel tests run much faster than Browser tests.

But Browser tests make requests to the test site using an internal web browser, I hear you say, whereas Kernel tests make API calls directly. Kernel tests have their uses for testing APIs, but Browser tests are needed to test actual HTML output.

Aha! Kernel tests can now make HTTP requests.

This is subject to a number of caveats and limitations: there is no session, and forms can't be submitted. And functionality such as a current user, blocks on the page, and page caching will need additional setup.

And more generally, with Kernel tests, modules are enabled but not installed: you need to handle things like entity schemas, database tables, and install config yourself in the test. The benefit though is that you only set up the parts of the module that you need for your test.

So not all Browser tests are suitable for conversion. But a lot of them are. We're already working on converting tests in core, and as this feature has been backported to Drupal core 11.x, contrib modules can make use of it too.

The benefits to conversion are tests that run faster, so less time developing and less time waiting for CI pipelines to run, and a lower energy footprint and lower costs for drupal.org. And they're easier to debug too.

And if you haven't yet written any tests for your module, now is an excellent time to start!

Do you need help with writing PHPUnit tests, or getting started with test-driven development? I'm available for hire - contact me!