drupal commerce

The ripple effect

Hello! I'm back. I've not made any blog posts in over a year and a half due to the site where my blog was before, drupaler.co.uk, closing down. And while it took me some time to get round to writing a Migrate script to import my posts from the old site's database, it was actually getting round to setting up this new domain that took the longest.

So what have I been doing all this time? Especially as I still don't have a single Drupal 7 site out there to my name? Well, these days I work on a humongous web application which has kept me busy for the last 18 months; it's a large Drupal site (we hit a million of one of its several custom entity types recently), but to the general public it's just a login page. I may talk more about the development challenges in future posts.

Prior to that, I was building what would have been one of the earliest big Drupal Commerce sites to launch... except that very shortly before launch in October 2012, the whole project got canned.

That was obviously rather demoralizing, as it had been a year in the building. However, it's interesting to look back and see just how much a large-scale project can contribute back to the Drupal community. The following is a list of the contrib modules that were created as part of the development of the project. Many of them are pretty simple; many of them don't get that much use. But they get some, and so it's interesting to see how much code a project can share if development is steered towards reusability, and also how a single project, even one that never sees the light of day, can have effects that ripple out and benefit many others.

Reference field option limit

This was built to improve the UI for selecting terms from large taxonomies, where one vocabulary groups another. For example, you could have a huge vocabulary of cities, and a smaller one of cities. Each city term has a term reference to the country it belongs to (the glee of the early days of Drupal 7: all the fields on all the things!), and the entity that you want to tag (in my case a product) has term reference fields to both the country and the city. The way this module works is that the city terms shown in one widget are filtered each time you change the selected country term in the other field's widget. So you select 'France' and the city field widget updates with AJAX to show only cities in France.

It sounds simple to explain (I hope!), but involves a fair amount of work under the hood in the form alteration to pull it off. It's one of my most popular modules, and has received a fair number of patches from other users.

Devel Contrib and Field Tools

These two are developer modules. Devel Contrib arose from my constantly needing to dsm() the data from info hooks such as hook_views_data() and hook_entity_property_info(). When the thing you're developing keeps going wrong, one of first things to do is poke around to check you've properly declared everything to the APIs you're using. I started out just having this as a couple of menu items declared in a custom module, but pretty soon I added more, and I wanted it on other development sites, so the obvious thing to do was clean it up and release it.

Field Tools was something I wrote because the Commerce site in question had dozens of different product types and corresponding node types, with lots of common fields, and I really didn't want to spend hours clicking through the field admin UI to set them up. This may seem like a case of condiment-passing, but I'm sure it's saved me hours of tedium: create the fields once, and then quickly clone them to any other entity types and bundles.

Field Instance Cardinality

A very small tweak module, this lets you override a particular instance of a multi-valued field and set it to be single-valued only. It's a hook_form_alter() hack made reusable by adding some field admin settings, and a good example of how site customization can often be done as modules rather than alter hacks.

Field Value Link Formatter

Very simple module: your taxonomy term fields (or other entity references) should link to a view with an entity ID argument. I guess the more traditional way of doing this is with a lot of path aliases for your taxonomy terms. I think we may also have used this to create Profile-style lists of related entities.

Commerce Shipping Weight Tariff

This was created to deal with the complex business rules we had for shipping costs. Written very near to our planned launch date, it's an example of what I call the 'skimp on the admin UI' contrib module: hardcode the settings, but do so in a way that's cleanly separated from the rest of the module, so that later on an admin UI can be added, perhaps contributed as a patch. Though I think that's yet to materialize for this one.

Views Grouped Table

This was created to help keep track of how the product entities and product display nodes all related to one another.

Views Dependent Filters

Allows the presence of exposed filters on a view to be controlled by values in another exposed filter. This was intended to handle filtering Views of products, though we later switched to using Views with Solr.

Taxonomy add previous

A little bit of UX sugar for when you're adding lots of taxonomy terms that are very similar. In our case, this was terms representing sizes. On creating a new term, this takes you straight back to the form for adding another term, and prefills the field values from the one you just added.

Flag Expire

Allows flags to have either an expiry date, or an expiry period. This was going to be used to feature products, or mark them as new, or discounted. (For new product, you could just automatically mark all products that were created in the last x days, say. But as I recall, the client didn't want ALL new products marked, just selected ones. The way clients do.)

As well as new contrib modules, the project resulted in work on existing contrib modules, in particular Flag, Data, Views Hacks, and Commerce Delivery.

Corralling permissions into a grid

I've just released Permissions Grid. It does what the name suggests: it presents related permissions in a grid, rather than the usual long list.

How are permissions structured into a grid? Well, only the ones that form natural groups are included: every set of permissions of the form 'create foo, edit foo, delete foo, create bar, edit bar, delete bar' is turned into a matrix of checkboxes with the verbs 'create, edit, delete' along the top, and the objects 'foo, bar' along the top. When modules such as node, taxonomy, and commerce define related permissions for nodes, vocabularies, and products respectively, that gives you something like this:

This gives an easy to grasp overview of what a role can do with different objects on the site: which node types can this role create? which can they edit, or delete? which product types can they edit? which vocabularies can they create terms in?

If this sounds and looks vaguely familiar, that's probably because this module has an ancestor: my Drupal 6 module node permissions grid module, which I wrote back when a site's content types started to become too numerous to easily make sense of. That operated only on node types, and like a great many contrib modules porting to Drupal 7, it's had to 'drop the node' and generalize. But in fact nothing restricts Permissions Grid to entities: all it cares about is permissions.

Structured permissions are declared to the module in an info hook, and each module may declare multiple sets of permissions. This allows for the fact that some modules add further vocabulary-related permissions which do not have the same pattern, and that commerce has entity permissions in both singular and plural form.

Are there any groups of permissions I've missed, whether in core or contrib? Post a feature request, or better still, take a look at the hook implementations already there and file a patch.

On rules versus hooks, or, abstraction shock

I need to add a bit of business logic to my Commerce site: a boolean field on product nodes marks that the corresponding products can't be delivered outside the UK.

And I know the way to do this in Commerce is to create a rule: react to the cart completing checkout, iterate over line items, check the corresponding products, and block the completion if the field in question is set.

Rules is great: with Rules, site builders can change site functionality and cause it to react to events. When non-techy people ask if my job involves designing websites, to put them right I say, 'I make websites go "bing!"'; and now, site builders can make them go 'bing!' too.

But I have a confession: I'm reluctant about using Rules. It's partly that I find the UI confusing, and it feels time-consuming to test them, but deeper than that I think it's just that I feel too far removed from the actual thing I'm trying to make.

And that makes me wonder: am I becoming a Drupal dinosaur?

Because I can imagine when Views first came along, developers who were used to writing their own query and formatting the result themselves, looking at the Views UI and thinking 'I don't feel in control of my lists of stuff any more'.

Or before CCK, developers wrote exactly the form elements they needed in the node form and saved it themselves in the database. I still sometimes speak to non-Drupal developers who want to be able to dump data into the node table directly (or pull it out) and when I tell them they can't, because the data that actually makes a node is spread out over the node table, the node_revision table, and then a multitude of field tables. And their feeling of disconnection at not being able to get their hands on 'the node' as a solid lump of database stuff must surely be akin to what I feel with Rules. And I'm going to call this feeling 'abstraction shock'.

I want to write a hook. I want to write the code for it, for it to feel like a solid thing. I know that my rule can (indeed, should) be exported to code, but I want code that I can read and see exactly what it says, rather than code that Rules will consume and understand. And most of all, I want to be able to put in debug statements to understand what I'm getting as I write it, and after I've written it when it's going wrong or when the site functionality has to change.

If that makes me a dinosaur, save me a seat next to the brontosaurus.

Dynamically changing Views table joins

I've recently had cause to make Views make joins to tables in peculiar ways. Here's some notes on the peculiar things I did with the views_join class to accomplish that.

First of all I'll briefly recap how we define a table to Views. Each item in the $data array returned to hook_view_data() represents all the information about a table. Each key in the array is a field on that table (well, or pseudofield), except for the 'table' key which has the basic data about our table, like this:

$data['my_table'] = array(
  // This defines how the table joins back to different bases.
  'table' => array(
    // How to join back to the base table 'crm_party'.
    'crm_party' => array(
      'left_field' => 'pid',
      'field' => 'pid',
    ),
  ),
);
// Now we can add field definitions on this table.

That's the simplest case. It says, 'to join back to {crm_party}, join on the column 'pid' on both tables'. (Note I will say 'column' when I am speaking of the database, and 'field' for Views, though that can mean both a field that you add to the view, and a field on the table that provides filters, sorts, or arguments.)

So adding a field on this table will cause Views to add this join clause to the query:

... JOIN my_table ON crm_party.pid = my_table.pid

We can easily join where the columns have different names, by giving different values for 'field' and 'left_field' in the table definition.

If the join requires conditions, that's where the 'extra' clause comes in, like this:

// How to join back to the base table 'crm_party'.
'crm_party' => array(
  'left_field' => 'pid',
  'field' => 'pid',
  'extra' => 'foo = 42',
),

This now gives us:

... JOIN my_table ON crm_party.pid = my_table.pid AND foo = 42

The 'extra' can also take an array, in which case each item is an array containing field, operator, and value. (If it seems a bit like the Database API, but not quite, that's because all this was introduced in Views 2 on Drupal 6).

// How to join back to the base table 'crm_party'.
'crm_party' => array(
  'left_field' => 'pid',
  'field' => 'pid',
  'extra' => array(
    // The 'extra' array is numeric, hence has no keys. This always looks odd to me!
    array(
      'field' => 'foo',
      'value' => 42,
      'numeric' => TRUE,
    ),
  ),
),

So far, this is all covered in the Advanced Help documentation contained within Views. But for our relationship handler from CRM Parties to attached entities, we needed a condition on the join depending on values selected in the UI. So the 'extra', defined in hook_views_alter(), won't do, as it's not changeable. Or is it?

When a relationship handler is adding itself to the query, the query hasn't been fully built yet. Rather, Views has a views_plugin_query_default object which will eventually be used to make a DatabaseAPI SelectQuery. This means we can actually reach into the table queue and change the definition for any table to the left of us, like this:

// Our relationship handler's query method:
function query() {
  // Call our parent query method to set up all our tables and joins.
  parent::query();

  if ($this->options['main']) {
    // This is a little weird.
    // We don't add an 'extra' (ie a further join condition) on our
    // relationship join, but rather on the join that got us here from
    // the {crm_party} table.
    // This means reaching into the query object's table queue and fiddling
    // with the join object.
    // Setting a join handler for the join definition is not useful, as that
    // would have no knowledge of the user option set in this relationship
    // handler.
    // @todo: It might however be cleaner to set one anyway and give it
    // a method to add the extra rather than hack the object directly...
    $table = $this->table;
    $base_join = $this->query->table_queue[$table]['join'];
    $base_join->extra = array(array('field' => 'main', 'value' => TRUE));
  }

When I wrote those comments in the code last week, I'd found that using a custom join handler isn't useful, because that has no knowledge of the relationship handler's data. However, this week I found myself working on a different case where I did need to find a way to do just that.

This week's problem was how to filter out Drupal Commerce products that are in the current cart, or more generally in any order (and the current cart's order ID can be supplied with a default argument plugin).

It seems a reasonable enough thing to ask of Views, but it's actually pretty complex, as what's in a cart or order is not products but line items, each of which refers to a product with a reference field.

After several failed attempts, I managed to write a query that produces the correct result, but it requires joining to a subquery which itself has the order ID within it (see the issue for gory details).

The first hurdle with this is easy to overcome: there's nothing wrong about telling Views about a table that doesn't exist. This is often done with aliased tables, but in fact it can be totally fictional provided we also provide our own join handler which understands what to do to the query. That can be anything, as long as the SELECT fields we also add make sense. Hence it's fine to do this in hook_views_data():

// Fake table for the 'product is in order' argument, made from a subquery.
$data['commerce_product_commerce_line_item'] = array(
  'table' => array(
    'group' => 'Commerce Product',
    'join' => array(
      // Join to the commerce_product base.
      'commerce_product' => array(
        'left_field' => 'entity_id',
        'field' => 'line_item_id',
        'handler' => 'views_join_commerce_product_line_item',
      ),
    ),
  ),
);

Our custom join class now has to add the subquery to the view, but it also needs the argument value to do this.

The way I worked around this was to override the ensure_my_table() method in the argument handler. Normally, this calls $this->query->ensure_table() which then creates the join, but ensure_table() can take a join parameter to work with. The overridden version of ensure_my_table() creates the join object, and sets the argument value on it:

function ensure_my_table() {
  // Pre-empt views_plugin_query_default::ensure_table() by setting our join up now.
  // Argh, hack this in for now. This may mean relationships using this break?
  $relationship = 'commerce_product';

  // Get a join object for our table.
  // This is of class views_join_commerce_product_line_item, which takes
  // care of joining to a subquery rather than a table.
  $join = $this->query->get_join_data($this->table, $this->query->relationships[$relationship]['base']);

  // We add the argument value to the join handler as it needs to use it
  // within its subquery.
  $join->argument = $this->argument;

This means that in the build_join() method for our custom join handler, views_join_commerce_product_line_item, we can rely on the argument value that the views has received:

function build_join($select_query, $table, $view_query) {
  // (snip...) build a SelectQuery object for the subquery
  // Set the condition based on the argument value.
  $subquery->condition('cli.order_id', $this->argument);
  // (snip...)
  // Add the join on the subquery.
  $select_query->addJoin($this->type, $subquery, $table['alias'], $condition);

The views_join class's build_join() method is where the Views system of building a query is translated into a DatabaseAPI SelectQuery object. Here we build up our own query (and we don't need Views-safe aliases, as it's a completely internal, non-correlated subquery), and pass it in as a join which uses it as a subquery.

It remains only for the argument handler's query() method to add the conditions for its field, using the alias and field names we gave for the subquery.

In conclusion, the data structure Views understands may appear to be a fixed, declared thing, but with a little bit of tweaking the way tables are joined in a Views query can be affected by both site configuration and user input.

Subscribe to RSS - drupal commerce