Corralling permissions into a grid

I've just released Permissions Grid. It does what the name suggests: it presents related permissions in a grid, rather than the usual long list.

How are permissions structured into a grid? Well, only the ones that form natural groups are included: every set of permissions of the form 'create foo, edit foo, delete foo, create bar, edit bar, delete bar' is turned into a matrix of checkboxes with the verbs 'create, edit, delete' along the top, and the objects 'foo, bar' along the top. When modules such as node, taxonomy, and commerce define related permissions for nodes, vocabularies, and products respectively, that gives you something like this:

This gives an easy to grasp overview of what a role can do with different objects on the site: which node types can this role create? which can they edit, or delete? which product types can they edit? which vocabularies can they create terms in?

If this sounds and looks vaguely familiar, that's probably because this module has an ancestor: my Drupal 6 module node permissions grid module, which I wrote back when a site's content types started to become too numerous to easily make sense of. That operated only on node types, and like a great many contrib modules porting to Drupal 7, it's had to 'drop the node' and generalize. But in fact nothing restricts Permissions Grid to entities: all it cares about is permissions.

Structured permissions are declared to the module in an info hook, and each module may declare multiple sets of permissions. This allows for the fact that some modules add further vocabulary-related permissions which do not have the same pattern, and that commerce has entity permissions in both singular and plural form.

Are there any groups of permissions I've missed, whether in core or contrib? Post a feature request, or better still, take a look at the hook implementations already there and file a patch.

Git tricks: repatching for an issue branch

My workflow for making patches is to use a feature branch for a single issue. Whether you're a contributor or a maintainer it lets you advance the fixing of the problem in small increments, and safely experiment knowing you can roll back.

But where it goes wrong is when your patch is superseded by a newer one in the issue queue, and you want to work on it some more. How do you update your branch for the ongoing work? As ever, with git there's a way.

Let's start with the basics first: you're making a feature branch to work on an issue. I tend to follow the naming pattern '123456-fix-all-the-bugs', but for this example I'll call it 'issue'.

// Make a new branch and switch to it.
$ git co -b issue
// Make lots of commits.
// Ready to make a patch:
$ git diff > 123456.project.issue.patch

(Note that if you can make your patch to show all your commits one by one, which can sometimes aid in making it clear what you're changing, but that's for another day.)

You've now got a patch which you're uploading to the issue queue, and your tree looks something like this:

* [issue] Last commit, ready to roll a patch!
* Fixed the foobar.
* Added a bizbax.
* [master]

Now someone else comes along to the issue queue, reviews your patch, and posts a new patch of their own. You in turn look at patch 2, and while it's an improvement, you think it needs still more work.

The problem is how to apply the patch to your repository. It won't apply to the tip of the issue branch, and if you checkout master, you can't get back to your issue branch. You can of course just discard your original issue branch, and create a branch issue2 for patch 2.

Or you can do this:

// Start on the issue branch.
// Stash any work in progress!
$ git stash
// Checkout just the *files* of master, while keeping the HEAD pointer on the
// issue branch.
$ git checkout master -- .
// This puts the files from master into the working tree, but keeps the index
// on the issue branch. In simpler terms, the reverse of patch 1 will appear
// staged (as git believes that your files *ought* to look like patch 1, but
// actually look like master).
// We want the index clean, so unstage everything:
$ git reset HEAD .
// Now apply the new patch.
$ patch -p1 < patch-2.patch
// Now commit this as patch 2.
// Remember to stash pop when you're done!

Because the working tree files (that is, the actual files on your system) look like the master branch, the patch applies cleanly. But because git still believes its on the tip of the issue branch, the commit you make goes on the tip of that branch, and the diff it records is effectively the interdiff between your patch-1 and the other contributor's patch-2. Your tree looks like this:

* [issue] Applied patch 2 from Ada Lovelace.
* Last commit, ready to roll a patch!
* Fixed the foobar.
* Added a bizbax.
* [master]

Result: you can now do more work on this branch, and make more commits, and when you're ready, diff against master to make patch-3, ready to upload to the issue queue.

Moving a git local branch from one local to another

You're maybe at one of the many Drupal Co-worker Friday events that are taking place around the world today. You've packed up your laptop and your lunchbox, and you're looking forward to a day out of the house with some human contact.

But yesterday you were halfway through a big piece of work on your project. And you were using a git local branch, of course. Why? Because it keeps your work isolated off the main development branch, allowing other work to continue independently. And because while your commits are only local you're free to reorder them, edit the log messages, fixup mistakes as if they never happened, and so on. In fact, if you so choose, merging your local branch in with the --squash option makes it look like you made all of the work in a single, perfect commit. Wow!

But that branch is on your desktop machine, and today you're on the laptop. How to get it over from one to the other? You don't want to push it to the remote, because then that means you can't rework it, and it doesn't really belong there anyway. You could make one big patch of your branch against the development branch, but then on your laptop you won't have the dozen or so commits you made yesterday (and you work with small, simple commits, because it makes it easier to roll back should you need to, and to see what's been changed). Furthermore, you've a bunch of changes that aren't yet committed because they're work in progress. You need those too, but not mixed in with what's already committed and reasonably stable.

The solution is the git format-patch command. At first try this is a rather weird one, which fills your folder up with cryptic mailbox files (which as far as I can tell are nigh on useless in the modern world of webmail). But it has an option which turns it into something very useful indeed: --stout. So much so that I've added it to my git global config thus:

  fp  = format-patch --stdout

So standing in your repository and doing

  git fp the-dev-branch > my-local-branch.patch

will give you one single file that comprises multiple patches, one for each commit. Copy that to your laptop by your favourite means, and over there do:

  git co -b my-local-branch
  git am  my-local-branch.patch

And all your commits from your desktop machine are reproduced on your laptop.

But what about your work in progress? I was coming to that. Before you begin, make one commit of all of them, perhaps with a log message to remind you it's the work in progress.

Then after you've done 'git am', all we have to do is kill off that final tip commit, while keeping the changes it contains in the filesystem. The command for that is git reset with the --mixed option:

  git reset --mixed HEAD^

All your work in progress changes are now 'unstaged changes', and your laptop's copy of the repository is in exactly the same stage as your desktop machine.

What about going back the other way? Well I'll figure that one out tonight, I expect ;)

PS. The return trip is easily done thus:

When you create the branch on the new machine, but before you apply the patch, add a tag: git tag mytag. Then at the end of the day, do the original process in reverse but take your diff from the tab: git fp mytag > homeward.patch

And on your home machine, remember to kill the 'work in progress' commit before you apply the patch, with 'git reset --hard HEAD^'. That's a hard reset this time, since the changes in that commit are in the new commits you're bringing back.

Hope your Drupal Coworker Friday was as productive as mine!

A concept for limiting taxonomy terms by common fields

There are several modules that provide taxonomy term widgets that are more efficient at drilling down into large and complex vocabularies of terms, but I've not yet found something that just limits terms in some way.

The use case is somewhat like this: I have products that are classified by sport and by team. And teams can be rugby teams or football teams, and so on. Having one vocabulary per type of team feels somewhat weak, but putting them all in one vocabulary means the user has to wade through a lot of irrelevant terms.

So I've been thinking of ways to limit the terms in the field widget. I don't think this is something I'm going to develop (for reasons that will become clear), but it's just a wacky idea[*] I'm putting out there.

I've discarded my initial thought to use a hierarchy of terms, because that would result in terms that are essentially empty: the group of football team terms is not correctly a team term. It would also mean needing to prevent its selection in the widget, and its appearance in filter forms.

What I've landed on as a concept instead seems wacky but I think makes a lot of sense: now we can have fields on anything, we can match them. What I mean by this is that we compare the value in the 'sport' field on the product, and the value in the identical sport field applied to the terms. (Yes, taxonomy terms on taxonomy terms. It was bound to happen eventually, right?)

In terms of IA, this is actually quite simple and logical: add a term reference field to the team vocabulary, and add the sport field, which is already on products, to the team terms (which the client will be doing; I have no idea what teams belong to what sport).

In terms of the work the UI has to do, it's not that complex either once it's laid out. It goes like this: when the value in the sport field changes, make an ajax call on the team field. In the callback, load all the terms. For each term, look at its team value. Does that match the current value of the sport field in the product form? If so, let the term into the new options array. If not, discard it.

It's simple, but I've hit a problem: we have two really good JavaScript systems in Drupal 7 that work very well individually, but as far as I can tell live in completely different universes. The states system is great for telling one form element to show, hide, or change its value depending on the value of another element. The ajax system is great for telling one form element to react to a change in itself by updating something via ajax. What I don't see how to do is get the term reference widget element to watch for changes in other elements (like the states system does), and then update itself with ajax (using the ajax system ideally).

So that's where I've hit the buffers: my JavaScript skills are pretty minimal, and so that's where I leave this latest wacky idea for now. I've got proof-of-concept code of a term reference widget updating itself via ajax based on form values, but the missing piece is getting it to react to changes in another field, whose form elements can't be altered in PHP (because the term reference widget is being changed in hook_field_widget_form_alter(), which only sees that form element). I'd be grateful for any hints; I actually think this model is not as wacky as it first seems and could have a lot of applications.

[*] Back at DrupalCon London I chanced upon a conversation webchick and stella were having about Coder review automation on Drupal.org, and prefixed a suggestion with, 'I've got a wacky idea...', to which webchick said, 'When are your ideas not wacky?' I don't know whether it was just an off the cuff quip, or whether I really do come up with a lot of crazy stuff. I guess I can live with that reputation ;)

Dynamically changing Views table joins

I've recently had cause to make Views make joins to tables in peculiar ways. Here's some notes on the peculiar things I did with the views_join class to accomplish that.

First of all I'll briefly recap how we define a table to Views. Each item in the $data array returned to hook_view_data() represents all the information about a table. Each key in the array is a field on that table (well, or pseudofield), except for the 'table' key which has the basic data about our table, like this:

$data['my_table'] = array(
  // This defines how the table joins back to different bases.
  'table' => array(
    // How to join back to the base table 'crm_party'.
    'crm_party' => array(
      'left_field' => 'pid',
      'field' => 'pid',
// Now we can add field definitions on this table.

That's the simplest case. It says, 'to join back to {crm_party}, join on the column 'pid' on both tables'. (Note I will say 'column' when I am speaking of the database, and 'field' for Views, though that can mean both a field that you add to the view, and a field on the table that provides filters, sorts, or arguments.)

So adding a field on this table will cause Views to add this join clause to the query:

... JOIN my_table ON crm_party.pid = my_table.pid

We can easily join where the columns have different names, by giving different values for 'field' and 'left_field' in the table definition.

If the join requires conditions, that's where the 'extra' clause comes in, like this:

// How to join back to the base table 'crm_party'.
'crm_party' => array(
  'left_field' => 'pid',
  'field' => 'pid',
  'extra' => 'foo = 42',

This now gives us:

... JOIN my_table ON crm_party.pid = my_table.pid AND foo = 42

The 'extra' can also take an array, in which case each item is an array containing field, operator, and value. (If it seems a bit like the Database API, but not quite, that's because all this was introduced in Views 2 on Drupal 6).

// How to join back to the base table 'crm_party'.
'crm_party' => array(
  'left_field' => 'pid',
  'field' => 'pid',
  'extra' => array(
    // The 'extra' array is numeric, hence has no keys. This always looks odd to me!
      'field' => 'foo',
      'value' => 42,
      'numeric' => TRUE,

So far, this is all covered in the Advanced Help documentation contained within Views. But for our relationship handler from CRM Parties to attached entities, we needed a condition on the join depending on values selected in the UI. So the 'extra', defined in hook_views_alter(), won't do, as it's not changeable. Or is it?

When a relationship handler is adding itself to the query, the query hasn't been fully built yet. Rather, Views has a views_plugin_query_default object which will eventually be used to make a DatabaseAPI SelectQuery. This means we can actually reach into the table queue and change the definition for any table to the left of us, like this:

// Our relationship handler's query method:
function query() {
  // Call our parent query method to set up all our tables and joins.

  if ($this->options['main']) {
    // This is a little weird.
    // We don't add an 'extra' (ie a further join condition) on our
    // relationship join, but rather on the join that got us here from
    // the {crm_party} table.
    // This means reaching into the query object's table queue and fiddling
    // with the join object.
    // Setting a join handler for the join definition is not useful, as that
    // would have no knowledge of the user option set in this relationship
    // handler.
    // @todo: It might however be cleaner to set one anyway and give it
    // a method to add the extra rather than hack the object directly...
    $table = $this->table;
    $base_join = $this->query->table_queue[$table]['join'];
    $base_join->extra = array(array('field' => 'main', 'value' => TRUE));

When I wrote those comments in the code last week, I'd found that using a custom join handler isn't useful, because that has no knowledge of the relationship handler's data. However, this week I found myself working on a different case where I did need to find a way to do just that.

This week's problem was how to filter out Drupal Commerce products that are in the current cart, or more generally in any order (and the current cart's order ID can be supplied with a default argument plugin).

It seems a reasonable enough thing to ask of Views, but it's actually pretty complex, as what's in a cart or order is not products but line items, each of which refers to a product with a reference field.

After several failed attempts, I managed to write a query that produces the correct result, but it requires joining to a subquery which itself has the order ID within it (see the issue for gory details).

The first hurdle with this is easy to overcome: there's nothing wrong about telling Views about a table that doesn't exist. This is often done with aliased tables, but in fact it can be totally fictional provided we also provide our own join handler which understands what to do to the query. That can be anything, as long as the SELECT fields we also add make sense. Hence it's fine to do this in hook_views_data():

// Fake table for the 'product is in order' argument, made from a subquery.
$data['commerce_product_commerce_line_item'] = array(
  'table' => array(
    'group' => 'Commerce Product',
    'join' => array(
      // Join to the commerce_product base.
      'commerce_product' => array(
        'left_field' => 'entity_id',
        'field' => 'line_item_id',
        'handler' => 'views_join_commerce_product_line_item',

Our custom join class now has to add the subquery to the view, but it also needs the argument value to do this.

The way I worked around this was to override the ensure_my_table() method in the argument handler. Normally, this calls $this->query->ensure_table() which then creates the join, but ensure_table() can take a join parameter to work with. The overridden version of ensure_my_table() creates the join object, and sets the argument value on it:

function ensure_my_table() {
  // Pre-empt views_plugin_query_default::ensure_table() by setting our join up now.
  // Argh, hack this in for now. This may mean relationships using this break?
  $relationship = 'commerce_product';

  // Get a join object for our table.
  // This is of class views_join_commerce_product_line_item, which takes
  // care of joining to a subquery rather than a table.
  $join = $this->query->get_join_data($this->table, $this->query->relationships[$relationship]['base']);

  // We add the argument value to the join handler as it needs to use it
  // within its subquery.
  $join->argument = $this->argument;

This means that in the build_join() method for our custom join handler, views_join_commerce_product_line_item, we can rely on the argument value that the views has received:

function build_join($select_query, $table, $view_query) {
  // (snip...) build a SelectQuery object for the subquery
  // Set the condition based on the argument value.
  $subquery->condition('cli.order_id', $this->argument);
  // (snip...)
  // Add the join on the subquery.
  $select_query->addJoin($this->type, $subquery, $table['alias'], $condition);

The views_join class's build_join() method is where the Views system of building a query is translated into a DatabaseAPI SelectQuery object. Here we build up our own query (and we don't need Views-safe aliases, as it's a completely internal, non-correlated subquery), and pass it in as a join which uses it as a subquery.

It remains only for the argument handler's query() method to add the conditions for its field, using the alias and field names we gave for the subquery.

In conclusion, the data structure Views understands may appear to be a fixed, declared thing, but with a little bit of tweaking the way tables are joined in a Views query can be affected by both site configuration and user input.

It's Amazing What You Find: Crusty Bits of the Menu System

I've been poking in the innards of the menu system the last few days. This is due to yet another client wanting to do something that goes completely against the grain of Drupal.

In this case, it's the way Drupal only shows you menu links you have access to. This to me seems perfectly reasonable good usability: why show you something you can't use? On the other hand, there is a long-standing feature in Comment module that shows anonymous users a 'Login or register to post comments' link on nodes, so 'incitements to action' or whatever the social media buzzword is do exist in Drupal.

So the challenge was to show a link that the anonymous user can't use, send them to login, and back to the link they wanted in the first place. The last part is just some hook_form_alter() work with form redirection (though it did allow me to discover that everything drupal_get_form() is passed is available to the alter hooks, just in a funny place). The access to the menu item is done by intercepting in hook_menu_item_alter() to save a twin of the menu item, and a checkbox in the menu edit form to trigger this. Even registering the path is easy: a custom menu access callback which takes as access arguments those of the original item plus its access callback and negates whatever the original item would return for access. The part I'm (so far) stuck on is getting hook_menu_alter() to know about what the admin user has done in hook_menu_item_alter(): the next job will probably involve creating a truly ugly query that uses %LIKE% to grab menu items based on their options array.

But that's not what I came here to tell you about today.

In my prodding around of the menu system, I found that menu router items have a 'block_callback' property, with its own database field and everything. Now get this: only one item in the entire {menu_router} table on D6 has this filled, and to boot, it has absolutely no effect. (It's the admin theme page, by the way.) This property is something to do with the way that the root admin page is made out of things that are sort of blocks, but not quite. If the property exists on the router item, then the callback is called to add (!!) to the content. (The admin theme page of course plays no part in the root admin page.)

So we had here a completely useless database field, probably left over from Drupal 5 or even earlier. By the way, in a standard Drupal 7 install, the database field is completely unused. I made a dummy patch to get the issue queue testbot confirm that we never get to this particular piece of code, and the superfluous field has now been removed. It's amazing what you find!

By the way, if anyone fancies some bikeshedding, I still don't have a name better than 'menu_login' for this module. To your paintbrushes!

The Oxford Comma

Here's a little function I wrote today because I needed to be able to turn a list of between one and three items into a string like 'apples, oranges, and pears', 'apples and pears', or just 'stairs'.

I figured I might as well handle everything in one place, and throw in the option to have an 'or' instead of an 'and'. There may be occasions you don't want the Oxford comma, but I can't think of any.

* Grammatically fun helper to make a list of things in a sentence, ie
* turn an array into a string 'a, b, and c'.
* @param $list
*  An array of words or items to join.
* @param $type
*  The text to use between the last two items. Defaults to 'and'.
* @param $oxford
*  Change this from default and you are a philistine.
function oxford_comma_list($list, $type = 'and', $oxford = TRUE) {
  $final_join = " $type ";
  if ($oxford && count($list) > 2) {
    $final_join = ',' . $final_join;
  $final = array_splice($list, -2, 2); 
  $final_string = implode($final_join, $final);
  array_push($list, $final_string);
  return implode(', ', $list);

Now the real question: how would you make this translatable?

Using Constants For Permission Names: WHY?

I keep seeing this sort of thing in so many modules:

define("MY_MODULE_PERM_ACCESS_WIDGETS", 'access widgets');
// Names changed to protect the guilty ;)

Am I missing something, or is this utterly pointless?

The only advantage I see is that you can change the permission string later on. But you actually can't change a permission string once you've made a release without an almighty amount of work in a hook_update_N()[*] to migrate users' existing permissions, so what is the point of using a constant apart from just creating shouty caps noise?

[*] Is there a helper function in core yet for migrating permission names during updates? There really should be.

Once & Only Once: The Conversion To Drupal 7's FieldAPI

I'm not a fan of repeating myself, or of doing the same work twice. So when I first got a look at FieldAPI back at DrupalCon Paris, my thought after 'Wow this is going to change everything,' was 'Every module that converts its custom data storage to this is going to be doing the same work, over and over'.

It seemed to me that we have a lot of modules that add things to stuff. The examples that spring to mind include Taxonomy image (add an image to a taxonomy term), Comment upload (add an uploaded file to a comment), User terms (you get the picture), and the daddy of them all, Image (which now we must call Image Oldskool, due to there being a shiny new Image module in Core).

What all these have in common is that on Drupal 7 their things-to-stuffness would be perfectly served by FieldAPI. Users get more flexibility, an expanding universe of formatters and widgets; module maintainers can write less code and in some cases even hang up their hats on a particular project as it can live entirely in configuration space. Everyone wins. But how to get there?

This matter has been bubbling at the back of my mind ever since Paris. On the one hand I could sort of see how this could be all done with a single framework: you tell it what sort of entity you are dealing with (node, comment, term), which bundle (article, page, which vocab), what your fields are, and how to load the old data. The framework creates the fields for you, then runs a batch to load each object, moves values around in clever ways that you don't need to worry about, and then saves the object. Banzai! Your data is now saved in a field.

On the other hand, this was clearly crack. Of the highest order.

And yet it works. It quite possibly still is crack, and some extra pairs of eyes to de-crackify it would be very welcome. But I can confirm that I've run data conversions on both User terms and oldskool Image nodes, and got term fields on my users and beautiful image fields on my nodes.

There remains some work to be done: I'd like to make conversion to file and image fields a bit more straightforward and create a nice way to specify how to populate the alt and title fields for images. And all this is largely academic and has only run on D7 databases with dummy D6 tables imported, since core's 6 to 7 upgrade process is not currently working. So I could use a hand. And if you have a things-to-stuff module, get in touch and give it a whirl.

Help needed: using jQuery to show passwords as you type

Today I found an article about password usability, which suggested that showing users what they type for their passwords is an improvement to usability.

You can try a working demo, which adds a 'show password' checkbox.

This had me wondering whether this is a feature we should consider having in Drupal.

I duly began writing a small module to do this, but I'm stuck on rewriting the Javascript from the article as well-formed Drupal-friendly jQuery. Now I can make jQuery do fancy things like expand things you click on and whizz things around, and my code so far replaces the password element with the new one, but making that new element itself clickable has me stumped: it's all a bit too meta.

I know when I'm beaten, so I'm blogging this to say: is there a jQuery whizz out there who would care to lend a hand? If so, please comment or email me, and let's make a new contrib module for this!


Subscribe to RSS - drupal