At the end of this piece, I wrote:
One of the remaining questions that I have is “why aren’t process behaviour charts more widespread?” Of course, one of the ironies here is that while XmR charts may have started in manufacturing, most modern lean manufacturers no longer use them due to the sheer deluge of data they have from computerised production. They have moved on from the kind of vanilla SPC we’re learning here. But statistical process control is used in other fields: medicine, for instance — the asthma case study in Making Sense of Data is but one example — which perhaps gives you a sense of the kinds of ‘natural’ processes it might be useful to use process behaviour charts with. If you do a google search for ‘run charts’ — which is a precursor form of the XmR chart — and you’ll find a ton of resources from medical websites. Here is one example.
But why not other fields? Why not tech? Why not business in general? Process behaviour charts are simply a tool to separate exceptional variation from routine variation, to help you tell signal from noise. That seems applicable to a lot of domains. I don’t have good answers here. I will say that I find it irritating to look at a raw time series today, without at least an average line plotted through the middle, because I do not know if I’m looking at something meaningful. I can’t tell if it’s signal or noise. I yearn for process limits, and I feel mild irritation when I hear someone say “ok that’s good right?” when they see a number go up.
I think I have part of my answer. I recorded a podcast interview with @eric yesterday where we talked about his work on Google’s revenue forecasting team (which he’s kindly shared some anecdotes on over at the comments thread on How To Become Data Driven.) And one thing he pointed out is that when you’re working with a process that is exposed to so many exogenous factors, it actually becomes more trouble than it’s worth to use an XmR chart.
So for instance, in the comments thread he talked about how Google’s revenues suddenly blipped in the months before the bank meltdowns in the 2007 global financial crisis, and also how their revenue per query experienced a sudden dip when Michael Jackson died. When you operate a large enough business, your business is basically affected by the entire world, which means any geopolitical or macroeconomic upheaval becomes a source of exceptional variation. In such a situation, there’s no alternative but to do the hard work of figuring out variation from ‘split everything down to the smallest variables, then study year-on-year changes for each of those variables’.
I’m willing to bet that Amazon has the same thing.
Incidentally, this turns out to be consistent with the principles in Wheeler’s books. Wheeler repeatedly says, in both Understanding Variation and Making Sense of Data, that “the purpose of data is insight, which means the best analysis is the simplest analysis that gives you needed insight.”
If you don’t need an XmR chart to tell you a change has occured, or if an XmR chart is too difficult for your context, don’t use an XmR chart.