Many companies are “data driven” but they make bad decisions based on data when they cannot or do not separate signals from noise.
2 Likes
@markgraban Thank you so much for coming into the forums and answering people’s questions about XmR charts — you’ve been teaching it for much longer than I even know about them.
This is just … wild to me. Pretty much everyone I’ve shown it to goes “omg, YES, this is what I need” — I’m basically taking what you are saying now and realising that I have not talked to nearly as many kinds of people as you have, and that these folks probably exist.
This is … bizarre. When I read Wheeler’s book, a lightbulb basically went off in my head and I realised this was what I was missing all these years. I can’t understand people who think like this.
1 Like
Definitely not unusual or bizarre. For many executives, “data driven” means “I use data to support the conclusion I have already decided upon, and ignore the data that contradicts it”. Data is not used to discover the truth, but as a cudgel or persuasion tool by filtering or massaging the data until it says what you want.
This may be a startup vs big company thing. At a startup, the feedback is too strong to ignore the data - if you don’t respond to what’s actually happening, you’ll be out of business. At a big company with inertia and momentum, it is very possible to be “successful” while ignoring the data, because the consequences may not be truly visible for years.
My favorite example of this was exploring a sensitive competitive question. We couldn’t answer it directly but came up with 4 different data analyses, none of which were definitive and all of which had issues, but which all pointed towards the same conclusion. The executives dismissed all four analyses because they contradicted their intuition (I mean, they said it was due to the incomplete nature of each one). I later discovered the (single) analysis they were basing their conclusions on, and it was a terrible analysis, lacking rigor, but it agreed with their gut feeling. Data driven!
Basically, you probably don’t talk to people whose ego matters more to them than truth 
5 Likes
I need to learn from how you’re pitching this to people, @cedric !
But as @eric wrote, it could be a matter of different audiences. The one startup I work with regularly (KaiNexus) has embraced XmR charts from the CEO and other senior leaders on down.
I’m working mainly with big organizations (including healthcare) where politics is sadly often more important than data.
There’s another dynamic that’s frustrating in the context of Lean management… which is supposed to be about continuous improvement.
I’ll show Process Behavior Charts to people as a better alternative to red/green “bowling charts” – and they’ll say things like, "But this is how our Lean consultant showed us how to display data last year and we can’t change it on people now (or admit we weren’t doing it the best way).
How is that table of numbers better than even a simple run chart?
Ego and politics.
3 Likes
It was surprising to me (at least the first few times) when I encountered situations similar to what you described.
I always thought that execs and leaderships genuinely want their companies to thrive, but I realize there are elements of ego and politics at play that blind them to data that were readily availabe, like what @markgraban said.
I saw it at my old company when the CEO decided on spending a year to roll out a new platform pricing plan that targets Enterprise companies without any solid reasoning or data.
As a Product Manager, I was intimately familiar with how our customers were using the product and I know our best customers were mostly SMBs who are still in a transitioning period where a single product in our platform were most valuable. They were using the product most frequently, exploiting it in every way that they can. Could have just asked me and I’d happily provide my perspective (I in fact did provide my perspective anyway, though it was to no avail).
Blinded to this fact, the CEO simply saw that revenue distribution skewed towards enterprises (honestly, how is that not the case) and decided that enterprise companies are the best customers.
After a year of efforts, the platform pricing launch was underwhelming, with many customers confused as to why they’re being forced to pay a lot more money for things that they don’t need.
But the worst thing is, I don’t think the company learned anything, because the feedback was so distorted that the CEO just said that “it’s because we didn’t execute good enough”.
2 Likes
I think you’re all clearly right — I’ve mostly been talking to startups, under 150 people.
Thank you for the correction.
Damn. Increasingly I believe I should just use these ideas for my own businesses and my own benefit, and not care that much about proselytising about them. Life is too short.
3 Likes
I agree that there is limited impact these kinds of ideas can have on larger organizations that are to some degree set in their ways. But keep this in mind:
- “Limited” is not the same as “none”. Even making pockets of a large organization more efficient and effective improves the lives of those folks, and serves as an example the larger org has a chance to learn from. And sometimes you even find fertile soil where these ideas can gain momentum and make a difference. Think of it more like your experiments to get on the front page of Hacker News - you can increase your propensity for success, but it will be unpredictable whether any given engagement takes off or not.
- One vector for changing how large organizations work is to make smaller organizations successful that then outcompete those larger organizations into oblivion. There is a reason the average age of large organizations continues to fall.
Of course, I may just be saying this since I benefit from your proselytizing and want you to keep at it! 
4 Likes