The fallacy of applying complicated models to complex problems (AKA why personality profiling is BS)

I find that some of my ideas take a few weeks, months or even years to form. This one took almost exactly a year before coalescing (coagulating?) in my mind.

Personality, strength, or psychometric models such as Myers-Briggs, DISC, Belbin, Predictive Index and the myriad others available, attempt to codify people and their preferences, personalities, behaviours and values into archetypes, using fixed (usually opaque) algorithms.

These archetypes are then categorised and collated into larger group types, and in many organisations, used  to inform everything from role selection, management approach, or even hiring decisions, (which is illegal in many cases).

But they don’t work.

The reason they will never work is because they try to map a complicated framework onto a complex problem.

As Cynefin shows us, complicated systems can be really big, and appear complex, but the laws of cause and effect don’t change. When you press the A/C button in your modern car (which is “complicated”), the A/C comes on, and the same thing happens every subsequent time you do it. But in a complex system such as a human being, that’s not the case – asking a teammate to help you out with a task one day results in them helping you, but on another day, they might tell you to stick it. Cause and effect change in complex systems; and humans are complex. Really complex.

Complicated systems can seem messy, but an action results in the same result each time. People are not like that. They are complex, and groups of people even more so. Cause and effect changes constantly – pressing the equivalent of that A/C button on a complex human has one effect today and a different effect tomorrow.

And that is why personality, psychometric, “strength” tests etc will never work in the way people desire them to.

All models are wrong. Some are useful.

The problem is when you use a model and apply it to a complex problem in the assumption that it’s right.

“It ain’t what you don’t know that hurts you, it’s what you know for sure that just ain’t so.”

And people selling these systems either know this, in which case they’re selling snake oil. Or they don’t know it, and they’re simply being optimistically gullible, looking for simple answers to complex problems. To be fair, we humans are almost infinitely susceptible to the seductive simplicity of personality archetypes, even more so when they’re about us. This is known as the Barnum effect, where it’s possible to give everyone the same description and people nevertheless rate the description as very accurate.

Flawed evidence of personality test reliability

MBTI fails on both validity and reliability tests, as do most other personality and psychometric tools. Proponents (usually people selling them) are keen to point out reliability measures that show, with a degree of error, that the same person taking the same test at a different time often obtains a similar result. This only serves to highlight the problem however – just as I would tell you my favourite colour is yellow if you ask me today, and I’d respond usually with the same a month later – it doesn’t follow that my favourite colour has anything to do with my personality, nor that my personality is stable over time. Equally, I may be lying. My favourite colour is actually blue.

Most of these systems apply an assumption of dichotomies, or even force them – you are either X or Y: cannot be both, and you cannot change from one to the other. This has been disproven too.

We should all be suspicious of algorithms that describe us or make decisions about us that are closed course, and psychometric tests are no different. Predictive Index have repeatedly declined to open source their algorithm, ostensibly to protect their intellectual property.

Even the most “trusted” test of all in academia, the “Big 5” has been found to be systematically sexist. Criticism of MBTI and others extends further, due to a highly westernised, English-language primary approach.

Dangerous tools?

Evidence shows that, far from being a “short-cut” to more insightful leadership, tools such as these can be harmful – they may convince managers that they’re doing “good management”, and discourage further effort to improve management and leadership behaviours. At worst, they’re actively discriminatory and detrimental to individual and team performance, reducing the quality of human interactions and decreasing levels of psychological safety.

What do you think? Are they a useful tool, or a dangerous over-simplification?