{"id":1466,"date":"2021-05-10T15:53:33","date_gmt":"2021-05-10T15:53:33","guid":{"rendered":"https:\/\/tomgeraghty.co.uk\/?p=1466"},"modified":"2024-01-22T16:13:06","modified_gmt":"2024-01-22T16:13:06","slug":"the-fallacy-of-applying-complicated-models-to-complex-problems-aka-why-personality-profiling-is-bs","status":"publish","type":"post","link":"https:\/\/tomgeraghty.co.uk\/index.php\/the-fallacy-of-applying-complicated-models-to-complex-problems-aka-why-personality-profiling-is-bs\/","title":{"rendered":"Critique of Personality Profiling (Myers-Briggs, DISC, Predictive Index, Tilt, etc)"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">I find that some of my ideas take a few weeks, months or even years to form. This one took almost exactly a year before coalescing (coagulating?) in my mind. I\u2019ve been thinking about personality tests in the context of efficacy, equity and neurodiversity recently, and it troubles me.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">I\u2019ve always found personality testing problematic &#8211; indeed any pseudo-Jungian approach to putting people into type categories I find highly distasteful and potentially harmful.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Critical literacy is sorely lacking in the business and management world. This is possibly largely because it\u2019s not rewarded: we reward confidence, sticking by decisions, bullishness and simple answers to complex problems.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In respect to diversity, inclusion, and equity, I just can\u2019t square the desire to categorise people and their personalities with the very real need for inclusion and diversity of ways of thinking. It\u2019s seems simply antithetical.<\/span><\/p>\n<h3><b>To summarise the flaws in personality testing:<\/b><\/h3>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">There is very little evidential basis behind personality profiling, and significant<\/span><a href=\"https:\/\/www.jstor.org\/stable\/1170497\"> <span style=\"font-weight: 400;\">evidence against it<\/span><\/a><span style=\"font-weight: 400;\">.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The models are usually based on false dichotomies of &#8220;big picture vs detail-oriented&#8221;, when there is no evidence that these exist.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The models are also based on<\/span><a href=\"https:\/\/news.harvard.edu\/gazette\/story\/2020\/09\/joseph-henrich-explores-weird-societies\/\"> <span style=\"font-weight: 400;\">WEIRD<\/span><\/a><span style=\"font-weight: 400;\"> (Western, Educated, Individualistic, Rich, and Democratic) societies, and fail to recognise collectivist, holistic strengths.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">They rarely address context and inter-relational behaviours, but instead make assumptions about behaviour from individualistic measures.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">They tend to assume that our personalities are largely fixed and unchangeable.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">These tools can lead to false and potentially harmful assumptions made about other people and the way they behave.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The tools may be used for unethical (and illegal) practices such as recruitment, selection for promotion, or other decisions made about someone without their consent.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">In my experience, they are one of the most highly weaponised management tools ever created.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Because they lead people to believe that they can understand someone based upon a profile, they can prevent further, discussion, examination and effort to understand people and their ever-evolving uniqueness.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The algorithms used are rarely open. Algorithms inherit the biases of those people that created them, and if we are making ourselves subject to analysis by algorithm, I want to know what it&#8217;s doing and who designed it.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Many tests are biased (see above) &#8211; for example, the<\/span><a href=\"https:\/\/qz.com\/1201773\/we-took-the-worlds-most-scientific-personality-test-and-discovered-unexpectedly-sexist-results\"> <span style=\"font-weight: 400;\">Big Five was shown to bias against women<\/span><\/a><span style=\"font-weight: 400;\"> and categorise them as more aggressive when answering identically to a man: because the original data model was flawed.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">To avoid a critique of poor reliability, we&#8217;re often told to avoid doing the tests more than once.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">When assigned a profile, we are generally not allowed to dispute it. Even though we have spent decades in our own minds, a five-minute test is assumed to know more about me, than me.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">Even scientists who are most concerned with assessing individual differences in personality would concede that our ability to predict how particular people will respond in particular situations is very limited.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Personality, strength, or psychometric models such as Myers-Briggs, DISC, Belbin, Predictive Index, Tilt and the myriad others available, attempt to codify people and their preferences, personalities, behaviours and values into archetypes, using fixed (usually proprietary and opaque) algorithms. There is usually a commercial reason that these tests are closed-source, because companies don&#8217;t want someone copying the code and using it for distributing it, but it also prevents detailed analysis and evaluation of the algorithm.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<h2><b>Repeatability and validity<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">These archetypes (such as &#8220;maverick&#8221;, or &#8220;Inventor&#8221;) are then categorised and collated into larger group types, and in many organisations, used\u00a0 to inform everything from role selection, management approach, or even<\/span><a href=\"https:\/\/raksheen.medium.com\/i-got-rejected-because-i-failed-a-personality-test-30f3a875007\"> <span style=\"font-weight: 400;\">hiring decisions, (which is illegal in many cases).<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400;\">In 20 years of management, I have never seen a psychometric analysis tool generate a constructive outcome, particularly from a diversity, equity and inclusion (DEI) perspective. I also find it interesting that personality testing *only* exists in the business world, not in the academic world of actual psychological study. Do business managers actually think they know something psychologists don\u2019t?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In my opinion (somewhat backed up by many years of experience and study), categorising people and attempting to simplify the complexities of our nature, in an attempt to make other people and ourselves more predictable, is certainly a seductive proposition. But it is error-prone, and <\/span><b>dangerous<\/b><span style=\"font-weight: 400;\">.<\/span><a href=\"https:\/\/www.linkedin.com\/pulse\/20130917155206-69244073-say-goodbye-to-mbti-the-fad-that-won-t-die\/\"> <span style=\"font-weight: 400;\">Adam Grant, organisational psychologist at Wharton, agrees.<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<h3><b>Psychometric analyses don&#8217;t work. Indeed, they are often damaging.<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">The reason they will never work is because they try to<\/span><a href=\"https:\/\/www.scientificamerican.com\/article\/how-accurate-are-personality-tests\/\"> <span style=\"font-weight: 400;\">map a complicated framework onto a complex problem<\/span><\/a><span style=\"font-weight: 400;\">. You may be familiar with<\/span><a href=\"https:\/\/www.independent.co.uk\/news\/carl-jung-a-psychic-pyramid-seller-1356142.html\"> <span style=\"font-weight: 400;\">Carl Jung<\/span><\/a><span style=\"font-weight: 400;\">, and his &#8220;<\/span><a href=\"https:\/\/conorneill.com\/2018\/04\/21\/understanding-personality-the-12-jungian-archetypes\/\"><span style=\"font-weight: 400;\">12 Archetypes<\/span><\/a><span style=\"font-weight: 400;\">&#8221; of &#8220;Ruler, Sage, Explorer, etc&#8221;, which are frequently criticised as mystical or metaphysical essentialism. Since archetypes are defined so vaguely and since archetypal images have been observed by many Jungians in a wide and essentially infinite variety of everyday phenomena, they are neither generalisable nor specific in a way that may be researched or demarcated with any kind of rigour. Hence they elude systematic study, which is true of many other domains of knowledge that seek to reduce complex problems and systems to simple, archetypal models and solutions.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">As Cynefin shows us, complicated systems can be really big, and appear complex, but the laws of cause and effect don&#8217;t change. When you press the A\/C button in your modern car (which is &#8220;complicated&#8221;), the A\/C comes on, and the same thing happens every subsequent time you do it. This is rather obviously not the case with people.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In a complex systems such as humans, asking a teammate to help you out with a task one day results in them helping you, but on another day, they might tell you to stick it; maybe they&#8217;re hungover, stressed and busy, maybe they&#8217;re tired, or maybe they just don&#8217;t feel like helping. Cause and effect change in complex systems; and humans are complex. Really complex. Which is why &#8220;<\/span><a href=\"https:\/\/reliabilityweb.com\/articles\/entry\/the-soft-stuff-really-is-the-hard-stuff\"><span style=\"font-weight: 400;\">the soft stuff is the hard stuff<\/span><\/a><span style=\"font-weight: 400;\">&#8220;.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Complicated systems can seem messy, but an action results in the same result each time. People are not like that. They are complex, and groups of people even more so. Cause and effect changes constantly &#8211; pressing the equivalent of that A\/C button on a complex human has one effect today and a different effect tomorrow.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">And that is why personality, psychometric, \u201cstrength\u201d tests etc will never work in the way people desire them to. People don&#8217;t fit into boxes, and neither should we try to.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<h3><b>All models are wrong. Some are useful.<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">The problem is when you use a model and apply it to a complex problem in the assumption that it&#8217;s right.<\/span><\/p>\n<p><i><span style=\"font-weight: 400;\">\u201cIt ain\u2019t what you don\u2019t know that hurts you, it\u2019s what you know for sure that just ain\u2019t so.\u201d<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">And people selling these systems either know this, in which case they\u2019re selling snake oil. Or they\u2019re simply being optimistically gullible, looking for simple answers to complex problems. To be fair, we humans are almost infinitely susceptible to the seductive simplicity of personality archetypes, even more so when they&#8217;re about us. This is known as the<\/span><a href=\"http:\/\/psych.fullerton.edu\/mbirnbaum\/psych101\/barnum_demo.htm\"> <span style=\"font-weight: 400;\">Barnum effect<\/span><\/a><span style=\"font-weight: 400;\">, where it&#8217;s possible to give everyone the same description and people nevertheless rate the description as very accurate.<\/span><\/p>\n<h3><b>\u00a0<\/b><\/h3>\n<figure id=\"attachment_2164\" aria-describedby=\"caption-attachment-2164\" style=\"width: 300px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-2164\" src=\"https:\/\/tomgeraghty.co.uk\/wp-content\/uploads\/2021\/05\/Screenshot-2023-12-22-at-10.03.17-300x300.png\" alt=\"\" width=\"300\" height=\"300\" srcset=\"https:\/\/tomgeraghty.co.uk\/wp-content\/uploads\/2021\/05\/Screenshot-2023-12-22-at-10.03.17-300x300.png 300w, https:\/\/tomgeraghty.co.uk\/wp-content\/uploads\/2021\/05\/Screenshot-2023-12-22-at-10.03.17-150x150.png 150w, https:\/\/tomgeraghty.co.uk\/wp-content\/uploads\/2021\/05\/Screenshot-2023-12-22-at-10.03.17-100x100.png 100w, https:\/\/tomgeraghty.co.uk\/wp-content\/uploads\/2021\/05\/Screenshot-2023-12-22-at-10.03.17.png 555w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><figcaption id=\"caption-attachment-2164\" class=\"wp-caption-text\">the barnum effect sketchplanations https:\/\/sketchplanations.com\/the-barnum-effect<\/figcaption><\/figure>\n<h3><b>Flawed evidence of personality test reliability<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">MBTI<\/span><a href=\"https:\/\/jobtalk.indiana.edu\/HRMWebsite\/hrm\/articles\/develop\/mbti.pdf\"> <span style=\"font-weight: 400;\">fails on both validity and reliability tests<\/span><\/a><span style=\"font-weight: 400;\">, as do most other personality and psychometric tools. Proponents (usually people selling them) are keen to point out reliability measures that show, with a degree of error, that the same person taking the same test at a different time often obtains a similar result. This only serves to highlight the problem however &#8211; just as I would tell you my favourite colour is yellow if you ask me today, and I&#8217;d respond usually with the same a month later &#8211; it doesn&#8217;t follow that my favourite colour has anything to do with my personality, nor that my personality is stable over time. Equally, I may be lying. My favourite colour is actually blue.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Most of these systems apply an assumption of dichotomies, or even force them &#8211; you are either X or Y: cannot be both, and you cannot change from one to the other.<\/span><a href=\"https:\/\/www.capt.org\/jpt\/pdfFiles\/JPT_(2009)_69.01.pdf\"> <span style=\"font-weight: 400;\">This has been disproven too.\u00a0<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400;\">When I did a &#8220;Predictive Index&#8221; test, I was told that I was far from empathetic, because I was evidence and data driven. According to PI, someone cannot be both evidence-oriented and empathetic. Not only is this offensive,<\/span><a href=\"https:\/\/journals.sagepub.com\/doi\/abs\/10.2189\/asqu.51.1.1\"> <span style=\"font-weight: 400;\">it&#8217;s completely unfounded<\/span><\/a><span style=\"font-weight: 400;\">. In fact research shows that <\/span><a href=\"http:\/\/asq.sagepub.com\/content\/51\/1\/1.short\"><span style=\"font-weight: 400;\">people with more rigorous and evidence-driven thinking skills are also better at understanding and managing emotions<\/span><\/a><span style=\"font-weight: 400;\">. These are simply not valid tests.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We should all be<\/span><a href=\"https:\/\/www.nytimes.com\/2018\/04\/04\/us\/politics\/cambridge-analytica-scandal-fallout.html\"> <span style=\"font-weight: 400;\">suspicious of algorithms that describe us or make decisions about us<\/span><\/a><span style=\"font-weight: 400;\"> that are closed source, and psychometric tests are no different. Predictive Index have repeatedly declined to open source their algorithm, ostensibly to protect their intellectual property.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The key to the Big 5 model is its simplicity. It doesn\u2019t sort anybody into a \u201ctype,\u201d just informs them where they fall on a continuum of personality traits. There are no tricks and no surprises to be revealed, and it&#8217;s not a black box. However, even though it&#8217;s the most trusted psychological profiling test in academia, the &#8220;Big 5&#8221; has been found to be systematically sexist. &#8220;<\/span><a href=\"https:\/\/qz.com\/1201773\/we-took-the-worlds-most-scientific-personality-test-and-discovered-unexpectedly-sexist-results\"><span style=\"font-weight: 400;\">Women are told they\u2019re significantly more disagreeable than men who answer questions identically.<\/span><\/a><span style=\"font-weight: 400;\">&#8220;<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Criticism of MBTI and others extend even further, often due to a highly westernised, English-language, neurotypical approach.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<h3><b>Dangerous tools?<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">Evidence shows that, far from being a &#8220;short-cut&#8221; to more insightful leadership,<\/span><a href=\"https:\/\/recruitingblogs.com\/profiles\/blogs\/what-no-one-tells-you-about-psychometric-testing\"> <span style=\"font-weight: 400;\">tools such as these can be harmful<\/span><\/a><span style=\"font-weight: 400;\"> &#8211; they may convince managers that they&#8217;re doing &#8220;good management&#8221;, and discourage further effort to improve management and leadership behaviours. At worst,<\/span><a href=\"https:\/\/www.theguardian.com\/tv-and-radio\/2021\/mar\/03\/they-become-dangerous-tools-the-dark-side-of-personality-tests\"> <span style=\"font-weight: 400;\">they&#8217;re actively discriminatory and detrimental to individual and team performance<\/span><\/a><span style=\"font-weight: 400;\">, reducing the quality of human interactions and decreasing levels of psychological safety.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Conversely,\u00a0 I&#8217;ve actually found value in doing &#8220;<\/span><a href=\"https:\/\/www.wizardingworld.com\/news\/discover-your-hogwarts-house-on-wizarding-world\"><span style=\"font-weight: 400;\">which Hogwarts house are you in?<\/span><\/a><span style=\"font-weight: 400;\">&#8221; or &#8220;<\/span><a href=\"https:\/\/www.buzzfeed.com\/whitneyjefferson\/which-sex-and-the-city-character-are-you\"><span style=\"font-weight: 400;\">Which sex and the city character are you?<\/span><\/a><span style=\"font-weight: 400;\">&#8221; quizzes with teams. They&#8217;re obviously nonsense, but they facilitated a good discussion with team members about preferences and styles &#8211; and it was much more fun than MBTI!<\/span><\/p>\n<p><span style=\"font-weight: 400;\">(In fact, those quizzes have an advantage over some of the &#8220;official&#8221; tests because they make no pretence of scientific accuracy.)<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Finally, I\u2019ve never come across a strongly competent leader who used personality testing and categorisation. It seems to me (and I\u2019m conscious of my own biases here) that these tests can sometimes risk replacing empathy. A way to feel like you\u2019re understanding people, and &#8220;doing the work&#8221; without actually putting in the effort to do so.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Personally, given all the flaws and limitations of personality profiling, I hope organisations stop using them, and businesses stop trying to make money out of them.\u00a0 We try not to use flawed tools to do finance, accounting, software development, design, or data analysis. Why is it acceptable to use flawed tools to understand and manage the most important thing in organisations &#8211; people? And why, when we realise that they&#8217;re flawed, are they so &#8220;sticky&#8221;? Why can&#8217;t we seem to get rid of them?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">What do you think? Are they a useful tool, or a potentially dangerous over-simplification of human nature?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Read more: https:\/\/adamgrant.substack.com\/p\/mbti-if-you-want-me-back-you-need and https:\/\/www.psychologytoday.com\/gb\/blog\/give-and-take\/201309\/goodbye-to-mbti-the-fad-that-wont-die<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I find that some of my ideas take a few weeks, months or even years to form. This one took almost exactly a year before coalescing (coagulating?) in my mind. I\u2019ve been thinking about personality tests in the context of efficacy, equity and neurodiversity recently, and it troubles me. I\u2019ve always found personality testing problematic &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/tomgeraghty.co.uk\/index.php\/the-fallacy-of-applying-complicated-models-to-complex-problems-aka-why-personality-profiling-is-bs\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Critique of Personality Profiling (Myers-Briggs, DISC, Predictive Index, Tilt, etc)&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,118],"tags":[45,123],"class_list":["post-1466","post","type-post","status-publish","format-standard","hentry","category-blog","category-psychological-safety","tag-leadership","tag-psychometric"],"_links":{"self":[{"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/1466","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/comments?post=1466"}],"version-history":[{"count":33,"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/1466\/revisions"}],"predecessor-version":[{"id":2188,"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/1466\/revisions\/2188"}],"wp:attachment":[{"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/media?parent=1466"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/categories?post=1466"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tomgeraghty.co.uk\/index.php\/wp-json\/wp\/v2\/tags?post=1466"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}