Simpsons Paradox and the Ecological Fallacy in Data Science

Simpsons Paradox

I’m currently studying for a Master’s Degree in Global Health at The University of Manchester, and I’m absolutely loving it. Right now, we’re studying epidemiology and study design, which also involves a great deal of statistical analysis.

Some data was presented to us from an ecological study (a type of scientific study, that looks at large-scale, population level data) called The WHO MONICA Project that showed mean cholesterol vs mean height, grouped by population centre (E.g. China-Beijing or UK-Glasgow).

In this chart, you can see a positive correlation between height and cholesterol, with a coefficient of 0.36, suggesting that height may be a potential risk factor for higher cholesterol.

However, when the analysis was re-run using raw data (not averaged for each of the population centres), the correlation coefficient was -0.11.

So, when using mean measures of each population centre, it appears that height could be a risk factor for higher cholesterol, whilst the raw data actually shows the opposite is slightly more likely to be true!

This is known as an “ecological fallacy” – because it takes population level data and makes erroneous assumptions about individual effects.

This is a great example of Simpsons Paradox.

Simpsons paradox is when a trend appears in several different groups of data but disappears or reverses when the groups are combined.

Table 1 in Wang (2018) is a relatively easy example. (This is fictional test score data for two schools.)

(Also, please ignore for a moment the author’s possible bias in scoring male students higher – maybe this is a test about ability to grow facial hair.)

male

male

female

female

School

n

average

n

Average

Alpha (1)

80

84

20

80

Beta (2)

20

85

80

81

It’s clear if you look at the numbers that the Beta school have higher average scores (85 and 81 for male students and female students respectively).

However, if you calculate the averaged scores for individuals in the schools, Alpha school has an average score of 83.8 and Beta has just 81.8.

So whilst Beta school *looks* like the highest performing school when broken down by gender, it is actually Alpha school that has the highest average scores.

In this case, it’s quite clear why: if you only look at the average scores by gender, it’s easy to assume that the proportion of male and female pupils for each school is roughly the same, when in fact 80 pupils at Alpha school are male (and 20 female), but only 20 are male at the Beta school, with 80 female.

Using gender to segment the data hides this disproportion of gender between the schools. This may be appropriate to show in some cases, but can lead to false assumptions being made.

The same issue can be seen in Covid-19 Case Fatality Rate (CFR) data when comparing Italy and China. Kegelgen et al (2020) found that CFRs were lower in Italy for every age group, but higher overall (see table (a)) in the paper.

The reason, when you see table (b), is clear. The CFR for the 70-79 and 80+ groups are far higher than for all other age groups, and these age groups are significantly over-represented in Italy’s confirmed cases of Covid-19. This means that Italy’s overall CFR is higher than China’s only by dint of recording a “much higher proportion of confirmed cases in older patients compared to China.” China simply didn’t report as many Covid-19 cases in older individuals, and the fatality rate is far higher in older individuals. Italy has a more elderly population (median age of 45.4 opposed to China’s 38.4), but other factors such as testing strategies and social dynamics may also be playing a part.

Another example of Simpsons Paradox is in gender bias among graduate admissions to University of California, Berkeley, where it was used in reverse. In 1973, the admission figures appeared to show that men were more likely to be admitted than women, and the difference was significant enough that it was unlikely to be due to chance alone. However, the data for the individual departments showed a “small but statistically significant bias in favour of women”. (Bickel et al, 1975). Bickel et al’s conclusions were that women were applying to more competitive departments such as English, whilst men were applying to departments such as engineering and chemistry, that typically had higher rates of admission.

(Whether this still constitutes bias is the subject of a different debate.)

The crux of Simpsons Paradox is: If you pool data without regard to the underlying causality, you could get the wrong results.

References:

Bokai WANG, C. (2018) “Simpson’s Paradox: Examples”, Shanghai Archives of Psychiatry, 30(2), p. 139. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5936043/ (Accessed: 21 October 2020).

Julius von Kugelgen, Luigi Gresele, Bernhard Scholkopl, (2020) “Simpson’s paradox in Covid-19 case fatality rates: a mediation analysis of age-related causal effects.” Arxiv.org. Available at: https://arxiv.org/pdf/2005.07180.pdf (Accessed: 21 October 2020).

P.J. Bickel, E.A. Hammel and J.W. O’Connell (1975). “Sex Bias in Graduate Admissions: Data From Berkeley”(PDF). Science. 187 (4175): 398–404. doi:10.1126/science.187.4175.398. PMID 17835295. https://homepage.stat.uiowa.edu/~mbognar/1030/Bickel-Berkeley.pdf

WHO MONICA Project Principal Investigators (1988) “The world health organization monica project (monitoring trends and determinants in cardiovascular disease): A major international collaboration” Journal of Clinical Epidemiology 41(2) 105-114. DOI: 10.1016/0895-4356(88)90084-4

Global Health Masters Degree (M.Sc.) at The University of Manchester – Review

This is a placeholder for upcoming review post regarding module content, teaching delivery, curriculum, online teaching, assessment methods etc.

The course is affiliated with the Humanitarian and Conflict Response Institute at the University of Manchester

The below is copied from the Global Health Masters Degree  (MSc) at The University of Manchester website pages

Community Approaches to Health : Examine issues of psycho-social care, behaviour change, aging, micro-insurance, advocacy, holistic health, HIV, nutrition, breast feeding, hygiene promotion and immunisation.

Ethics, Human Rights and Health : Consider the role of gender, health inequalities, dignity, legal frameworks, rights based approaches to health, reproductive rights, Millennium development goals 4, 5, and 6, child rights, and accessing illegal drug users and commercial sex workers.

Health Systems and Markets: Look at the social determinants of health, the work of civil society organisations, the interfaces between states and economies, organisational change, health financing, urban health, rural access, food security, agriculture, and eradication programming.

Risk, Vulnerability and Resilience: An introduction to public and global health, risk assessments and management, epidemiology, population ageing, the determinants of child survival, and pandemics.

You will be able to engage fully with the course content and other students via lectures, discussion boards, group work, online chat, question and answer sessions with the tutor, and peer-to-peer feedback and assessment.

PGCert, PGDip and MSc awards

You can exit the course with a PGCert award after Year 1, a PGDip after Year 2, or an MSc after Year 3.

Teaching and learning
The course will begin with an online induction session that explains how the course will progress and how you can fully engage with the curriculum and the online classroom environment. It will set out the key contacts and what each student can expect.

Academic and pastoral support will be offered online by the programme director, course leaders and teaching assistants, who will be responsible for monitoring progression through the course. A dedicated programme administrator will be responsible for dealing with day-to-day enquiries.

The course lasts for three years in total. You will study four course units in each of Years 1 and 2. Each of the four units comprises eight weeks of teaching followed by one week of assessment.

You will complete each unit in turn before progressing to the next. The format is designed to be adaptable to the needs of professional students and provides opportunity for reflection between units.

Year 3 comprises the dissertation for the MSc award. Students will submit a research proposal and be allocated a dissertation supervisor. You will then be guided through key milestones in the completion of your dissertation.

The course has been designed to recreate a classroom learning environment in an online format. You will be able to engage fully with the course content and with peers via lectures, discussion boards, group work, online chat, question and answer sessions with the tutor, and peer-to-peer feedback and assessment.

Coursework and assessment
All assessment will take place online. Each of the four units in Years 1 and 2 will conclude with a selection of assessments, including multiple choice tests, group assignments such as wikis, and prose-based assessments.

Certain academic pieces placed in the discussion forums are used as part of the overall assessment process for each unit (10%).

Each student will provide a 350 to 500-word (excluding references) written academic piece expressing a view or perspective upon a question raised by the tutor/convenor weekly during the course of the course.

This will provide eight pieces of primary work that will be submitted to the discussion board per course unit. Engagement on the discussion boards is required throughout the course.

You will also receive formative feedback and guidance throughout the course, which will enable you to progress and develop your confidence and analytical skills.

Course unit details
You will study four course units in each of Years 1 and 2. Each of the four units comprises eight weeks of teaching followed by one week of assessment.

Year 3 comprises the dissertation for the MSc award.

Exit awards

You will receive 60 credits for the successful completion of each year of the course, totalling 180 credits for the MSc award.

It is possible to exit the course earlier than this with 60 credits for a PGCert award or 120 credits for a PGDip.

All of the credits you earn will be transferable to other academic institutions.

10 travel tips you might not read elsewhere.

May partner and I have been on the road for around 9 months now, travelling around Europe and South East Asia, and I’ve learned a few things about myself, the world, and how to travel. Some things are obvious, like how important it is to have decent travel insurance, buying local pay-as-you-go sim cards instead of racking up a big roaming bill, and setting some sort of spending budget…

However, here are ten tips (and a few bonus ones) that you might not read on the average travel blog.

Vietnamese lizard
  1. Everything breaks, so get good at fixing stuff, and learn to sew. Pack superglue, electrical tape, and a small sewing kit, and you can fix just about anything. I’ve repaired shoes, shorts, sleeping bags, watches, sunglasses, bags and cameras over the past few months.
    Repair on a sleeping bag.

    Sugru is also a great resource, but each pouch has to be used immediately once it’s opened, so take some, but use it as a last resort. And seriously, learn some basic sewing skills, like darning socks, sewing on buttons, or repairing rips and tears.

    solar charger, water purifier, backpack
    Halfway up to Annapurna base camp
  2. A good first aid kit is crucial. Stock up on antiseptic, plasters, bandages, bite cream, water purifying tabs, tweezers and anything else you might need personally. I even took some emergency tooth filling repair stuff, and I used it. You can also use superglue to patch up wounds, but be careful as it can get very hot when it dries, and the standard stuff sets stiff so it’s not good for cuts over skin that moves, like the soles of your feet.
    Kathmandu, Nepal

    Zinc oxide tape is great to prevent blisters and secure bandages. Electrical tape serves as a superb fix for a wound dressing if you’re going to get it wet. I cut my toes fairly badly while cliff jumping (whilst climbing up, not jumping down) and used bandages and electrical tape for a couple of weeks to keep it bandaged while in the sea.

    Cliff jumping in Vietnam
    Kayaking and paddle boarding in Vietnam

    Also stock up medicine when possible. Painkillers aren’t always available and you can go through them quicker than you’d expect. Take with you antihistamines, sleeping pills, anti-diarrhoea and rehydration salts. If you can, get some antibiotics like amoxicillin for general wound or tooth infections, and metronidazole for stomach bugs  and amoebic nasties.

    Our trekking guides, Santosh and Puskar

    If you find yourself somewhere like Nepal where you can get antibiotics over the counter, buy some. Buy more than you think you need (but not so many that you’ll be thrown in jail for smuggling prescription drugs). If you get an infection and you’re in deepest Cambodia, you really don’t want to leave it until you can find a reliable doctor.

    Walking in the Himalayas

    Buy hand sanitiser when you can find it cheaply, because it gets expensive when you’re remote. You’ll probably acclimatise to the local stomach bugs eventually, but using hand sanitiser regularly will reduce the likelihood of getting a bad one. I wasn’t careful enough in our first week in Kathmandu, and after vomiting in the street, and a very tense taxi ride, rather regretted it.

    Pub Street, Siem Reap, Cambodia
  3. Get a decent knife and learn how to use it and look after it properly. Buy one that locks open so that you don’t cut your fingers off when closing it. Keep it sharp using a decent stone or steel. Sharpening a blunted knife is really difficult, but keeping a knife sharp just takes a bit of discipline. I carried around a stone in my bag for 8 months just for this purpose. You’ll use your knife for everything from preparing food, repairing clothes, and cutting hair!

    Look after your knife.
  4. Eat local food whenever possible. Some of the most amazing food I’ve had travelling has been in the cheapest street stalls and markets. However, good local food isn’t always available. You’ll often find yourself forced to eat whatever is provided, and it might not be to your (or frankly, anyone’s) taste. Tabasco sauce can make all sorts of bland, weird, slightly off, or otherwise less-than-great food palatable. Take a little bottle of your your own with you, and even carry a few spices, salt, pepper, and sugar.
    Nepalese street food on the road to Pokhara.

    Cambodian market food
  5. Keep your hip flask topped up. I recommend whisky because it’s drinkable by itself and goes well mixed with lots of stuff from coke to coffee, gin is pretty nice to carry around but good luck finding tonic in rural SE Asia. Cognac could work for you if you’re an artist or something. Vodka if you’re doing the whole serious alcoholic thing. When you get invited to an impromptu beach party, or a chillout on a porch, you’ll be pleased you’ve got your old faithful hip flask with you.

    Chilling in Lisbon
  6. Buy a bunch of dry bags of different colours and sizes. You can almost never have too many dry bags. They’re really useful for simply keeping your kit organised and separated, so you’re not hunting around for your socks every day or wondering where your favourite big-night-out T-shirt is. It also means that if your bag gets wet or something springs a leak inside, most of your stuff will be ok. I used a big red one to keep dirty washing in (red for danger, obvs), and also used various dry bags for trips to the beach, backpacking in the rain, or kayaking trips. 

    Unpacking along the Annapurna Trek
  7. Be careful using squat toilets. In many parts of Asia, you’ll come across squat toilets. Once you get used to them, they’re actually pretty good, as long as they’re kept reasonably clean. However, make sure you zip up your pockets if possible, or at least put your phone and wallet somewhere else when you’re using a squat toilet. If something falls out, you really don’t want to be rummaging around down there, however fancy and expensive your phone is. 

    Riding around Koh Chang, Thailand.
  8. When travelling, if there are seatbelts, wear them. The same goes for helmets whilst riding motorbikes. It’s cool to be safe, kids. Driving standards outside Europe and the US are significantly lower, and in many places there’s not even a requirement to pass a test in order to drive on public roads. Our coach from Kathmandu to Pokhara overturned after having to avoid a speeding car on the wrong side of the road who’d miscalculated an overtake. We were fine, as was everyone in the coach apart from our Annapurna guide, who smacked his knee hard. We were lucky, but it could have been much worse.
    Everyone was fine, fortunately.

  9. Wear synthetic underwear. Seriously. If you’re walking a lot, or spending a lot of time in hot and humid conditions, you don’t want to be wearing cotton underwear, because they absorb water and will at best be very uncomfortable, and worst cause such severe chafing that you can barely move, or it gets infected. Synthetic underwear doesn’t absorb water, so it’s way more comfortable, particularly for trekking and/or humid weather. It’s also great for impromptu outdoors swimming, because your pants will dry quickly and frankly they’re also less transparent when wet than cotton pants…
    Swimming in cold water in El Bosque, Andalusia.

    Snorkelling in Thailand.
  10. Go offline. Going off-grid can be a great experience, especially if it’s for a decent amount of time. If you get a chance to get out into the wilderness, the mountains, or out to sea, then use that opportunity to go fully offline and away from the distractions of modern life. We went off-grid for about 12 days on our trek to Annapurna Base Camp, and it made an amazing experience even better.
    Prayer flags at Annapurna base camp

    Although initially you might suffer from bad FOMO, or fear that your parents and friends might worry if they don’t hear from you (by the way, you should probably tell people you’re going offline lest they spark an international manhunt), after a few days you’ll feel back in touch with the real world a little more, maybe feel a bit more calm, and able to be more present in the moment. You might even find that after a week or two of being off-grid, you really don’t want to reconnect after all.

    Taking the night train to Saigon
Jade relaxing on a snorkelling trip in Thailand

Finally, here’s a few bonus tips that didn’t quite make it to the top ten:

  • Shower gel is a fine alternative to washing machine detergent. Just don’t use too much.
  • Toilet paper is valuable stuff, always keep some with you, especially on a mountain trek. A roll of toilet paper can cost the same as a night’s accommodation high up in the mountains.

    The Himalayas at sunrise
  • Always carry snacks, because you will definitely find yourself in places or on journeys for considerable lengths of time with no access to food. Individually wrapped cereal bars are great. Try to avoid things that melt.
  • Take digital photos of your passport and other important documents, in case you lose them, or in case you need to show them to someone but don’t have the documents on you.
Putting our feet up during a Himalaya trek.
  • Take rechargeable batteries for torches and other gear, and a charger for them. 
  • Learn to open beer and wine bottles without openers. Don’t use your teeth. 

    Mucking about (fallen angel pose) in Lisbon.
  • Carry some US dollars for emergencies. Almost everywhere accepts them as currency, or at least to change them. They’re often essential to pay for visas at the border around SE Asia too.
  • If it’s within your budget (or someone else’s), get yourself a proper adventure camera. Jade bought me the amazing Olympus Tough TG-5, which is waterproof, drop-proof, and packed with features.

    Olympus TG5 camera.
  • Get a Curve card. It’s a Mastercard, so it’s accepted nearly everywhere, and you can use it instead of your credit and debit cards, so you can keep them safe somewhere and only expose your Curve card to potentially risky ATMs and restaurant owners. Curve also converts currencies for you, saving you money on fees. Shameless promo: sign up for Curve here with code NPWZA and you’ll get £5, and so will I.
  • Take a tablet or laptop with you in order to work, research and book travel, or simply watch Netflix. We spent many, many hours on shonky wifi connections from nearby cafes watching Anthony Bourdain on Netflix. Try to download stuff to watch or listen to later if you know wifi might be patchy.

    Lemon tree in Prado Del Rey, Andalusia
  • Practise mindfulness and meditation; you’ll many have periods of time when all you can do is sit and wait, so you may as well put it to good use, and you can continue to practise when you’re back in the “real” world. If you’re new to it or prefer a little guidance, there are some great apps out there for meditation practice, such as Headspace.
Getting to Annapurna base camp

Travelling is very much about experiencing abstract, intangible things. Meeting new people, seeing different parts of the world, experiencing other cultures, eating different food and finding ways to be at ease with discomfort such as sleeping in bad beds or walking for hours with a heavy pack on your back.

Don’t worry about buying souvenirs. They’re just added weight. Take photos, record the sounds, make memories and friends.

Vietnamese fishing boats

As Bourdain says: “It seems that the more places I see and experience, the bigger I realise the world to be. The more I become aware of, the more I realise how relatively little I know of it, how many places I have still to go, how much more there is to learn.

Wild swimming in a pool in Koh Chang, Thailand.
Jade at Ninh Van Bay, Vietnam.
Getting bamboo tattoos in Thailand
Barcelona beach

Leave a comment with any travel tips you have.