I usually cringe when I read a comment by a skeptic arguing that “correlation does not prove causation”. Of course, it’s true that correlation does not prove causation. It’s even true that correlation does not always imply causation. There are many great examples of spurious correlations which demonstrate clearly just how silly it is to extrapolate cause from correlation. And the problem is not trivial. Headlines in popular press articles alone can be very damaging as most people simply accept them as true.

BUT…

I cringe because I am afraid that this oversimplification leads people to think that correlation plays no role in causal inference (inferring that X causes Y). It does. In fact, it plays a very important role that skeptics should be just as aware of as the sound bite “correlation does not imply causation”. And that is that causation cannot be logically inferred in the absence of a correlation.

What’s more, that sound bite does nothing to educate people about how and when we should infer cause. So let’s take a look at both problems.

Causation From Correlation

A classic example used to illustrate the problem is the very real relationship between ice cream sales and violent crime. As you can see, when sales of ice cream go up, violent crime increases.

So, should we stop selling ice cream? Of course not.

There are basically two problems with drawing causal conclusions from a correlation:

1. There may very well be a causal relationship, but the causal arrow is unclear. For example, it could be that eating ice cream makes people violent (“sugar high” is a myth, but perhaps it’s milk allergies?). Or, it could be that people get hungry after they’ve just held up a liquor store.
2. There is another variable involved. Most people eventually realize that the correlation of ice cream sales and violent crime is spurious. In other words, it is the result of a common cause–temperature. People are much more likely to eat ice cream in the summer, when it’s warm outside, and they are much more likely to commit violent crimes for various reasons.

Causes

So if correlation doesn’t prove causation, what does?

Well, nothing does. We can’t “prove” and that’s not really what science tries to do. But that’s a point for a different day.

So when can we reasonably infer that X causes Y? It is difficult to reach the bar of causal inference, but the requirements for doing so are actually pretty simple.

First, let’s define “cause” and “effect”*:

Cause
“…that which makes any other thing, either simple idea, substance or mode, begin to be…”

Effect
“…that, which had its beginning from some other thing…”

Confused? Well, let’s simplify it a bit:

A cause is a condition under which an effect occurs.

An effect is the difference between what happened (or is) and what would have happened (or been) if the cause was not present.

I should note here that an “effect” is always a comparison of at least two things. Everything is relative; that’s often a difficult concept to wrap one’s brain around when we are talking about specific examples, but it’s important.

So, let me repeat: a cause is a condition under which an effect occurs.

Causal conditions can be:

• necessary
• sufficient
• necessary AND sufficient
• neither necessary nor sufficient

A condition is necessary if the effect cannot occur without the condition. For example:

To receive credit for a course, you must be enrolled in the course.

In this case, the condition is necessary, but not sufficient. You don’t get credit if you are not enrolled, but enrollment does not guarantee credit (you usually need a passing grade, too).

A condition is sufficient if the effect always occurs when the condition is met. For example:

Decapitation results in death (in humans, at least).

In this case, the condition is sufficient, but not necessary. Nobody can survive without their head, but death can occur in many ways.

For a condition to be both necessary and sufficient, the effect must always occur when the condition is met and it must never occur unless the condition is met. For example:

To win the lottery, you must present a ticket with the correct numbers to the appropriate authorities.

Or this:

To be a parent, you must have a child.

The last one is the tricky one. A cause can be neither necessary nor sufficient, but if it is neither, it must meet another requirement: it must be a non-redundant part of a sufficient condition. This would make it an:

Insufficient
Non-redundant
Unnecessary part of a
Sufficient condition

Or an INUS condition.

The truth is that most causes in the world are INUS conditions. In the social sciences, we deal mostly with INUS conditions.

The big question is how do we identify them?

Well, let’s look at the question of what might cause a forest fire. Some possible causes:

• A lit match tossed from a car
• A lightening strike
• A smoldering campfire

None of these are necessary conditions for a forest fire to start. Science assumes that all effects have causes, so we assume that something occurs to start a fire, but it does not need to be something described here. But all are INUS conditions.

Taking just one as an example, a lit match from a car is not necessary for a forest fire to start since fires can start a number of ways, nor is it sufficient. If tossing a lit match out a car window always ended with forest fires, we’d have a lot more forest fires. There are other conditions which must be met: the match must remain hot long enough to start combustion, there must be oxygen to feed it, and the weather and leaves must be dry enough to keep from smothering it. If those things are met, then the condition as a whole is sufficient. But it must also be non-redundant. If something else in the mix does the job of the match, then the match cannot be considered a cause. Well, oxygen alone cannot start a fire, nor can dry weather, therefore the lit match is a non-redundant part of a sufficient condition.

Another good example of an INUS condition is the presence of a condom to prevent pregnancy. The condom is not necessary to prevent pregnancy; there are many ways. The presence of the condom does not guarantee prevention (effectiveness is ~98% and efficacy even lower). Efficacy is lower than effectiveness, primarily due to compliance. In other words, you have to use the condom right to prevent pregnancy, and even then there are ways the condom can fail. However, when everything is perfect, it prevents pregnancy. Condoms are an unnecessary, insufficient, but non-redundant part of a sufficient condition in the prevention of pregnancy.

So how can we identify an INUS condition?

Causal Inference

In essence, to logically infer that X caused Y, we need to meet three requirements:

1. We must know that X preceded Y. It is not possible for a cause to follow or even coincide with an effect. It must come before it, even if it is fractions of a second.
2. X must covary with Y. In other words, Y must be more likely to occur when X occurs than when X does not occur.
3. The relationship between X and Y is free from confounding. What this means is that no other variable also covaries with X when #1 and #2 are met.

*Definitions adapted from Shadish, Cook, & Campbell’s book “Experimental and Quasi-Experimental Designs for Generalized Causal Inference”

Let me explain each by using some examples:

EXAMPLE 1: A lit match (A) causes a forest fire (B) – YES!

1. A precedes B. – MEETS

• Lit match occurs before forest fire.

2. A covaries with B. – MEETS

• A forest fire is more likely to occur when there is a lit match than when there is no match.

3. The relationship between A and B is free from confounding.  – MEETS

• The lit match does not correlate with other factors, such as oxygen being present or leaves being dry.
• Oxygen is present whether the match is there or not.
• The leaves are dry whether the match is there or not.

Let’s look at one of the headlines I saw a few years ago on the New York Times website, claiming that good grades in high school mean better health in adulthood. Without going into detail about the study, let’s look at the criteria:

EXAMPLE 2: High grades in high school (A) cause better health in adulthood (B) – NO.

1. A precedes B. – MEETS

• High school grades occur before health in adulthood is measured.

2. A covaries with B. – MEETS

• Grades were positively correlated with adult health measures.

3. The relationship between A and B is free from confounding. – FAILS

• On average, persons with higher high school grades have more access to resources than those with lower grades.
• On average, persons with higher high school grades are more intelligent than those with lower grades.
• On average, persons with higher high school grades are more motivated than those with lower grades.

(and probably a lot more)

All of these things are more plausible explanations for the correlation than “grades are good for your health”.

But notice that correlation is a requirement to infer cause, always. What I see all too often are detailed, lengthy explanations for things that are not correlated. An excellent example is “Lunar Fever”. I’ve seen lots of explanations for why emergency rooms and police stations are busier during a full moon, from very good (e.g., the light from the moon makes it more likely that people will be out and about) to looney (the human body is full of water, which is affected the way the tides are affected). The first explanation might be the most parsimonious one, but it’s still useless because THERE IS NO CORRELATION. Studies are pretty clear that E.R.s and police stations are not busier during a full moon than other times of the month.

EXAMPLE 3: The full moon (A) causes people to act nutty (B) – NO.

1. A precedes B. – MEETS

• Behavior is measured after the full moon appears. 

2. A covaries with B. – FAILS

• There is no correlation between behavior and moon phases.

3. The relationship between A and B is free from confounding.  – PROBABLY MEETS

• There might be variables which are correlated with the full moon that have nothing to do with the moon phase, but it’s not really a relevant question if #2 isn’t met.

So how do we meet these requirements?

Ÿ#1: To establish temporal precedence, we conduct experiments.

#3: We eliminate confounding variables by isolating the hypothesized cause – the only difference between one condition and another is the causal variable. To do this we need:

• a Counterfactual (equivalent comparison/placebo)
• Random assignment (explained below)
• Controls to avoid other confounds such as expectation (blind, double-blind, randomized order)

ŸIf, after measuring the hypothesized effect, the outcome establishes covariation (#2), the only explanation for that covariation is cause.

By the way, we eliminate hypothesized causes in the same manner, by setting up conditions in which the only explanation for the outcome is that A does not cause B.

EXAMPLE 4: Test the hypothesis that acupuncture (A) reduces pain (B). 

1. We conduct an experiment comparing acupuncture to doing nothing. In doing this, we’ve established temporal precedence because the treatment precedes the measure of pain.
2. We find that when we compare the pain of controls to those who have received acupuncture, the former has more pain than the latter, establishing a correlation.
3. However, are there confounding variables? Yes, there are. The reduction in pain could be caused by anything that covaries with acupuncture, including the fear of being stuck with needles and the expectation that the treatment will work.

What we have in this case is an improper counterfactual. Participants were not blind to the treatment or its expected effects. Also, the body probably releases endorphins in response to being stuck with needles, so while they might claim that acupuncture reduces pain, the explanation isn’t where or how the needles were placed.

When we change the counterfactual by comparing acupuncture to sham acupuncture, the correlation disappears. Those who receive the sham treatment are in no more pain than those who received the “real” treatment.

When no randomly-assigned, double-blind, placebo-controlled experiments are possible…

One “control” that is absolutely necessary in any experiment to eliminate confounding variables is random assignment to treatment groups. In other words, the people who receive, say, the sham treatment in the acupuncture example are assigned to that treatment by a random process (like rolling a die). We do this because if we used any criterion other than randomness to assign, that criterion could explain any differences (or lack thereof) in outcomes. For example, if I put all of the people with headache pain in one condition and all of the people with back pain in the other, the outcome could be explained by the fact that these people have different medical conditions.

This is a problem for a lot of research, especially in education and health. For example, we cannot ethically assign people to smoke, randomly or otherwise, so how can we eliminate confounding variables? People who choose to smoke are different from people who do not in many, many ways, and any of those ways can explain higher rates of cancer. But I don’t know anyone who would deny that smoking causes cancer.

So what happens when we cannot do the kinds of “true” experiments that control for all possible confounds? Do we give up?

Of course not.

In these cases, we rely on evidence converging from different approaches to the question until the odds tell us that it is highly, highly unlikely that the correlation is spurious.

When it comes to smoking causing cancer, we first establish a correlation with temporal precedence. That’s easy. Smokers are much more likely to get cancer later in life than nonsmokers. But, since we cannot eliminate all confounding variables, we must conduct many different studies, eliminating hypothesized explanations. We know, for example, that smoking causes cancer in rats (that has its own ethical problems, but it’s been done). We can’t be sure that the effects are the same in humans, but when we reconcile that with other studies in humans that control for variables such as access to health care and amount of exercise, the probability that smoking does not cause cancer is reduced. The more studies and the more hypotheses eliminated, the more likely the remaining hypothesis (that smoking causes cancer) is the correct one.

I invite you to think about how we know that vaccines do not cause autism. While we cannot (ethically) randomly assign some kids to get vaccines while others do not, the answer becomes clear when approached from many different angles.

So I hope I haven’t tied your brain in knots with this over-simplified, yet lengthy explanation of causal inference. It’s a topic near and dear to my heart as a methodologist and one that I think skeptics should get a handle on if at all possible.

NOTE: this post also appears on the wonderful site about crazy coincidence, theoddsmustbecrazy.com.

Monty Hall

I have always been amused and intrigued by responses to “The Monty Hall Problem”, especially when I talk about it to audiences with a high concentration of engineers and mathematicians. If you are familiar with it, but you’ve always struggled with an unsettled feeling of “this can’t be right”, read further and let me know if my explanation of the solution helps to alleviate the discomfort. If you are not familiar, I guarantee you will give your brain a workout by reading on.

First posed to statisticians in 1975, “The Monty Hall Problem” is well-known among academics because it still sparks debate. Many seem to think that disagreements about its solution stem from issues in the clarity of the problem, but I contend that it really stems from human flaws in the way that we process information.

I often discuss this problem in statistics and cognitive psychology courses for several reasons. It is a great exercise in probability calculation and it can be used to teach basic mathematical modeling (and its purpose). An added benefit, since almost all of my students were psychology majors, is that it also illustrates a flaw in human cognition as well as a pattern of problem solving. Even a knowledgeable statistician feels the need to run simulations to see the solution in action. Even then, fully grasping the mechanisms behind the answer often requires brute force cognition.

In general, human beings have a very difficult time wrapping their brains around concepts of probability. It is much like a visual illusion; we know that the lines are parallel/the circles are the same size/there is no motion, but we can’t make our brains process it in a way that represents that reality. It’s just not how our visual system works. I hypothesize that one of the reasons that probability is such a difficult field for most people is that it involves theory and models, which are distinct from observations and we must represent them differently in our minds to properly deal with them. Applications of probability often involve switching gears from the realm of models to data or vice versa and this is where I think most mathematicians get side-swiped in The Monty Hall Problem.

The Poser

In essence, here’s the problem:

You are a contestant on Let’s Make a Deal!and Monty loves your creative costume (a teddy bear carrying a human doll), so he calls on you to make a deal. Monty says, “There are three doors – Door #1, Door #2, and Door #3. Pick one and you get to keep whatever is behind it.”You’ve seen the show (you weren’t just walking down Ventura Boulevard in a teddy bear costume for fun), so you know that it is highly likely that there is a coveted BRAND NEW CAR! behind one of those three doors. If you choose wrong, however, you might end up with an ostrich…

Everyone hopes for a car. Some get donkeys or other animals.

You choose Door #3.

Monty then says, “Let’s see what’s behind Door #1!” and the door opens to reveal one of the many consolation prizes (and product placements), a lifetime supply of Rice a la Roly.

Cool! You might get that car after all!

Well, the show was successful because the shell-game-huckster-style of Monty Hall rarely stopped there. In this case, he does what he often does, offers to let you switch from your first choice (Door #3) to the only remaining option, Door #2.

Should this woman switch?

Should you? Does it matter?

Not the Problem

Before I get into the solution, let me first deflect a common complaint from mathematicians. The most well-known version of the problem, from its Wikipedia entry:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1 [but the door is not opened], and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

This version does not specifically state the name of the show or indicate the way that game shows of its era worked. If you have never seen the television show (i.e., you are younger than 35), or any game show of its kind, let me explain. Monty is in control of almost everything that happens. The only thing “contestants” can do is make choices when Monty offers them. As you will see, they had more control over their odds of winning than once thought, but Monty manipulates some of the build-up by choosing which items to reveal at different steps in the game.

Unfortunately, many probability theorists and mathematicians took issue with the lack of clarity in the problem (context is important sometimes). This provides a face-saving ‘other version’ for the geeks who get it wrong the first time. But whenever I hear comments like, “Okay, given this version, that Monty knows where the car is.” I usually think, “Of COURSE he knows where the car is! There is no other way to play the game!” and wish that people were more able to accept that they are just as human as everyone else.

The problem itself is written clearly, though: it specifically states that a door without a car behind it is revealed before you are given the option to switch. If the situation was a fully-randomized, double-blind game (like “Deal or No Deal”), then the option to switch would not even be on the table if the car is behind the revealed door. There would be no problem in that case. Therefore, the problem is a question of whether you should switch in a controlled setting – one in which the only participant who doesn’t know the location of the big prize is you.

The issue of knowledge is a factor in our processing of the problem, but it’s not what Monty knows that matters. It’s what you (the subject of the problem) know.

So, let’s put that complaint behind us and get back to the problem.

The Answer, and How to See it for Yourself

Hopefully, if the problem is new to you, you’ve spent some time trying to solve it instead of going with your first gut feeling, which was probably, “It doesn’t matter.”

It does. You should switch.

If you don’t believe me, try running some simulations. You’ll have to run a lot in order to get a large enough sample to be certain to see the trend, but here are a few ways to do it:

• Use your favorite program (MATLAB, R, etc.). There is a good database of pre-written simulators for this here. I am partial to Excel myself, even though it’s a bit more cumbersome. I just don’t remember enough code to use another program.

• Use a web-based simulator. Do it at least a hundred times, choosing to switch for half of the trials, and keep a tally your results.

• Use a die to simulate the outcome, assigning 1-3 to “Door #2″ and 4-6 to “Door #3″ (e.g., if you roll a 5, Door #3 is the one with the car). Roll at least a hundred times, choosing to switch for half of the trials (before rolling!). Keep a tally of the results.

What you will see is that switching will result in winning a car in approximately 2/3rd of the trials while staying will only provide a win in 1/3rd of them.

I know what you’re thinking. “But, there are only two doors left, so it should be 50/50!”

Why it is so Difficult to Accept

Human cognitive development is an interesting process. We learn to interpret information from the environment very quickly so that we can respond to that environment, but learning to reason hypothetically takes more time. Even adults with scientific training have a difficult time separating the concept of variables (each has a set of possible values) and data (values which are known).

In practice, hypothetical situations are often conditional (e.g., “If A, then B”). We tend to use information about what is to reason about what could be. We do this because it often works, but it is one of the ways in which our brains can lead us astray. For example, given the premise, “If I study, I will get a good grade on the exam”, what is the most sound conclusion when presented with a good grade? The most common response is, “I must have studied”, but that is not sound. In this premise, studying provides a guarantee for a good grade, but there is no statement that studying is the only way to get a good grade. It does not, for example, read, “If and only if I study…”

In the case of the Monty Hall Problem, the probability of winning is set before you pick a door. No matter which door you choose, the probability is 1/3rd. This is because there is a 1/3rd probability that the car is behind the door you chose given the information you had when you chose it. In reality, the car is behind one of the doors, so the probability it is behind Door #2 is 100% if it is there and 0% if it is not there. Probability is not a useful way to discuss what is or what happened; it is a tool for predicting what is likely to be true/happen.

The new information provided by revealing a loser changes the circumstances and this where we get trapped in our representations of models and data, possibilities and facts.

You had a 1/3rd chance of winning because there were three, equally-likely locations to choose from. It seems as if cutting the choice down to two should change the odds of winning to 1/2. It seems that way because we are focused on the probability that a given piece of information is true (e.g., that the car is behind Door #1) and not the probability that an event will occur (e.g., that we will win the car). The probability that we will win the car relies on the number of possible states of reality. This, in turn, initially relies on the number of locations for the car. When the situation changes, we try to adjust probabilities based on possible locations (which has changed) rather than on the number of possible states of reality (which has not).

Basically, when Monty makes the second offer, the offer is to switch from the door we have (#3) to the door we don’t have (#1 or #2). It does not matter that only one of those doors is left; there is still only a 1/3rd chance that our door has the car and a 2/3rd chance that the set of the other two contains the car.

Getting Un-Stuck

If you change the way you represent the problem from the beginning, the solution might seem more reasonable. Specifically, instead of thinking in terms of assigning probabilities to doors, think in terms of assigning probabilities to outcomes: winning verses losing.

Three possible states of reality, each with one winner and two losers

Let’s go step by step…

Monty asks you to pick a door from three choices. Behind one of those doors is a car. There are three possible locations and it must be in one of them, so there are three possible states of reality.

You choose to bet on Door #3; there is a 1/3rd chance that you will win the car.

There is a 2/3rd chance that you will not win the car.

This would be true no matter which door you chose.

Monty reveals that one of the remaining doors is a loser. At least one will be a loser since there is only one winner and you can choose only one. The car, however, does not move. Even though there are only two locations left, so there are still three possible states of reality. What’s changed is that we now know more about each of those possible states (there are fewer locations for the car to be):

So, if we model the problem in terms of the probability of winning with Door #3, the model itself does not change after the losing door is revealed. What changes is that we would no longer want to choose that door, so it is no longer among our options. This leaves us with only two options: keep the door we have or switch to the remaining door. The odds of winning/losing with Door #3 have not changed, but eliminating an option allows us to make a better choice – switch.