Note: This post also appears on Insight, the official blog of the Skeptic Society.

A few months ago I wrote about the psychology of vaccine denial. In the post I discussed two publications, one of which (Nyhan, et al.) found:

Corrective information reduced misperceptions about the vaccine/autism link but nonetheless decreased intent to vaccinate among parents who had the least favorable attitudes toward vaccines. Moreover, images of children who have MMR and a narrative about a child who had measles actually increased beliefs in serious vaccine side effects.

None of the interventions increased parents’ intent to vaccinate.

Then, a couple of weeks ago, a friend sent me a link to this piece describing research which seems to contradict that finding. The authors (Horne, et al.) concluded that

…highlighting factual information about the dangers of communicable diseases can positively impact people’s attitudes to vaccination.

These two conclusions seem to contract each other. Which should we believe?

Many times this question comes down to the quality of the research. In this case, I believe these are both fairly well-designed studies. One, however, is more precise than the other in several ways. I believe that precision highlights the complexity of the issue as well as giving us a better idea of the direction that vaccine promotion should take.

Let’s look at the differences in sampling and method between the two studies.

The Horne study sampled 315 men and women. In the Nyhan study, the final sample was 1759 parents with children under the age of 18. In most research, 315 subjects is more than sufficient and more is not always better. The danger in larger samples is to find effects that are statistically significant, but not practically significant. However, when comparing conflicting findings, it is best to bet on the side of the larger sample.

Then there’s the question of limiting the study to parents. Although Horne compared parents to non-parents and found no significant differences in attitudes or effects, noise is noise. These two groups of people vary, and the attitudes of non-parents are not particularly relevant. Limiting the study to parents would give me more confidence in the robustness of the findings and their application in real-world activism.

Still, if both are reasonably well-designed studies by competent researchers, the end results should not contradict each other. So there must be more going on. And there is.

For one thing, this is a great example of how complex social sciences are. We should never make policy decisions based on a single study and this demonstrates why. Replication, especially with variants of measures and materials, is essential to learning the best methods of persuasion.

For another, these studies differ in more than just sampling techniques. The Horne study is much simpler and, in fact, oversimplifies. Nyhan, et al. included three outcome measures, each addressing a specific attitude:

1. The belief that vaccines cause autism.
2. Perceived risk of side effects from vaccines.
3. Intent to vaccinate one’s child/children.

By contrast, the Horne study involved a single measure which combined answers to five specific questions (such as “I intend to vaccinate my child.” and “Doctors would not recommend vaccines if they were unsafe.” to come up with a more vague “vaccine attitudes” scale. Even if the answers to these questions are highly correlated, how interventions affect those answers may be very different. They certainly were in the Nyhan study. And if “effective” is defined as increasing intent to vaccinate, then the Horne study does not answer the question it purports to answer. Personally, I am more interested in intent to vaccinate than I am in any other aspect of “vaccine attitudes”, so the Nyhan study’s findings are much more meaningful to me.

In general, it is best to measure outcomes of interest as specifically as possible, but of course the more outcomes a researcher studies, the larger the sample must be.

Finally, and perhaps the most important difference between these two studies, is the timing of the experimental portion. When measuring the effect of treatments or interventions on attitudes, an experiment should be spaced over time. A researcher will measure the attitude, then wait before applying a treatment and measuring the attitude again. When polled about attitudes, those attitudes are brought to mind. This affects our receptiveness to relevant information in complex ways, ways that vary based on a number of other factors such as the strengths of our attitudes and the way the questions are worded. However, allowing subjects to forget about the initial survey provides a more accurate picture of how people confronted with information in the real world may respond to it.

The Horne experiment was conducted a day after the initial screening while the Nyhan experiment occurred about two weeks after initial screening.

My conclusion? I think the issue is complex, but while Horne’s findings appear easier to understand, Nyhan’s findings are more specific, answer more interesting questions, and can be more easily viewed within the framework of well-established knowledge about human decision-making (e.g., cognitive dissonance).

That, and we need more research if we are to develop effective ways of increasing vaccination rates.

Horne Z, Powell D, Hummel JE, & Holyoak KJ (2015). Countering antivaccination attitudes. Proceedings of the National Academy of Sciences of the United States of America, 112 (33), 10321-4 PMID: 26240325

Nyhan B, Reifler J, Richey S, & Freed GL (2014). Effective messages in vaccine promotion: a randomized trial. Pediatrics, 133 (4) PMID: 24590751

When I saw the Washington Post headline “Internet searches are convincing us we’re smarter than we really are” in my Facebook feed yesterday, I was only a little bit skeptical. Most readers are probably aware that I have been studying self-esteem and narcissism for some time, particularly the aspect of overconfidence. Over confidence prevents learning and interferes with rationality, so it is important to understand its sources.

It seems to be a rare moment these days when I can point to a mainstream media piece reporting a finding from the field of psychology without mistakes ranging from a minor distortion of the implications to facts so incorrect that they’ve reported the opposite of what was found. I’m thankful to say that this time the Washington Post’s piece is quite good, although the headline doesn’t quite fit.

The research is not flawless, but the authors address most of the limitations by running a series of experiments. The overall large sample size and a number of controls and checks compensate for the fact that it was conducted online. It’s not perfect, but few studies in this field are. It is just one series of experiments, so any conclusions drawn should be tentative.

All of that said, the findings are interesting. What they found: people who were asked to use the internet to find or confirm their answers to a series of questions gave, on average, higher ratings of their ability to answer questions in a different domain than those who were asked not to consult the internet. This finding held across experiments and the researchers were able to ferret out some details, including:

• This was an increase in confidence among internet users, not a decrease in confidence among those not allowed to consult the internet.
• The increase was not due to access to information or even the use of the internet to get the information. It was the act of searching for that information that caused the increase in confidence.
• Participants were not considering their access to knowledge in their responses about ability, but their knowledge. They appeared to be conflating shared knowledge (the internet) with personal knowledge (what’s in their heads).

In one experiment, half of the participants were asked to search for specific web pages (e.g., a scientificamerican.com page about dimples on golf balls) while the other half received the information on those pages. This shows that the difference in confidence cannot be attributed to the knowledge itself, but the act of using the internet to search for it. In another, participants did not rate their ability to answer the subsequent questions, but instead predicted their brain activity while doing so. We cannot attribute the difference in confidence levels to assumptions that access to information would be available. People really believe they possess the knowledge.

Overconfidence is one characteristic in a list of those associated with narcissism, so it is useful to look at these findings in relation to narcissism in general, especially considering the historical context. The world wide web’s emergence is relatively recent. In The Narcissism Epidemic, Twenge and Campbell document a sharp increase in narcissism over the past 30-40 years. Although they attribute that increase to changes in culture, tracing it back to a best-selling self-help book called I’m Okay, You’re Okay that was popular in the early 1970s, cultural trends of entitlement can be seen decades earlier in “you deserve it” advertising approaches. Certainly the self-esteem movement of the 80s and 90s moved it along. So far, the internet’s blame has been limited to the vanity aspect of narcissism. There has been quite a bit of research suggesting that social media users are, on average, more narcissistic than others, but such correlations are confounded by factors of age, gender, and others difficult to tease out. It would also be extremely difficult to determine the direction of cause.

The experiments are limited to internet search and the specific characteristic of overconfidence, but they do not suffer from the same problems of confounding as those related to social media use. These findings suggest that searching the internet actually causes an increase in one’s self-assessed knowledge. Perhaps people think of the internet as an extension of the self.

The bottom line, I think, is that overconfidence clearly has many sources. Given this research, it appears that we can count the existence of Google™ among them.

Fisher M, Goddu MK, & Keil FC (2015). Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge. Journal of experimental psychology. General PMID: 25822461

I don’t know if this could really be called “new”, but it’s a form of anti-intellectualism that usually goes unnoticed. I find it particularly frustrating because I so often see it often among people who claim to respect knowledge, education, and expertise. It is an ironic lack of respect for that same knowledge, education, and expertise.

The Psychology of Vaccine Denial

I’m sure you’re wondering what I’m talking about here, so I will get to the point. Rebecca Watson wrote a short piece, published in Skeptical Inquirer Online, that seems to question the potential effectiveness of a bill currently in the works in California which would eliminate non-medical exemptions for vaccination requirements to attend public school. I say “seems to” because it’s actually unclear.

My main criticism of the piece itself is not that much of what she says is blatantly wrong, but that the piece doesn’t go anywhere and the research cited doesn’t support the weak, barely identifiable thesis at all. It is disjointed and doesn’t flow well. The transition from the topic of education to that of the bill is a huge leap. Her conclusion makes little sense given the rest of the piece. It’s only a few paragraphs (very short for SI), but in those few paragraphs she manages to treat some important research shallowly and selectively, missing the valuable knowledge that a nuanced look at the findings would provide. I won’t make that mistake here.

She cites two articles, the first mention is this:

Researchers Brendan Nyhan and Jason Reifler have spent the past few years conducting studies that seem specifically designed to depress science communicators. Last year, they published a paper in which they showed that correcting myths about the MMR vaccination actually decreased a parent’s intention to vaccinate.

What’s missing is that this was only true among those “with the least favorable vaccine attitudes”.

Even showing participants images of sick children was counterproductive, increasing their belief that vaccines are connected with autism.

Yes, but the “even” part is very misleading. Emotional pleas such as describing disease risks and showing images of or telling stories about children with diseases all increased this belief, but education refuting a link successfully reduced that same belief. 

From the article:

“Autism correction” is most effective in reducing agreement with the autism misperception. Strong agreement declines from a predicted probability of 8.9% to 5.1% (and likewise for other response options). By contrast, the predicted probability of strong agreement increases to 12.6% for “Disease images.” Similarly, the predicted probability of believing serious side effects from MMR are very likely increased from 7.7% among control subjects to 13.8% in the “Disease narrative” condition.

This combination of results tells us a lot about what is happening when people are confronted with different strategies, yet nothing Watson wrote went beyond the few bits she selected from the abstract.

For the second citation, Watson writes:

Last month, they conducted a similar test using the common belief that the flu vaccine causes the flu. The results were the same: correcting the misconception only decreased the subjects’ self-reported intention to get vaccinated.

But this is what the article’s abstract actually says:

Corrective information adapted from the Centers for Disease Control and Prevention (CDC) website significantly reduced belief in the myth that the flu vaccine can give you the flu as well as concerns about its safety. However, the correction also significantly reduced intent to vaccinate among respondents with high levels of concern about vaccine side effects – a response that was not observed among those with low levels of concern.

After reading the article, I can tell you that the education measures worked across the board–in every concern level, educating people about the vaccine significantly reduced belief in the myth. However, those with the most concern about side effects dug in when it came to intent to vaccinate–not everybody, those with the most concern. (BTW, they didn’t conduct the test “last month”.)

These are finer points, but they are far from trivial. The details are what tell us what’s going on. I would not expect someone without an education in psychology to recognize the implications, although Abbie Smith, who reviewed the first study in a blog post last year, managed quite a bit of insight (which she wrote about with care, describing the findings in detail and not speculating beyond the what happened in the study).

And this is where the anti-intellectualism is most apparent in Rebecca’s piece:

At this point, we can only guess as to the reason why this happens.

No, Rebecca, at this point, you can only guess. So, if you don’t know why it happens, then nobody does?

To anyone who has studied decision making, reason, attention, or just about any area of social psychology for a few years, this statement is absurd. The pattern of results found in these two studies is exactly what I would have predicted. We have decades of research telling us why this happens.

Some brief explanations are provided right there in the articles’ discussion sections. The authors mention loss framing, danger priming, and other effects.

For a much more in-depth look, I will ask you to read Mistakes Were Made, But Not By Me by Carol Tavris and Elliot Aronson. The nutshell is that people do all sorts of mental gymnastics to reduce something we call cognitive dissonance–a tension between contradictory attitudes or an attitude and a behavior–in ways that allow us to avoid changing the behavior or strongly-held attitude.

In this case, when those holding strong anti-vaccine attitudes accept that their expressed reasons (e.g., autism) for those attitudes are invalid, they simply find another reason to maintain the attitude (e.g., side effects).

People are invested in the choice not to vaccinate, not the reason for the choice.

This would be especially true for those who have acted on that choice. The alternative is to accept that they have put their children at risk for no reason.

So, although Watson is not incorrect in reporting that some approaches backfired, she failed to see or report the nuances in these findings that tell us why and what we might do about it. And there’s more that I would not expect a layperson to recognize.

These studies only measured attitudes immediately following education–education that worked in dispelling myths about those vaccines. What I would like to see is follow up research examining attitudes months or years afterward. What happens, for example, when people are educated, then given time to change their attitudes without threat to their egos and identities? I predict that a large portion of them will change their minds. Much of the resistance is probably rooted in ego threat. Giving people time and space may allow them to save face while changing the attitude to reduce the cognitive dissonance associated with the conflicting ideas.

Furthermore, in these laboratory studies, parents are asked to report their attitudes prior to the exposure to materials. This is a form of declaration, committing people to a viewpoint that they then feel compelled to defend. That’s not what happens in real world situations.

So her next paragraph…

Do people hold their anti-vaccination beliefs so deeply that correcting a misconception only encourages them to spend time digging around for another reason to hate vaccines? If so, then the answer may be to address the underlying reasons for the belief instead of the scientific facts.

How are these two sentences connected? That people find other reasons to maintain a behavior or attitude is not evidence that there is some hidden reason. And expressed reasons are precisely what what are addressed in the studies she cited. She’s come full-circle with nothing at all to show for it.

Cognitive dissonance and the unconscious strategies people use to reduce it are human nature. We cannot “address” human nature so easily. We can educate people about human nature and how it does not always lead us to the best decisions to meet our goals (and by “we”, I mean people who have studied human nature, such as social scientists with years of training and knowledge), but of course that’s a much broader goal. Increasing vaccination rates is a public safety issue that must be addressed with more urgency and specificity.

Finally, all of this came down to this one guess of hers:

For instance, perhaps the belief is rooted in a fear of government control over individual choices.

Um, seriously? This came out of nowhere as if she just didn’t have an ending to her story, or perhaps couldn’t come up with a good segue to get to the one thing that she meant to talk about: the California bill that may eliminate personal belief exemptions for unvaccinated kids to attend public schools. Rebecca’s logic is that if fear of vaccine harm is actually rooted in fear of government control, then the bill might make matters worse.

This is a huge leap. For one thing, she cites no research showing that fear of government control has anything to do with the average vaccine denier’s choices. Even if it did, the very research she cited shows that removing all government involvement in vaccination would not change intent to vaccinate (those with the most concern would simply find another thing to worry about). But more importantly, she begs the question:

But will the law (which already exists in West Virginia and Mississippi) only encourage the anti-government anti-vaccine activists to band together and renew their efforts to fight for their freedom to harm innocent kids?

Well, now, why not take a few minutes and do a little research to find out how laws affect vaccine rates?

According to Pew,  “states with the strictest immunization laws tend to have the highest immunization rates” (they have a nice graph sorted by vaccination rates and Mississippi is at the top).

Not surprising. People tend to follow the law, and if they want to send their kids to public schools, they must vaccinate. But does this change attitudes? I think a lot of people would say that they don’t care, as long as it changes the behavior, but I think we can all agree that changing the attitude would be best.

And stricter immunization laws will change attitudes and beliefs about vaccines. How do I know this? Simple: cognitive dissonance.

One early finding in studies of cognitive dissonance theory is that it is often easier for people to change an attitude than it is to change a behavior. We have seen numerous examples of this, not only on laboratory studies, but in real-world behaviors such as smoking and exercise habits. Once invested in a behavior, the attitude follows as a matter of reducing the tension because we are invested in the behavior, not the reason for the behavior.

In this case, the biggest thing currently in the way of attitude change is the personal belief exception. Remove that, and behaviors must change. Once behaviors change, attitudes will follow, especially with education, which will pave the way for attitude change by giving parents a way to engage in the behavior (of sending their child to public school) without dissonance. This is especially true when the parent has not declared their attitude prior to education, as they do in a laboratory study.

What the research cited suggests, when included in the context of decades of psychological research about the relationships among attitudes, behaviors, and values, is that a combination of stricter laws and education correcting myths about vaccines is not only highly likely to increase vaccination rates, it will also decrease perceptions of risk of harm from vaccines.  Giving people facts does indeed work. It works to educate people about facts. If you want them to change their attitudes, however, you need to dig a little bit deeper.

To head off what will surely be a the first thing Watson’s supporters will point out: what’s the difference we came to the same conclusion? Her argument is “I think we should try X because nothing else seems to work” and mine is that X is what the science suggests. Only one of these is a valid argument. The ends do not justify the means.

The New Anti-Intellectualism

The implication that just anyone can write about this stuff with authority is the kind of anti-intellectualism I’m referring to in the title.

And before you assume that I am saying that skeptics have nothing to say, think again. Pseudoscience and fraud, the core of skepticism, are not science. Skepticism is a field in and of itself, very distinct from science. It includes scientific thinking and it benefits, as every field does, from the products of science, but it is not science.

This piece is poorly researched, weak, and reads like a book report that someone started, put away, then suddenly realized it was due and wrote the rest while the other kids were watching a film in class. A big part of that is the fact that Watson simply does not know the field, something she has demonstrated repeatedly. Yet, knowing her track record in this area, CFI decided to commission and publish this. The poor quality of the piece is a side effect of overconfidence coupled with a lack of expertise, but it further points to a huge drop in standards by SI Online. Not that SI hasn’t published some misses in the past, but this sad little piece is just one of many lesser-quality articles recently appearing there, including one in which the author describes party/county fair psychics as harmless fun.

So, am I saying journalists and other non-scientists (e.g., skeptics) should never write about science? No, I am not.

It is fine for non-experts to write on topics when they do so with great care. I cannot stress this enough.

A non-expert can do a great job when they do a proper amount of research by talking with experts (rather than spending a few minutes Googling and picking sentences out of abstracts that one believes supports one’s already-formed opinion), when they discuss experiments and studies accurately without omitting important details, when they properly credit the sources of ideas and opinions, when they follow what they find rather than start with a conclusion and attempt to support it, and when they refrain from stating their personal opinions as authoritative. Watson rarely appears to do any of those things when she writes about science. She simply writes and speaks with an air of confidence and that seems to be enough to make some people think that she is clever and knowledgeable.

Good science journalism allows the researchers’ voices to be heard, not the author’s. Think about that.

I have written before about the dangers and hypocrisy of speaking and writing on topics which require expertise one does not have (here, here, and here, for example–two of which are also about Rebecca Watson). It’s actually a topic that has received a lot of coverage, from Massimo Pigliucci’s talk at TAM8 to articles by Daniel Loxton (yes, I’ve linked to them both before and for good reason). I expect to see this kind of thing all over the blogosphere, but to see it on the Skeptical Inquirer’s site is disheartening, especially on the heels of other pieces that fall far below their old standards.

As I stated in this post more than three years ago, we (skeptics in general) criticize Jenny McCarthy and Bill Maher because they don’t have the expertise to make the statements they make. We criticize “the Food Babe” and many, many others for the same reasons. We tell people not to take medical advice from a Playboy Bunny and a talk show host, yet we (skeptics again) give a microphone to a blogger to talk about the psychology of vaccine denial simply because she calls herself “Skepchick”? How is this justified?

Now, I am perfectly aware that many people don’t believe that psychology is a science or that expertise in the field is actually a thing. I deal with that kind of anti-intellectualism every day. But I am still stunned when I see such blatant disregard for it among people and organizations who wave the flag of “listen to the experts” when it suits their purposes. CFI, you should be ashamed.

So this Good Morning America piece showed up in my Facebook feed the other day touting the sensational headline “Odd-Defying Babies Born 10:11 12/13/14″.

Now, I think it would be adorable to have a baby born on 10:11 12/13/14 (in America, of course. In Europe, that would be 10:11 13/12/14, which just doesn’t hold the same cuteness). Human beings love the symbolism that comes from conventions such as labeling and ordering. My good friend and fellow Skeptic, Ani Aharonian (Insight blogger and guest blogger here) was married on 5/8/13 for a reason (can you guess?). But “odds-defying”? No. These babies defied no odds.

Odds are a property of something that is yet to be. They are really only valuable as a means of predicting something and have no value after the event has occurred. This is a bit like post-hoc (after-the-fact) thinking about lottery outcomes. Did your next-door neighbor defy the odds when they won the lottery? Well, it depends on your perspective, especially in time.

A given individual has an extremely slim chance of winning the jackpot of a lottery. For a Powerball lottery, a single ticket has a one in 175 million chance of winning. However, if 525 million tickets are sold, there’s a pretty good chance that at three, much less one, will be winners. Is it “odds-defying” that your next-door neighbor won? Well, perhaps from your perspective, but not to a total stranger in another state.

Similarly, the question of predicting the odds that any given child is born at that time is quite different from the post-hoc consideration of the odds. The prediction actually changes whether you are predicting the birth from before or after conception. Clearly, if you are carrying a child whose due date is around 12/13/14, the chance are much, much higher than if you have yet to conceive (especially if it’s already April 2013). But nobody even thinks much about these things until after the fact, probably because they are more concern about having a healthy child and mother at the end of it all. Post-hoc, we are really not talking about odds anymore, so unless someone predicted a very different outcome for one of these births, there was no defying of odds.

Well, except this:

Approximately, on average, eight babies are born in the U.S. every minute, so there should be around eight babies in the U.S. right now with the “lucky” birth time of 10:11 12/13/14. There are, according to the article, only two. It’s entirely possible that the author missed some, but let’s say she didn’t. That’s an unusual event. Probably not *very* unusual, as I’m sure the number of children born each minute varies a great deal, but still quite different from what we would predict given what had happened in the past. It’s also a bit interesting that both were born in the mid-west–one in the city of Cleveland, Ohio and the other in the relatively small city of Billings, Montana. Cleveland is not a small town, but why are the biggest metropolitan areas such as Los Angeles and New York not represented? For that matter, they are both girls, yet averages tell us that one should be a boy. Probably because the variability in these events is very, very high and while averages are the best predictors we have, “best” isn’t always very good.

But the reason I wanted to talk about the piece wasn’t just the sensational headline. The contents are pretty eye-rolling, too.

“We knew she was going to be born today [Saturday], we just didn’t know it would be at 10:11 a.m.,” Campbell said in a statement. “Everyone is telling us we should play the lottery. We feel this is a lucky day and are excited to get family photos with Santa.”

It’s a number. It’s cute and all, but really, it’s just a number. It won’t help you win the lottery. It won’t help her succeed in life.

These conventions don’t care who you are. They don’t care where you live or what you ate for breakfast. So don’t complain about the lack of diversity in this sample, either. The universe is colorblind. Sort of.

Responses to requests, demands, and criticism in the blogosphere in recent months has prompted a great deal of discussion, most of it terribly unproductive. In fact, most of it has been downright silly – a childish back-and-forth which, to an outsider, might appear to be violent agreement. In other words, camps do not appear to disagree, in general, about foundational issues, yet the bloodshed continues. Need I provide examples? I don’t think so*.

I hate to harp on a point (I really do), but oversimplification and shallow treatment of issues appears to be at the source of so much of the animosity that I think that rational discussion could be had if a short checklist were followed which included keeping one’s mind open to the possibility the other person is not evil simply because they criticized something or failed to submit to demands.

I am short on time and not prepared to discuss “Atheism Plus” in detail at the moment, but the discussion of it provides an excellent example or two that I think provide some insight into how discussions devolve into battles.

First, there is a slippery slope involved which is accelerated by crowd behavior and by unproductive reactions to criticism. We may, for example, start with a civil discussion about whether or not gender disparity in local groups can be attributed to a barrage of unwanted sexual attention women may receive at meet-ups. A number of views will be expressed, some with comments about their own experiences:

Person A: “I don’t do that.”

Person B: “I’ve been groped at meet-ups and it made me feel powerless and alone.”

Person C: “That’s never happened to me.”

Person D: “I think we should ban people who do that kind of thing.”

Person E: “So, I can’t ask a woman out at a meet-up?”

Person F: “Wait, I go to meet-ups to meet men and I like it when they grab me. I can take care of myself and I don’t want that behavior banned.”

Person G: “I’m not going to attend meet-ups anymore if people think that groping is okay.”

…and so on.

None of these views should shut down discussion. The refusal to concede that one’s own view may not be “right” is what turns discussions like these into battles of wills. Note that the original talking point was simple and there are small steps away from it as people talk rather than listen or make assumptions about what was said rather than ask for clarification. Those small steps add up. One day, a woman casually asks that men put a little more thought into when and how they proposition women and a few months later dozens of people are painting everyone who doesn’t support a rather specific call to action as a misogynist or ‘gender traitor’ while some of those called misogynists and gender traitors have dismissed the original problem altogether. This helps no one.

Those promoting “A+” have painted critics with a broad brush; we are “haters” who are “against social justice”. A post by Greta Christina on the issue of inclusiveness provides some insight:

An atheist movement cannot be inclusive of atheist women… and also be inclusive of people who publicly call women ugly, fat, sluts, whores, cunts, and worse; who persistently harass them; who deliberately invade their privacy and make their personal information public; and/or who routinely threaten them with grisly violence, rape, and death.

An atheist movement cannot be inclusive of atheists of color… and also be inclusive of people who think people of color stay in religion because they’re just not good at critical thinking, who blame crime on dark-skinned immigrants, who think victims of racial profiling deserved it because they looked like thugs, and/or who tell people of color, “You’re pretty smart for a…”.

In addition to holding up the reprehensible behavior of a few trolls as representative of the community as a whole, these statements are so full of subtext that they cry out for scrutiny. There are clearly false dichotomies buried in there as many of the proponents of A+ and many of their readers have expressed the desire not simply exclude the asshats who “publicly call women ugly” or “who deliberately invade their privacy”, but also anyone who dares to question whether such things have actually happened in given situations.

As has been said many times, we should be charitable when someone’s meaning is not entirely clear – give them the benefit of the doubt when we have little evidence of malice. This requires empathy. It requires us to resist defensive reactions and reconsider our views when we realize that we have failed in that regard.

Greta also notes that to provide a safe space for people of color, they must exclude “people who think people of color stay in religion because they’re just not good at critical thinking”. I found this particularly interesting in light of the fact that the belief that everyone with faith in a deity of some sort is “not good at critical thinking” is a widespread view among atheists (and skeptics, unfortunately). PZ Myers, one of the founders/owners of FreeThoughtBlogs said  this of the religious in a debate a few months ago (one I urge you all to watch: http://youtu.be/ZsqqFpWh7m8 ): “There’s something wrong with their braaains!”

It may be that Greta meant to refer to those who claim that people of color are generally poor critical thinkers and this explains lower rates of atheism. However, the math does not add up. Try constructing a syllogism from these statements. The proportion of believers in the population of people of color is higher than the general population. Believers are poor critical thinkers. Therefore…

So, who is right? Well, neither is right. Or correct.

Out of curiousity, I watched a recording of a few people discussing “Atheism+” [A+]. Much of this particular discussion involved defending the approach of A+ and suggesting that critics are somehow against social justice in general. I won’t got through the entire discussion; many of the arguments were straw men, which are not relevant. However, many were based on unsupported assertions (assumptions) and that is directly relevant.

One of the participants, Debbie Goddard (of CFI On Campus) attempted to address real criticisms rather than discuss those straw men and from her comments the disagreements became more clear. At one point, Stephanie Zvan criticized skeptics for ignoring evidence, noting that “We have mountains of evidence that ‘treating people equally’ is not treating people equally.” Debbie clarified this by expressing her belief that “color-blindness” is wrong.

That is when I realized that what they are talking about here are legitimate and rational disagreements over how to approach social injustices.

Legitimate and rational disagreements. Meaning that neither view is so well-supported that they can claim to know what’s best.

Yet people attempting to discuss these things rationally have been vilified and views have polarized. And the people who were speaking in this recording were doing so with such certainty that they were “right” that they failed to see that legitimate and rational disagreement was even possible.

And this has happened with many on both sides of the issue with most of the ‘dust ups’ in the community. I think a lot of the problem lies in treating these topics as simple when, in fact, they are not. As Ron Lindsay stated in a recent post on A+:

Social justice is great. After all, who’s against social justice? It’s when one starts to fill in the details that disagreements arise.

And it’s the details that matter here.

There are some who argue that, because minorities are at a disadvantage due to a history of oppression, they require special protection in order to reach equality. There are others who argue that such protection is both unnecessary and racist/sexist/___ist in and of itself. And there is a full spectrum of positions in the grey area in between these two views. 

What Stephanie claimed is that science tells us that the first view is “right”. Her certainty in that conclusion is clear from the video. Yet, she is wrong – sort of.

There are three details that we should consider. I am going to ignore one which comes from that grey area because it is extremely complicated, and that is the question of whether equal opportunity or equal outcome should be the goal. In other words, what “equality” means [If you claim that the answer to that question is no-brainer, you are making my point]. The other two major issues are the evidence for the claim and the evidence which suggests the best courses of action to correct injustices, which is the whole reason for asking the question in the first place.

We all know that stereotypes exist and that racism, sexism, any-ism, are alive and well in our society. And there is plenty of evidence that implicit biases exist. In fact, they are impossible to eliminate. We favor people whom we view as “like us” in many different ways. Depending on one’s definition of “ingroup” in a given context, we favor those who fit it. However, we are capable of making choices and taking actions which render such favor powerless. We are capable of overcoming these biases just as we are capable of overcoming other cognitive biases. Not eliminating, overcoming.

So science tells us that we have implicit biases which require a special effort on our part to overcome. Stephanie is right, no?

Not so fast.

Science may be able to tell us if affirmative action has contributed to the huge reductions in racism and related outcomes which have occurred in recent decades, but it cannot tell us if affirmative action is a good idea today simply based on the knowledge that we need to make a conscious effort to overcome biases. Even the first question is difficult to assess confidently, but I suspect it can be done and I suspect that the answer will be, “Yes. Yes, it has.”

But this is an extremely complex issue and it is further complicated by the fact that we all have dog in the race. We all care about it because we all identify with one or more of those man made categories we sum up as the variable “race”.

My personal views about special protection are like most of my political views (this IS a political issue, after all): very centrist. I believe that we need to pay attention to things like gender parity if we are interested in decreasing it. I am not convinced, however, that quotas are entirely appropriate in all situations. And if you think that science has the answer to whether my views are “correct”, I challenge you to prove so.

And here’s where I say that in my view, both Stephanie and Debbie are wrong. What I won’t do is reject their views outright and wonder why they can’t just see the truth that I think is written in “mountains of evidence”. I won’t do that because, although I am confident in my own conclusions, I am open to the possibility that I am wrong about this very complex, emotionally-charged issue.

Why I think they are wrong:

The goal is not to place blame for disparities, but to reduce them. If the major source of disparity is discrimination, then the act of discriminating needs to be reduced. Science does provide us with information which is useful in efforts to reduce interracial and other inter-group tensions. What the evidence suggests is not the multiculturalism approach that Debbie believes is best, but what she rejected: color-blindness (and gender-blindness, etc.). Or perhaps a better term would be color-not-noticing, but that doesn’t roll of the tongue very well.

We all have multiple identities. I am a woman, a scientist, an educator, a skeptic, an activist, a blogger, etc. There are always people with whom I share some identities and not others. When the context focuses on a specific value or identity, those with whom I share that value or identity are part of my ingroup. Ingroup/outgroup classification changes with context, but some are more flexible than others.

Decades of applied research has failed to demonstrate that interracial tension in schools can be reduced by increasing discussions of cultural differences and celebrating diversity. This should not be surprising given the mountains of research that Stephanie mentioned about ingroup/outgroup mentality. Attention to differences increases that tension.

What reduces the tension? Focus on similarities, seeing people as part of the ingroup and ignoring the differences which are present in a given context. Reducing the amount of “othering” we engage in. The best way to do that is to focus on commonalities. For example, the work that Chris Stedman, author of a soon-to-be-release book entitled Faitheist: How an Atheist Found Common Ground with the Religious  does has been criticized by PZ Myers and others because it brings people of different religious affiliations (and none) together to work toward common prosocial goals. Just yesterday a group of interfaith activists (as they call themselves) spent the day picking up trash on a beach to make it safer and cleaner.

Am I suggesting that people suppress parts of themselves about which they are proud? Let me make this clear: Hell, no. 

If that is what you’re taking from this post, you need to look outside of yourself and try to see the bigger picture. What I am saying is that my gender identity should have zero bearing on whether I am hired for a job or asked to speak at a conference or viewed as a sexual object in a professional context. Does that mean that I should not be proud to be a woman? Of course not.

Interfaith work does not suggest that people ‘check their religion at the door’, either. The work benefits more than just the likelihood that they will accomplish common goals.  Working together exposes each participant to people with whom they both share ideology and differ in ideology. Focus on the common ideology reduces the tensions caused by differences in other views and that reduction spreads to the differences themselves.

For example, a 2009 Gallop Poll result which most will find unsurprising is that people are much, much less likely to oppose same-sex marriage if they know someone who is gay/lesbian. There are certainly problems with drawing causal conclusions from such a study, but the effect is large and the findings are consistent with many lines of research which converge.

As I stated before, this is a complex issue. You may completely disagree with my argument, but to dismiss it altogether would be ludicrous, not to mention closed-minded and, dare I say it?, anti-intellectual.

I prefer to be recognized for my work rather than patronized because I am female. You may not see the issues the way I do, but calling me a misogynist for that disagreement is not only outrageous, it’s insulting and wrong.

When you speak with such certainty about how right and moral you are in relation to your critics without considering the possibility that you may be missing a nuance or two, you cannot hold any sort of moral or intellectual high ground.

My purpose here is not to argue about the topic of social justice, but to make the point that certainty, particularly about moral questions, is something we all need to be careful about. Too much (more than what is warranted) and it gets in the way of rational discussion. Too much and it divides people when no division is necessary. Too much and it is counterproductive. Too much and it is not confidence; it’s arrogance.

NOTE: Before you start commenting that Atheism Plus is about “allowing these discussions” because nobody else will, let me remind you that nobody ever said that discussions about evidence were outside the scope of Skepticism (one of the primary reasons put forward for the founding of A+) just because they relate to issues of social justice. In fact, quite the opposite is true and I think that this post is a good example of how science and skepticism can be applied to those areas.

*For those not following the ‘rationalist’ blogosphere, I apologize for my lack of links to the incidents I mentioned here. Frankly, there are too many and it’s difficult to know where to start or to choose one link which clearly demonstrates what’s happened. It seems to me that one does not need the background information to understand the example, but I cannot tell for certain.

The highlight of TAM2012 was an easy pick. That does not mean that the talks were bad by any stretch of the imagination. In fact, despite what some felt was a scarcity of “big draw” speakers (e.g., high-profile science communicators like Neil deGrasse Tyson and Bill Nye or high-profile atheists such as Richard Dawkins), the talks were as excellent as always. This was no surprise to me, though, because I have come to expect that kind of quality from those in the line-up.  I could list the talks I particularly enjoyed, but that would be far too long a post and my Twitter feed recorded some of the highlights. Many will be also posted by the JREF in coming months.

The meeting was smaller than last year (~1200 vs. >1600), but this is a good turnout considering that last year the line-up included both Tyson and Nye. Sizable, also, despite the hubbub that led some people to ‘boycott’, the economy, the growing number of skeptic, secular, and atheism conferences offered each year, and (probably the biggest factor, but the one that everyone seems to forget) the fact that Comicon was held in San Diego the same weekend!

For my part, I was honored to participate in a discussion on the main stage on the Future of Skepticism with an impressive panel: SkeptiCamp creator Reed Esau, skeptical IT guru Tim Farley, and long-time activist Jamy Ian Swiss (moderated by D.J. Grothe). I also presented a workshop on skepticism in classroom settings for a third time, along with Matt Lowry,  and I would like to thank the wonderful panel of educators (Dale Roy, Dean Baird, Ani Aharonian, and Sachie Howard) who took the stage for a round table-style Q & A with only a couple of hours’ (or less) notice.

So, the weekend was a good one. And the video embedded here was the stand-out highlight of it. If you have read more than a post or two on this blog, it will be immediately clear to you why it was the highlight and why I found it important enough to urge you to watch it. I should also note that almost everyone I spoke with at TAM found this talk to be, far and above, the best of the weekend if not more. Please watch it before continuing.

Jamy spoke clearly about the difference between discussion of and battling over issues such as scope, definitions, and goals. What he hinted at, but did not say, is that discussion can only happen among those who are educated about those issues (or those who are trying to educate themselves about them). With a few exceptions, it is usually when people who do not fully understand the nature of what we do insist on being allowed to redefine our work that distinctions become battle lines.

One sign that someone does not fully understand scientific skepticism is something Jamy hit hard – that skepticism, secularism, and atheism are different things. When we all understand this (good fences), we can identify our common goals and work together (good neighbors). The differences are complex, but as Jamy noted, we have general rules for practical purposes that allow us to operate while the philosophical discussions can continue among those interested. However, shallow treatment of the issues (or outright dismissal of the ‘rules’) is an ironic form of anti-intellectualism.

When Elizabeth Cornwell’s TAM2012 talk is posted, I hope you will revisit this post. She discusses the characteristics and behaviors of cyberbullies and it should be clear how it fits here. You might notice the enormous overlap in the sets of people who conflate atheism/skepticism and those who argue for verbal aggression (A.K.A., bullying and ridicule) as a means of outreach (and, apparently, as a general communication style).  It does not need to be this way.

I attended a couple of workshops on Thursday and one was interesting as well as relevant. “Coalition Building for the Skeptical Activist” was lead by the most qualified person I can think of to lead such a thing, Doubtful News founder Sharon Hill. Also on the panel were Kitty Mervine, whose website helps those who believe that they were abducted by aliens connect with other possible abductees and learn about alternative explanations for their experiences, Chris Stedman, an interfaith activist and author of the upcoming book “Faitheist“, American Atheists president Dave Silverman, and vice president of the Secular Coalition for America David Noise. What an interesting combination.

Sharon, Kitty, and Chris are all known for their bridge-building style. Chris’s efforts center around coalitions with diverse groups to work toward common goals. Chris is not a skeptical activist, yet his work and ours overlap in several areas. Chris is the kind of “good neighbor” that Jamy discussed in his speech.

Silverman and Noise, on the other hand, seemed odd choices for a workshop on coalitions. Noise seemed to echo a lot of what Silverman said; he seemed more of an activist for atheism than secularism. During the panel, the language and content both Silverman and Noise provided was related to ingroup-outgroup thinking. They stopped short of discussing the kinds of militant strategies PZ Myers has talked about, but considering that Silverman describes his organization as the “Marines of the Freethought Movement”, it is cause for concern. I heard nothing about building coalitions from either of them, only unsupported assumptions and uncreative, brute-force solutions to problems.

On Thursday, before the main stage events even began, Silverman tweeted this:

Tabling at #tam2012. You can be a skeptic and you can be a theist. But if you’re both, you’re not very good at one of them.

— David Silverman (@MrAtheistPants) July 12, 2012

There are so many things wrong with this statement that it’s hard to know where to start, but I wanted to reply with this entire post. The next morning I gave in to temptation and tweeted:

Pondering #TAM2012 tweets. “If you believe X, you’re not good at skepticism” is poor skepticism (it’s Belief Bias; form of confirm. bias). — Barbara Drescher (@badrescher) July 13, 2012

While Silverman is not known for “waging war” with skeptics over where to draw lines, he has attempted to redefine skepticism (or perhaps simply shown his lack of understanding of it). Furthermore, this kind of insult (which, I will note once again, comes from a place of ignorance) to skeptics who are not atheists does not even remotely resemble an attempt at discussion. Neither did his reply to Kylie Sturgess when she dared to disagree:

#thetruthhurts “@kyliesturgess: Couldn’t disagree MORE: You can be a skeptic & a theist. If you’re both, you’re not good at one of them.”

— David Silverman (@MrAtheistPants) July 13, 2012

‘Infographics’ seem to be the hot thing lately and they really, really bother me. I am usually fine with funny ones, but too often they portray a warped view of the world which is designed for the advancement of an agenda. I may even agree with that agenda, but whenever I see summations with percentages and shocking titles, my skeptical senses tingle.

My example is not quite an ‘infographic’, but the problems are the same: where do the numbers come from and do they mean what they appear to mean? Campaigns rely on the fact that people, in general, are cognitive misers. We generally will not go out of their way to analyze information, especially if it speaks to our world view.

Today a friend posted this on Facebook with the comment that we should be able to trust the data because the source is UNICEF. As usual, the headline itself is grossly misleading, but this is not apparent unless you click through it. I did and found myself on another non-UNICEF page which included more details and a link to the UNICEF press release.

First, let me make the point that the accuracy of data are not usually the biggest problem. Yes, people make up stuff and that stuff gets quoted, etc., but you don’t need to make up numbers to mislead people. How the data are manipulated and framed are the more common problems with these kinds of reports. Looking at the report that my friend posted, although data may be accurate, the frame is questionable and should be insulting to someone who is actually living in poverty. It is not, in my opinion, a measure of the proportion of children “living in poverty”.

I was prepared to take on the information in the press release, but when I read it I discovered that there was a layer of ‘warping’ between UNICEF and the other reports. The release describes a combination of two measures of ‘poverty’. Of course, one needs to download and at least skim the full report to get the big picture, and who is going to do that? Well, I will, of course, but first let me address the report that was posted to my friend’s page.

Image provided in the UNICEF press kit for their "Report Card" on poverty.

This report cherry-picked one of the measures – the one they could most easily use to twist into an image of the U.S. as not-so-child-friendly.

I take issue with such manipulation. Science is too easily abused for the purposes of selling something, even with good intentions. Although evoking sympathy may prompt people to act, it also warps our views of reality. If we cannot view information objectively, we cannot make the best decisions about how to use the resources at our disposal to meet our goals. In the end, everyone loses.

The definition of ‘poverty’ used in this situation is “living in a household in which disposable income, when adjusted for family size and composition, is less than 50% of the national median income”. In other words, they took the income for each household in the country, subtracted an estimate of the cost of basic expenses and adjusted it for family size/structure to determine ‘disposable income’ (more on that later), divided all of the households in half according to the resulting value, then counted the children in each group. They did this for each country separately. Then they ranked the countries accordingly.

There was no standard for ‘poverty’ applied to all countries, so the comparison is severely limited in terms of what it can tell us. The UNICEF report notes a number of justifications for their choices, mostly related the problems associated with alternative measures. I agree with many of their notes about other measures, but that does not solve the problems associated with this measure. In addition, their methodology for determining disposable income and adjusting for family size and structure was very questionable and involved a complicated formula that I will not even attempt to explain.

They found that, in the U.S.A., just over 23% of the children live in households in the bottom half. Frankly, I was surprised by this number. I thought that it would be much higher. It means that more than 3 out of 4 children live in households with an above-average amount of disposable income!

The comparison states that there are 34 other countries in which the proportion of children in the top half of the country’s households in terms of disposable income is higher than the proportion in the U.S.

Poverty, in this analysis, does not mean what the dictionary or Wikipedia say it means (“…an economic condition of lacking both money and basic necessities needed to successfully live, such as food, water, education, healthcare, and shelter.”), yet people will still think of it as such.

This is poor perspective.

The report provided justifications for the 50% threshold, however, I cannot help but be reminded of how often someone is shocked when they hear a report that 50% of people are below average in some desirable measure. It seems to me that no matter how much is done to improve the lives of those children, we will always have a ‘bottom half’. This is one of the problems with framing things in relative terms.

Of course we want all children to prosper, but just as we want everyone to have an above-average IQ, we cannot achieve such a thing when we define our goals in relative terms. The only way to increase the number of children in the upper half of that distribution is to increase the number of adults in the lower half. That’s tough when you consider that adults need to care for children, so nobody who cares for children can be in the lower half, either.

Perhaps that bottom half should be made up entirely of childless, middle-aged people? Or the elderly?

As the press release noted, the UNICEF includes two measures of ‘poverty’. The other measure defined poverty in a standardized manner which was more consistent with the traditional definition. They listed 14 items which were considered essential and considered a child to be “living in poverty” if they lacked two or more of those items. However, this analysis was limited to European countries. So, there is nothing in the report that tells us the proportion of children in the U.S. who are living in poverty as it is traditionally defined. Nothing.

Furthermore, the original post and the one it linked to suggest that we don’t take care of our children. Programs like National School Lunch Program (established in 1946), Head Start, and even basic public assistance are designed to provide those basic necessities, yet receiving them does not affect whether they are considered “living in poverty” by the UNICEF definition, which considers reported income.

Whether or not you believe those programs are sufficient is not something I am even qualified to argue about and not the point. I think that efforts to promote social programs, something which requires some political maneuvering and framing, have redefined what “poverty” means. That may have helped sell those programs, but in the long run we need to readjust in order to see things the way they are. My point is that losing perspective is never a good thing. In the end, we need to see reality if we are to determine how best to distribute resources and services to achieve goals like reducing economic disparity.

When we talk about poverty, I would like to see a more nuanced approach. For example, the most common measure of need used by public schools is whether a child qualifies for the National School Lunch program. Children who are homeless are lumped in with kids whose parents may struggle to make ends meet, but have enough to eat, clothing, shoes, a roof over their heads, and even cell phones. I do not mean to minimize the problems of people who are barely scraping by, but when their problems are not distinguishable from those who are literally going without essentials, that is shameful.

I hope I grabbed your attention with that title, but do not expect to find the answer to that question here. What I am going to discuss today is a study that many people seem to think answers that question, but it doesn’t.

As I noted in my last post, the study I’ll be discussing was grossly misreported, starting with its press release. Since the study itself appears to be behind a pay wall for most people, I’ll describe as much detail as I can in a blog post as I discuss the study’s validity and findings of the study, published in the Journal of Social Psychological and Personality Science and titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.

But for those who are not at all interested in the research methods or a breakdown of why I rate the quality of the study the way I do, I will give you the the bottom line so you can skip the rest or only read the sections that interest you (I’ve used headings to make it easier).

Summary

I think that the findings will hold up to replication, despite some issues I have with the way they did a few things. Overall, the research quality is quite high.

The groups they compared did not include atheists, agnostics, believers, non-believers, highly religious, or any other label that you can throw at it. In the studies they used raw religiosity scores and made some comparisons of “higher” and “lower” using values from the distribution. In a sense, the compared those who scored in the lower half of the sample to those who scored in the upper half.

They found:

• Differences in prosocial behavior cannot be dismissed as due to political affiliation, socio-economic status, or other factors often held up as responsible.
• Religiosity is correlated with trait compassion; the more religious, the more compassionate.
• Trait compassion is related to prosocial behavior in general. This relationship is stronger in the less religious than in the more religious. This does not mean that the less religious are more compassionate (see number 1) or that the less religious are more prosocial. It just means that compassion is a bigger factor in prosocial behavior in the less religious.
• The findings of the first study can be interpreted one way that isn’t discussed in the paper: when the relationship between compassion and religiosity is accounted for, the more religious are not more prosocial than the less religious.
• The findings in the second study, which involved inducing feelings of compassion, were similar for generosity, except that the more religious were more prosocial even after accounting for compassion.
• The findings of the second study also included a different pattern when the prosocial behavior was giving to charity. Compassion induced more giving, but the effect was weak and did not differ much across religiosity. Religiosity had a significant affect on charity. This can be explained by the guidelines provided by many churches for how much of one’s salary one should give.
• In the third study, in which state compassion (how compassionate the individual felt at that time) was measured and the prosocial behavior measure involved real-world cash, religiosity was not related to either compassion or prosocial behavior.
• In the third study, state compassion was positively correlated with prosocial behavior, but the effect was greater in the less religious than in the more religious.

What the findings as a whole say to me, and what I believe the press report tried, but failed, to express, at least with convincing support: We do not need religion to be prosocial. We need compassion.

This is great news for secularists.

However, it doesn’t say anything negative about religion or the religious, nor does it provide anything that should make atheists feel superior. It just shows that one can be good without God; that motivations can come from other sources.

Now on to the details…

NOTE: to keep this as short as possible, I’ve included a lot of links to terms and demonstrations. Where I describe problems in more detail I still water-down quite a bit. I will do my best to make it understandable without rambling on and on, but keep in mind that it takes many years to learn enough about research design and statistics to understand why some of these are problematic. Furthermore, not all researchers will agree on the consequences of some of these problems. I am still learning this stuff myself (probably always will be learning).

The Study (description)

The article reports three studies, each related to the relationship between compassion and prosocial behavior in less-religious individuals. I have created graphs using the information in the paper, but in some cases I did not have exact numbers, so while the relationships are visually accurate, there are only values where I could use exact numbers.

Theoretical Foundation

The introduction discusses research which documents that religiosity is associated with prosocial behavior. Specifically, religious people give more and volunteer more than nonreligious people, over and above what they give to and do for religious organizations. The researchers note that the nonreligious do give; when we compare groups, we do so using averages. However, it may be that the motivations for prosocial behavior vary in a way that interacts with religiosity. In other words, the more religious among us may be motivated to prosocial behavior by one set of factors and the less motivated by another.

The researchers hypothesized that compassion is a more influential factor in prosocial behavior for the less religious than for the more religious among us.

Study 1

The first study examined the relationships among religiosity and traits of compassion and prosocial tendencies. What this basically means is that situational factors were not involved; traits are a matter of personality or attitude. For example, “trait anxiety” refers to how anxious a person is in general, while “state anxiety” refers to how anxious that same individual feels in a given situation.

This study involved analyzing data from a 2004 “survey”. I put that term in quotes because it usually refers to a set of questions that do not measure more than what is apparent at face value. Established measures of latent variables (variables which cannot be measured directly such as feelings and attitudes) are usually called an “inventory” or “scale” and we refer to them loosely as “measures”. In this case, the survey involved such measures and I want to make that clear.

The sample was comprised of 1337 participants and covariates (variables other than those of interest which could explain differences among the groups) of gender, political orientation, and education were included in the analysis. The variables of interest were religiosity, compassion, and prosocial behavior. Religious identity (identification with a specific religion or no religion) was also considered.

Results

Correlations

• Covariates had little impact on the results.
• Trait compassion was positively correlated with religiosity* and prosocial behavior. On average, the more compassionate the individual, the more religious they were and the more the more prosocial they were.
• The relationship between religiosity and prosocial behavior was marginally significant (statistically).

Hypothesis Test (See Figure 1)

Figure 1: Study 1 results. "Higher" and "lower" are defined here as 1 SD from the mean.

• A regression analysis revealed an interaction of religiosity and compassion on prosocial behavior. What this means: The effect of compassion on prosocial behavior differed among levels of religiosity.
• More specifically, the level of trait compassion affected prosocial behavior less as religiosity increased.
• There was also a main effect of compassion, but that was apparent in the correlational analysis.
• There was no main effect of religiosity on prosocial behavior. This is interesting, because they found a marginally significant correlation, but it does not mean the there are no difference in prosocial behavior. I would interpret these findings, when put together, as suggestive of little or no difference between the more religious and the less religious in prosocial behavior over and above the differences accounted for by compassion.

The authors discuss the findings a little differently, though, focusing on the differences in the way that compassion affected prosocial behavior (the interaction in the first hypothesis test result) and ignoring the way that the effect of religiosity disappeared when compassion was entered into the equation. It seems more interesting to me to treat compassion as the moderator. It also makes more sense in the end.

Study 2

This study was experimental in that the researchers manipulated state compassion. In other words, they induced feelings of compassion in half of the participants and compared the amount of prosocial behavior those participants engaged in to the amount of such behavior in a control condition.

The sample included 101 participants and the study was conducted online, so the age range was exceptional (from 18 to 68 years). Participants were randomly assigned to one of the two conditions and each watched a short video under the guise that there would be a test of memory afterward.  The videos were established manipulations of feelings of compassion and neutral emotion (i.e., other researchers tested their effectiveness). Following the video, participants completed two tasks which are well-established measures of prosocial behavior commonly used in such research.

Results

• Again, covariates had little impact on the results.

Hypothesis Tests (See Figure 2)

Figure 3: Results of Study 3. Values are relative.

There were two tests since the participants completed to different prosocial tasks, one involving generosity and the other involving charity.

For the generosity task:

• This time there were a main effects of both religiosity and compassion on prosocial behavior. The more religious, the more prosocial. Those who watched the compassion-inducing video were the more prosocial on average than those who watched the neutral video.
• The interaction appeared again in the manner as in Study 1.

For the charity task:

• There were main effects of both religiosity and compassion on prosocial behavior.
• There was no interaction.

This is where they screw up, in my opinion.

The pattern of the moderation was in the predicted direction but failed to reach statistical significance.

This is not an acceptable statement unless the findings are marginal. This was not. The p-value was .408. This is not even close to meaningful. Still, they went ahead with the analysis of the interaction and reported an effect of compassion on charity for the less religious participants and no effect for the more religious.  The problem is that post-hoc analysis like this assumes that a significant interaction was observed. Their tests inflated alpha (the probability of a Type I Error) and can only mislead. They stated that they had found “partial support” for their hypothesis, but they did not in this case.

The relationships in the generosity task are very clear when we look at a Figure 2. The interaction is the interesting finding. Compassion had little effect on the more religious, but a very large effect on the less religious, who gave practically nothing when compassion was not induced. There is no analysis to tell us if the less religious surpassed the religious by a statistically significant amount when compassion was induced, but they were clearly out done by the more religious when not made to feel compassion.

The charity task showed no such interaction and the authors did not include a graph of this effect that I could recreate, nor did they provide the information to make one.

Study 3

For this study, the sample of 120 completed a state compassion inventory (a measure of their feelings of general compassion at the moment) and a series of “economic tasks designed to measure their generosity, trust, trustworthiness, and motivation to reward others’ generosity.” What differed in this study, however, was that the ‘points’ they earned in these tasks could be exchanged for cash at the end of the study. Participants did not know how much cash, but they knew that the more points they earned, the more cash they would receive.

Results

Hypothesis Tests (See Figure 3)

The findings of this study were very different from the other two.

• State compassion was not related to religiosity.
• Religiosity was not related to prosocial behavior.
• There was an interaction of religiosity and compassion on prosocial behavior. The amount of compassion felt had more of an effect on the behavior the less religious than it did on the more religious.

The graph of these findings, a reproduction of their graph since they did not provide information to create one that would make more sense (to me anyway), is a bit misleading. The values are z-scores, so they are relative to one another and not actual values. What is interesting, though is how little the prosocial score varied in the more religious group and how that line barely dips below the mean value (represented by 0).

There is also a problem with the press release in that it makes the claim that the high state compassion/less religious group out-performed the others. There is no statistical analysis comparing the groups in that way, so this is a misstatement. We do not know if less religious individuals are more generous than more religious when motivated to act prosocially. We just know that they are more generous when motivated by compassion than when compassion is low.

The Study Overall

As I noted, my opinion of the studies as a whole is relatively high, but I do have some major criticisms. Some of the language makes me cringe (e.g., results are the product of statistical tests, so “We tested our results”…), but I have seen more and more of this as scientific reports in general have grown sloppier.  Study design and method is much more important, as is the quality of the reporting beyond language.

The authors also throw around the term “robust”, claiming in the first study that the relationship between compassion and prosocial behavior is “particularly robust” for less religious individuals. That term refers to findings which are “sturdy” and will stand up when some supports are removed – effects which appear to hold up in different situations. Since this was one analysis of one data set, that term just doesn’t work. It does not fit in any of their uses of it.

In fact, they err in Study 2 by saying that the effect was “attenuated” for the more religious. That term is relative; attenuated compared to what? The effect was not “robust” in one condition and “attenuated” in another; they can only be compared to each other. The effect was greater in the less religious than the more religious.

Missing Information

There are a number of bits of information which are considered to be, at minimum, required for a good research report. A general rule of thumb for methods and results sections is to include enough (without being redundant) information to allow other researchers to replicate (in a strict sense) the study and to confirm that the statistical findings are properly interpreted.

I am not sure that this article meets that criterion. The methods are pretty well fleshed out and the paper is full of statistics, but some descriptive statistics are missing that I would have liked to have seen (e.g., means reported overall for measures, but not by group) and there was not enough of the right information to recreate them.

Grouping the Data and Errors of Generalization

One overall criticism which warrants discussion is in the grouping of data. There are some problems with this and they are related. The sensitivity of the religiosity measure is one problem that, by itself, is not a big target for criticism. Combined with the second problem of grouping participants, though, it becomes more serious.

The practice of comparing groups of people based on a variable which is distributed on a spectrum is a common one. The question the researcher wants to answer is important in deciding whether to group and, in this case, I do not disagree with that choice, but I question how they grouped and how it was communicated. If the data are clustered (the distribution is multi-modal), grouping is simplified, but if the data are distributed more loosely, it can be tricky and dangerous.

First, the researcher loses information, therefore they lose sensitivity and usually lose power. The sensitivity problem is relevant in the first study, but mostly because it makes the findings difficult to interpret.

Second, if the way that the grouping is communicated is not consistent and clear, it is likely to be misinterpreted, compounding any existing problems with the method. I discussed this problem in my last post. Most of the reports referred to the groups compared as “highly religious” verses “atheists and agnostics” or something like that. However, where are all of the people in the middle (i.e., most likely the bulk of the sample)?  Within each group there was variation in religiosity and comparisons are made using averages. Generalizing only works when the samples are representative of the population of interest and this applies in either direction of the generalization (i.e., specific to mixed or mixed to specific).

Third, researchers must decide where to draw the lines between high and low (and anything in between). Since the majority of variables in psychology are normal distributed (therefore symmetrical), the lines are usually drawn using rankings of sample values and the most common way to split a sample in half is to put all values above the median into “higher” and those below into “lower” (called a “median split”). However, ease is not a good reason to use this technique.   Here is an interesting demonstration of the dangers of dichotomizing normally-distributed variables.

But… religiosity is not usually distributed normally; it’s usually skewed. Skew means that it’s not symmetrical, so a median-split would make even less sense.

In this case, it seems that the authors tried to have the best of both worlds by treating religiosity as a continuous variable, but doing post-hoc analysis on it, discussing it, and graphing it as if it were dichotomous, choosing values which were one standard deviation from the mean in both directions as the central tendencies of each group. The biggest problem with this is the assumption of normality. If the variable is not normally-distributed (and I suspect that it is not), this grouping is a bit tough to swallow.

When this problem is mixed with a limited range as it is in the first study (the religiosity scale only had four points), it’s a problem. The four possible values were 1 = no religion, 2 = not very strong (religious identity), 3 = somewhat strong, and 4 = strong. Since the mean was 2.99, the bulk of the sample were fairly religious. one standard deviation (1.03) below the mean is not exactly in non-believerland and one above is off the scale (literally). It is just very difficult to see where “higher” leaves off and “lower” takes over.

Although the range is adequate in the other two studies, the problem of discussing groups which do not actually exist and have fuzzy definitions remains. In my opinion that is one of the reasons it was so misreported.

But, overall, the research is of a relatively high quality and interesting. I would like to see more variation in the prosocial tasks, given that the outcome of the charity task was so different from the tasks of generosity.

It seems that the less religious are at least as generous as the more religious, but their reasons for acting prosocially differ. I would like to see the day when, as a group, we are generous and prosocial consistently, without the need to be provoked and without needing to feel an emotional connection to the receiver.

Saslow, L., Willer, R., Feinberg, M., Piff, P., Clark, K., Keltner, D., & Saturn, S. (2012). My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals Social Psychological and Personality Science DOI: 10.1177/1948550612444137

I had promised myself that I would spend less time ranting about the problems of the activist community, but I was so disappointed and frustrated during a Twitter exchange with Melody Hensley (of CFI-DC, caveat: she was speaking for herself, not necessarily CFI) the other night that I felt it prudent to bring it up once again, or at least a part of it.

First, I want to address the tired complaint that traditional skeptics exclude “the god question”.

Yup, we do. But before you roll out the silly paragraphs in which you substitute “God” and “religion” with “ghosts” and “the paranormal”, understand this: we also don’t address “the ghost question”.

Or “the psychic question” or “the Bigfoot question” or “the angel question”.

Statements such as “There are no ghosts” with claims that this is more than a personal conclusion are not good scientific skepticism*. Neither is “All psychics are fakes”. Neither is “there is no God”.

I can’t tell you if ghosts exist. I can tell you that I don’t believe in ghosts. I can explain why I don’t believe in them. I can give you alternative explanations for the noises coming from your attic. I can discuss reasons that you might ‘feel’ that ghosts exist. But I cannot prove to you that there are no such thing as ghosts.

I can devise an experiment to show that your dog is not psychic, but I can’t prove that psychic energy doesn’t exist.

I can explain the mechanics of sleep paralysis and the nature of memory, but I can’t say for certain that aliens did not abduct you if you remove the testability of your claim by adding things like “they reset the clocks”.

I cannot prove that there is no dragon in your garage if it does not interact with the world in measurable ways. I can only say, “I am not convinced.”

What I personally believe about these things is irrelevant. It is poor skepticism, poor science, and poor reasoning to include my beliefs in a discussion of your claims. (NOTE: “Belief” is defined in my posts as “that which one holds to be true”.)

I will not speak for everyone who has “harped” about this issue, but I can tell you that this has always been my bottom line in these arguments, so those who would take it out of context and build straw men like “she says that religion is off-limits”, don’t bother.

What I really want to talk about is about here is why this isn’t good skepticism. I’d also like to refute the tired argument that only atheists are good skeptics.

Since there are several versions of this argument and I acknowledge that they carry different meanings, I am also arguing against the following claims:

• Only atheists are rational.
• Theists/Deists may be good skeptics when it comes to other areas, but they are not skeptical about religion.
• Agnostics and theists/deists do not ‘go far enough’.
• There are no reasons to believe in/is no evidence for the supernatural.

The problem with these claims is that they are based almost entirely on a conclusion – the conclusion that there is no god (atheism). It is human nature to judge the validity of arguments by the believability of the conclusion. For example, consider the following syllogisms and decide whether each is valid or invalid:

Some students are tired.
Some tired people are irritable.
Therefore, some students are irritable.

All dogs have four legs.
Daisy is a dog.
Therefore, Daisy has four legs.

If I study, I will get a good grade on the exam.
I got a good grade on the exam.
Therefore, I studied.

If you are like most of my students, you identified the first and third as valid, but the second as invalid. Unfortunately, the opposite is true. An argument is valid if and only if the conclusion logically follows from the premises. Validity is not truth. This is important, because none of us actually knows with 100% certainty what is and is not true.

When we assume that we know what is true, we fail to evaluate arguments on their own merits. If it we were wrong, we perpetuate and strengthen our misguided beliefs instead of discovering our errors.

To know how strong a conclusion is, we must examine two things: 1) the validity of the argument that produced it and 2) the strength of the premises.

The validity of the argument lies only in its logical progression, so we can evaluate this without going beyond what is presented. However, the strength of the premises is another matter. To evaluate those, we must consider their sources. In science, some are conclusions of other arguments (often previous research) and we must evaluate that research to know how strong the premise is. Others are a matter of observation, which is subjected to interpretation and induction. For example, in the famous syllogism about Socrates mortality, the strength of that conclusion relies on the assumption that the premise “All men are mortal” is accurate. Since not all men have died, we don’t actually know with 100% certainty that all men are mortal. We accept it based on a large number of observations and converging evidence, but certainty is not possible.

In my examples, the second syllogism is logically sound, but most people reject it because they “know” that not all dogs have four legs. Perhaps you’ve seen or heard of a three-legged dog (I met one once named “Tripod”) or you have knowledge of how it could occur. This does not make the argument invalid, but it does address #2, making the conclusion unsupported. We cannot determine if it is true from this argument.

The first and third arguments are invalid because the logic is unsound. We may “know” that the conclusions are true, but we can’t know that based on these arguments, so if we want to convince others we need to come up with better evidence/arguments.

The tendency to judge conclusions based on current beliefs is a product of how our brains evolved and developed – a side-effect of what makes us successful organisms. It is human nature, it is wrong and must be overcome if one is to be consistently rational (This, by the way, is a bit of a pipe dream, but I think it’s a good goal).

This problem pops up in a host of cognitive tasks and is a manifestation of the most influential of human frailties: the confirmation bias. This makes it extremely resistant to correction, especially in real-world contexts. In my experience, the concept of “validity” is difficult for many people to grasp because of this problem.

So, going back a few paragraphs, note that I wrote, “The problem with these claims is that they are based almost entirely on a conclusion – the conclusion that there is no god (atheism).”

Reason is about the validity of arguments, so judging a conclusion as valid or invalid without examining the argument is itself an irrational act. Without the argument, your only yardstick is your own belief about the truth of that conclusion. Although we have reasons for our beliefs, so do the people whose beliefs we’re evaluating. Everyone thinks that their beliefs are well-reasoned and accurate. That’s why they believe them!

If you find their conclusion unbelievable, then by all means, be skeptical, but to call it “irrational” without evaluating the argument is to say that you are 100% certain that there is no rational argument. That is the very definition of arrogance and it is not scientific.

Science does not tell us what is (true). Science tells us what is likely (to be true) and, in most cases, how likely. It does so by making arguments. Science is shared knowledge, not because it tells us facts, but because we can discuss the evidence and logic processes behind why we should be reasonably certain of many things. Although science is both a process and a set of knowledge (I’ll call them ‘facts’), the facts in that set are the products of the process. This may include negatives such as “my dog is not psychic” and “vaccines do not cause autism”, but testing is required to make such conclusions scientific.

Science is not about those facts, though. It’s about the process of discovery. When scientists make arguments (by publishing papers), they cite previous literature by noting the findings and, in some cases, describing how those findings were produced. They do not list facts; they discuss evidence.

Scientists don’t judge conclusions. Scientists judge arguments. Scientists look at the whole argument – the assumptions, evidence, and methodology that make up the premises as well as the logic that holds them together – and judge if the conclusions logically follow from those premises.

Likewise, scientific skepticism is about testing claims, examining evidence, and providing natural explanations for the evidence. If there is no evidence to examine, there is nothing to discuss.

Because science ignores untestable claims and because some scientists (e.g., Carl Sagan) have discussed the reasonability of belief without evidence, many people oversimplify the issue (as Melody did in this Twitter conversation) by making the statement that belief in God is within the scope of scientific skepticism because “You don’t believe something without scientific evidence”.

This is misguided for several reasons, but the two main reasons are:

1. The assumption that most or all religious belief is completely blind faith is simply wrong. “Evidence”, scientific or otherwise, comes in all manner of forms. In a 1998 study conducted by the Skeptic Society, the most popular answers among believers for why they believe in God involved empirical evidence and/or reasoning (you can find this in Michael Shermer’s How We Believe). These people certainly think that they have good reasons to believe. How people interpret evidence varies a great deal. Most real-world questions are very complex and what seems an obvious conclusion to one person may seem ridiculous to another. Skepticism is about how we interpret evidence, how we reason, and how we consider alternative explanations, not about the conclusions we eventually draw.A good example of interpreting evidence is the study that prompted me to send a tweet to Melody in the first place. She’d first shared a link, then tweeted it again, this time directly to Daniel Loxton, editor of Junior Skeptic, with the comment “More good news about atheists that you might find hard to believe:” Only Melody knows what she meant by the comment, but when I read the study I was disappointed to find that it is not “good news about atheists” at all. I will include details in a post to follow shortly, but the gist is that article’s spin is a very loose interpretation of the findings that fails to mention some sobering facts. Its overgeneralizations and assumptions are criminal. For example, the study reported doesn’t involve atheists. Researchers measured religiosity, then divided participants in half (“more religious” and “less religious”), a common practice in social psychology. Sure, there were atheists among the “less religious”, but we don’t draw conclusions about a part of a sample. Furthermore, taking the three studies as a whole, the ‘more religious’ were more compassionate and more prosocial than the ‘less religious’ half. In essence, anyone wanting to spin the findings another way could easily do so by noting that the less religious half onlyacted prosocially when moved to do so by compassion, whereas the more religious were consistently prosocial. This is not a finding that atheists are better people and, quite frankly, I am sick of people trying to prove such nonsense. Promoting the fact that one can be good without God does not require atheists to be morally superior and, as this study shows, it is a good thing that it doesn’t.Again, we all think that our beliefs are well-reasoned, but what’s more interesting is that people tend to assume that those who disagree do so because either they “need to” (In that same Skeptic Society survey, the most popular reasons believers gave for other people’s beliefs in God involved the stereotype of comfort and meaning to life.) or they are not as rational.
2. Even if there was literally zero evidence, the “null hypothesis” argument is an oversimplification of a concept borrowed from statistical rules and applied to the assumption that science makes that every hypothesis is testable. Science makes this assumption, but it also acknowledges that the assumption could be wrong by excluding the possibility of 100% certainty and by limiting its scope to testable hypotheses. You cannot invoke science as an answer to claims it cannot test, nor can you claim that someone’s conclusion is wrong because science cannot test it; that’s circular reasoning unless you’re saying that science is the only way to know something. By that logic, people cannot be scientific thinkers and also have morals**. Science (shared knowledge) may ignore the supernatural, but people (personal knowledge) do not and cannot use scientific processes to examine every question and still manage to function in the world, so if you want to attack person knowledge as wrong, you’ll have to do better than “it’s not scientific”.

So, what about the other versions of this claim?

Are atheists more rational than believers? Probably on average, considering the other variables which are correlated with atheism. However, given how poorly all human beings are at reasoning, that isn’t saying much.

Is atheism rational? I can’t answer that. Atheism is a conclusion. Whether it’s a rational conclusion depends on why the individual drew that conclusion.

Is religion rational? Again, I can’t answer that and neither can you. It’s a conclusion. Whether it’s a rational conclusion depends on the reasoning of the individual and the evidence they considered.

From what I know about how human beings process information, I can see a great many valid arguments for the existence of God that would be perfectly rational. They’d have to have some extraordinarily well-supported premises in order to convince me, but lacking support for those premises won’t make them irrational. Reasoning well does not require convincing others.

As I’ve said twice now, everyone thinks that their conclusions are the ‘right’ conclusions. So what makes your conclusion (that there is no God) better than someone else’s?

If your answer is “mine is well-reasoned”, that’s not a comparison. See two sentences back.

To know that you are “right” and they are “wrong”, you actually have to examine their argument/evidence. If you haven’t examined their argument, then who are you to tell someone that they are irrational? Who are you to tell them that they have no evidence when you haven’t even bothered to ask them what their evidence is? This is exactly what you’re doing when you claim that any belief in a god is “irrational” or make a blanket statement about the intelligence or cognitive abilities of those with religious beliefs. It’s elitist, arrogant, bigoted wishful thinking.

You cannot judge an argument by its conclusion, no matter how unbelievable the conclusion seems to you.

Finally, the following is a list of things that I have NOT said. In fact, I do not believe that any of the people accused most of making these statements actually has:

• “Religion is off-limits in skepticism.” There are plenty of testable claims related to religion, but test the claims and discuss the evidence rather than attacking belief in them.
• “Stop talking about ‘the god question’.”I have no problems with debates about the existence of God. What I have a problem with is criticizing conclusions as rational or irrational without examining the argument that produced them and calling such criticisms “skepticism”.
• “Stop promoting secularism.” I am a strong advocate of secularism, but promoting atheism is, in my opinion, no different from promoting any other set of conclusions. Freedom from religion requires freedom of religion. Removing religion from government does not mean taking it away from its citizens.
• “Atheism is not the only valid conclusion of properly-applied skepticism.” I did not say that it was, either. Hopefully my discussion explains why that question is not relevant.

I had planned to ignore Melody’s accusations that my criticisms are ‘mean-spirited’, but I cannot do that, either. It certainly is mean-spirited to attack people, but that is not what I have done. You won’t find a personal comment about Melody here. If it is mean-spirited to address (or even attack) what people say and do, especially when one finds what they promote to be harmful, then our whole business is ‘mean-spirited’.

*Given that all of the major organizations in skepticism have adopted scientific skepticism, this is what I’m discussing. If you would like to argue that activism should go beyond scientific skepticism, please do so elsewhere. For that matter, this post is not even about limiting scope for practical and strategic purposes, which is an entirely different cup of tea.

**Yes, I am aware that Sam Harris claims that science can tell us what is moral, but so far his arguments fall far short.

NOTE TO WOULD-BE COMMENTERS: Please do not comment if you have not actually read (not skimmed, READ) the post. Also, before you write a comment about how Dawkins and others (i.e., persons whose credentials you think I should not question) argue that science refutes the existence of God or should include “the god question”, I recommend a thorough review those arguments (that means more than reading a couple of blog posts by bystanders or comment threads). Their treatment of the subject is much more considered than the oversimplification I’m addressing here and their arguments are not as shallow. Finally, if your plan is to quote from Sagan’s Demon-Haunted World, you might want to read the whole book first.

NOTE: this post also appears on the wonderful site about crazy coincidence, theoddsmustbecrazy.com.

Monty Hall

I have always been amused and intrigued by responses to “The Monty Hall Problem”, especially when I talk about it to audiences with a high concentration of engineers and mathematicians. If you are familiar with it, but you’ve always struggled with an unsettled feeling of “this can’t be right”, read further and let me know if my explanation of the solution helps to alleviate the discomfort. If you are not familiar, I guarantee you will give your brain a workout by reading on.

First posed to statisticians in 1975, “The Monty Hall Problem” is well-known among academics because it still sparks debate. Many seem to think that disagreements about its solution stem from issues in the clarity of the problem, but I contend that it really stems from human flaws in the way that we process information.

I often discuss this problem in statistics and cognitive psychology courses for several reasons. It is a great exercise in probability calculation and it can be used to teach basic mathematical modeling (and its purpose). An added benefit, since almost all of my students were psychology majors, is that it also illustrates a flaw in human cognition as well as a pattern of problem solving. Even a knowledgeable statistician feels the need to run simulations to see the solution in action. Even then, fully grasping the mechanisms behind the answer often requires brute force cognition.

In general, human beings have a very difficult time wrapping their brains around concepts of probability. It is much like a visual illusion; we know that the lines are parallel/the circles are the same size/there is no motion, but we can’t make our brains process it in a way that represents that reality. It’s just not how our visual system works. I hypothesize that one of the reasons that probability is such a difficult field for most people is that it involves theory and models, which are distinct from observations and we must represent them differently in our minds to properly deal with them. Applications of probability often involve switching gears from the realm of models to data or vice versa and this is where I think most mathematicians get side-swiped in The Monty Hall Problem.

The Poser

In essence, here’s the problem:

You are a contestant on Let’s Make a Deal!and Monty loves your creative costume (a teddy bear carrying a human doll), so he calls on you to make a deal. Monty says, “There are three doors – Door #1, Door #2, and Door #3. Pick one and you get to keep whatever is behind it.”You’ve seen the show (you weren’t just walking down Ventura Boulevard in a teddy bear costume for fun), so you know that it is highly likely that there is a coveted BRAND NEW CAR! behind one of those three doors. If you choose wrong, however, you might end up with an ostrich…

Everyone hopes for a car. Some get donkeys or other animals.

You choose Door #3.

Monty then says, “Let’s see what’s behind Door #1!” and the door opens to reveal one of the many consolation prizes (and product placements), a lifetime supply of Rice a la Roly.

Cool! You might get that car after all!

Well, the show was successful because the shell-game-huckster-style of Monty Hall rarely stopped there. In this case, he does what he often does, offers to let you switch from your first choice (Door #3) to the only remaining option, Door #2.

Should this woman switch?

Should you? Does it matter?

Not the Problem

Before I get into the solution, let me first deflect a common complaint from mathematicians. The most well-known version of the problem, from its Wikipedia entry:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1 [but the door is not opened], and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

This version does not specifically state the name of the show or indicate the way that game shows of its era worked. If you have never seen the television show (i.e., you are younger than 35), or any game show of its kind, let me explain. Monty is in control of almost everything that happens. The only thing “contestants” can do is make choices when Monty offers them. As you will see, they had more control over their odds of winning than once thought, but Monty manipulates some of the build-up by choosing which items to reveal at different steps in the game.

Unfortunately, many probability theorists and mathematicians took issue with the lack of clarity in the problem (context is important sometimes). This provides a face-saving ‘other version’ for the geeks who get it wrong the first time. But whenever I hear comments like, “Okay, given this version, that Monty knows where the car is.” I usually think, “Of COURSE he knows where the car is! There is no other way to play the game!” and wish that people were more able to accept that they are just as human as everyone else.

The problem itself is written clearly, though: it specifically states that a door without a car behind it is revealed before you are given the option to switch. If the situation was a fully-randomized, double-blind game (like “Deal or No Deal”), then the option to switch would not even be on the table if the car is behind the revealed door. There would be no problem in that case. Therefore, the problem is a question of whether you should switch in a controlled setting – one in which the only participant who doesn’t know the location of the big prize is you.

The issue of knowledge is a factor in our processing of the problem, but it’s not what Monty knows that matters. It’s what you (the subject of the problem) know.

So, let’s put that complaint behind us and get back to the problem.

The Answer, and How to See it for Yourself

Hopefully, if the problem is new to you, you’ve spent some time trying to solve it instead of going with your first gut feeling, which was probably, “It doesn’t matter.”

It does. You should switch.

If you don’t believe me, try running some simulations. You’ll have to run a lot in order to get a large enough sample to be certain to see the trend, but here are a few ways to do it:

• Use your favorite program (MATLAB, R, etc.). There is a good database of pre-written simulators for this here. I am partial to Excel myself, even though it’s a bit more cumbersome. I just don’t remember enough code to use another program.

• Use a web-based simulator. Do it at least a hundred times, choosing to switch for half of the trials, and keep a tally your results.

• Use a die to simulate the outcome, assigning 1-3 to “Door #2″ and 4-6 to “Door #3″ (e.g., if you roll a 5, Door #3 is the one with the car). Roll at least a hundred times, choosing to switch for half of the trials (before rolling!). Keep a tally of the results.

What you will see is that switching will result in winning a car in approximately 2/3rd of the trials while staying will only provide a win in 1/3rd of them.

I know what you’re thinking. “But, there are only two doors left, so it should be 50/50!”

Why it is so Difficult to Accept

Human cognitive development is an interesting process. We learn to interpret information from the environment very quickly so that we can respond to that environment, but learning to reason hypothetically takes more time. Even adults with scientific training have a difficult time separating the concept of variables (each has a set of possible values) and data (values which are known).

In practice, hypothetical situations are often conditional (e.g., “If A, then B”). We tend to use information about what is to reason about what could be. We do this because it often works, but it is one of the ways in which our brains can lead us astray. For example, given the premise, “If I study, I will get a good grade on the exam”, what is the most sound conclusion when presented with a good grade? The most common response is, “I must have studied”, but that is not sound. In this premise, studying provides a guarantee for a good grade, but there is no statement that studying is the only way to get a good grade. It does not, for example, read, “If and only if I study…”

In the case of the Monty Hall Problem, the probability of winning is set before you pick a door. No matter which door you choose, the probability is 1/3rd. This is because there is a 1/3rd probability that the car is behind the door you chose given the information you had when you chose it. In reality, the car is behind one of the doors, so the probability it is behind Door #2 is 100% if it is there and 0% if it is not there. Probability is not a useful way to discuss what is or what happened; it is a tool for predicting what is likely to be true/happen.

The new information provided by revealing a loser changes the circumstances and this where we get trapped in our representations of models and data, possibilities and facts.

You had a 1/3rd chance of winning because there were three, equally-likely locations to choose from. It seems as if cutting the choice down to two should change the odds of winning to 1/2. It seems that way because we are focused on the probability that a given piece of information is true (e.g., that the car is behind Door #1) and not the probability that an event will occur (e.g., that we will win the car). The probability that we will win the car relies on the number of possible states of reality. This, in turn, initially relies on the number of locations for the car. When the situation changes, we try to adjust probabilities based on possible locations (which has changed) rather than on the number of possible states of reality (which has not).

Basically, when Monty makes the second offer, the offer is to switch from the door we have (#3) to the door we don’t have (#1 or #2). It does not matter that only one of those doors is left; there is still only a 1/3rd chance that our door has the car and a 2/3rd chance that the set of the other two contains the car.

Getting Un-Stuck

If you change the way you represent the problem from the beginning, the solution might seem more reasonable. Specifically, instead of thinking in terms of assigning probabilities to doors, think in terms of assigning probabilities to outcomes: winning verses losing.

Three possible states of reality, each with one winner and two losers

Let’s go step by step…

Monty asks you to pick a door from three choices. Behind one of those doors is a car. There are three possible locations and it must be in one of them, so there are three possible states of reality.

You choose to bet on Door #3; there is a 1/3rd chance that you will win the car.

There is a 2/3rd chance that you will not win the car.

This would be true no matter which door you chose.

Monty reveals that one of the remaining doors is a loser. At least one will be a loser since there is only one winner and you can choose only one. The car, however, does not move. Even though there are only two locations left, so there are still three possible states of reality. What’s changed is that we now know more about each of those possible states (there are fewer locations for the car to be):

So, if we model the problem in terms of the probability of winning with Door #3, the model itself does not change after the losing door is revealed. What changes is that we would no longer want to choose that door, so it is no longer among our options. This leaves us with only two options: keep the door we have or switch to the remaining door. The odds of winning/losing with Door #3 have not changed, but eliminating an option allows us to make a better choice – switch.