Note: This post also appears on Insight, the official blog of the Skeptic Society.

A few months ago I wrote about the psychology of vaccine denial. In the post I discussed two publications, one of which (Nyhan, et al.) found:

Corrective information reduced misperceptions about the vaccine/autism link but nonetheless decreased intent to vaccinate among parents who had the least favorable attitudes toward vaccines. Moreover, images of children who have MMR and a narrative about a child who had measles actually increased beliefs in serious vaccine side effects.

None of the interventions increased parents’ intent to vaccinate.

Then, a couple of weeks ago, a friend sent me a link to this piece describing research which seems to contradict that finding. The authors (Horne, et al.) concluded that

…highlighting factual information about the dangers of communicable diseases can positively impact people’s attitudes to vaccination.

These two conclusions seem to contract each other. Which should we believe?

Many times this question comes down to the quality of the research. In this case, I believe these are both fairly well-designed studies. One, however, is more precise than the other in several ways. I believe that precision highlights the complexity of the issue as well as giving us a better idea of the direction that vaccine promotion should take.

Let’s look at the differences in sampling and method between the two studies.

The Horne study sampled 315 men and women. In the Nyhan study, the final sample was 1759 parents with children under the age of 18. In most research, 315 subjects is more than sufficient and more is not always better. The danger in larger samples is to find effects that are statistically significant, but not practically significant. However, when comparing conflicting findings, it is best to bet on the side of the larger sample.

Then there’s the question of limiting the study to parents. Although Horne compared parents to non-parents and found no significant differences in attitudes or effects, noise is noise. These two groups of people vary, and the attitudes of non-parents are not particularly relevant. Limiting the study to parents would give me more confidence in the robustness of the findings and their application in real-world activism.

Still, if both are reasonably well-designed studies by competent researchers, the end results should not contradict each other. So there must be more going on. And there is.

For one thing, this is a great example of how complex social sciences are. We should never make policy decisions based on a single study and this demonstrates why. Replication, especially with variants of measures and materials, is essential to learning the best methods of persuasion.

For another, these studies differ in more than just sampling techniques. The Horne study is much simpler and, in fact, oversimplifies. Nyhan, et al. included three outcome measures, each addressing a specific attitude:

1. The belief that vaccines cause autism.
2. Perceived risk of side effects from vaccines.
3. Intent to vaccinate one’s child/children.

By contrast, the Horne study involved a single measure which combined answers to five specific questions (such as “I intend to vaccinate my child.” and “Doctors would not recommend vaccines if they were unsafe.” to come up with a more vague “vaccine attitudes” scale. Even if the answers to these questions are highly correlated, how interventions affect those answers may be very different. They certainly were in the Nyhan study. And if “effective” is defined as increasing intent to vaccinate, then the Horne study does not answer the question it purports to answer. Personally, I am more interested in intent to vaccinate than I am in any other aspect of “vaccine attitudes”, so the Nyhan study’s findings are much more meaningful to me.

In general, it is best to measure outcomes of interest as specifically as possible, but of course the more outcomes a researcher studies, the larger the sample must be.

Finally, and perhaps the most important difference between these two studies, is the timing of the experimental portion. When measuring the effect of treatments or interventions on attitudes, an experiment should be spaced over time. A researcher will measure the attitude, then wait before applying a treatment and measuring the attitude again. When polled about attitudes, those attitudes are brought to mind. This affects our receptiveness to relevant information in complex ways, ways that vary based on a number of other factors such as the strengths of our attitudes and the way the questions are worded. However, allowing subjects to forget about the initial survey provides a more accurate picture of how people confronted with information in the real world may respond to it.

The Horne experiment was conducted a day after the initial screening while the Nyhan experiment occurred about two weeks after initial screening.

My conclusion? I think the issue is complex, but while Horne’s findings appear easier to understand, Nyhan’s findings are more specific, answer more interesting questions, and can be more easily viewed within the framework of well-established knowledge about human decision-making (e.g., cognitive dissonance).

That, and we need more research if we are to develop effective ways of increasing vaccination rates.

Horne Z, Powell D, Hummel JE, & Holyoak KJ (2015). Countering antivaccination attitudes. Proceedings of the National Academy of Sciences of the United States of America, 112 (33), 10321-4 PMID: 26240325

Nyhan B, Reifler J, Richey S, & Freed GL (2014). Effective messages in vaccine promotion: a randomized trial. Pediatrics, 133 (4) PMID: 24590751

Responses to requests, demands, and criticism in the blogosphere in recent months has prompted a great deal of discussion, most of it terribly unproductive. In fact, most of it has been downright silly – a childish back-and-forth which, to an outsider, might appear to be violent agreement. In other words, camps do not appear to disagree, in general, about foundational issues, yet the bloodshed continues. Need I provide examples? I don’t think so*.

I hate to harp on a point (I really do), but oversimplification and shallow treatment of issues appears to be at the source of so much of the animosity that I think that rational discussion could be had if a short checklist were followed which included keeping one’s mind open to the possibility the other person is not evil simply because they criticized something or failed to submit to demands.

I am short on time and not prepared to discuss “Atheism Plus” in detail at the moment, but the discussion of it provides an excellent example or two that I think provide some insight into how discussions devolve into battles.

First, there is a slippery slope involved which is accelerated by crowd behavior and by unproductive reactions to criticism. We may, for example, start with a civil discussion about whether or not gender disparity in local groups can be attributed to a barrage of unwanted sexual attention women may receive at meet-ups. A number of views will be expressed, some with comments about their own experiences:

Person A: “I don’t do that.”

Person B: “I’ve been groped at meet-ups and it made me feel powerless and alone.”

Person C: “That’s never happened to me.”

Person D: “I think we should ban people who do that kind of thing.”

Person E: “So, I can’t ask a woman out at a meet-up?”

Person F: “Wait, I go to meet-ups to meet men and I like it when they grab me. I can take care of myself and I don’t want that behavior banned.”

Person G: “I’m not going to attend meet-ups anymore if people think that groping is okay.”

…and so on.

None of these views should shut down discussion. The refusal to concede that one’s own view may not be “right” is what turns discussions like these into battles of wills. Note that the original talking point was simple and there are small steps away from it as people talk rather than listen or make assumptions about what was said rather than ask for clarification. Those small steps add up. One day, a woman casually asks that men put a little more thought into when and how they proposition women and a few months later dozens of people are painting everyone who doesn’t support a rather specific call to action as a misogynist or ‘gender traitor’ while some of those called misogynists and gender traitors have dismissed the original problem altogether. This helps no one.

Those promoting “A+” have painted critics with a broad brush; we are “haters” who are “against social justice”. A post by Greta Christina on the issue of inclusiveness provides some insight:

An atheist movement cannot be inclusive of atheist women… and also be inclusive of people who publicly call women ugly, fat, sluts, whores, cunts, and worse; who persistently harass them; who deliberately invade their privacy and make their personal information public; and/or who routinely threaten them with grisly violence, rape, and death.

An atheist movement cannot be inclusive of atheists of color… and also be inclusive of people who think people of color stay in religion because they’re just not good at critical thinking, who blame crime on dark-skinned immigrants, who think victims of racial profiling deserved it because they looked like thugs, and/or who tell people of color, “You’re pretty smart for a…”.

In addition to holding up the reprehensible behavior of a few trolls as representative of the community as a whole, these statements are so full of subtext that they cry out for scrutiny. There are clearly false dichotomies buried in there as many of the proponents of A+ and many of their readers have expressed the desire not simply exclude the asshats who “publicly call women ugly” or “who deliberately invade their privacy”, but also anyone who dares to question whether such things have actually happened in given situations.

As has been said many times, we should be charitable when someone’s meaning is not entirely clear – give them the benefit of the doubt when we have little evidence of malice. This requires empathy. It requires us to resist defensive reactions and reconsider our views when we realize that we have failed in that regard.

Greta also notes that to provide a safe space for people of color, they must exclude “people who think people of color stay in religion because they’re just not good at critical thinking”. I found this particularly interesting in light of the fact that the belief that everyone with faith in a deity of some sort is “not good at critical thinking” is a widespread view among atheists (and skeptics, unfortunately). PZ Myers, one of the founders/owners of FreeThoughtBlogs said  this of the religious in a debate a few months ago (one I urge you all to watch: http://youtu.be/ZsqqFpWh7m8 ): “There’s something wrong with their braaains!”

It may be that Greta meant to refer to those who claim that people of color are generally poor critical thinkers and this explains lower rates of atheism. However, the math does not add up. Try constructing a syllogism from these statements. The proportion of believers in the population of people of color is higher than the general population. Believers are poor critical thinkers. Therefore…

So, who is right? Well, neither is right. Or correct.

Out of curiousity, I watched a recording of a few people discussing “Atheism+” [A+]. Much of this particular discussion involved defending the approach of A+ and suggesting that critics are somehow against social justice in general. I won’t got through the entire discussion; many of the arguments were straw men, which are not relevant. However, many were based on unsupported assertions (assumptions) and that is directly relevant.

One of the participants, Debbie Goddard (of CFI On Campus) attempted to address real criticisms rather than discuss those straw men and from her comments the disagreements became more clear. At one point, Stephanie Zvan criticized skeptics for ignoring evidence, noting that “We have mountains of evidence that ‘treating people equally’ is not treating people equally.” Debbie clarified this by expressing her belief that “color-blindness” is wrong.

That is when I realized that what they are talking about here are legitimate and rational disagreements over how to approach social injustices.

Legitimate and rational disagreements. Meaning that neither view is so well-supported that they can claim to know what’s best.

Yet people attempting to discuss these things rationally have been vilified and views have polarized. And the people who were speaking in this recording were doing so with such certainty that they were “right” that they failed to see that legitimate and rational disagreement was even possible.

And this has happened with many on both sides of the issue with most of the ‘dust ups’ in the community. I think a lot of the problem lies in treating these topics as simple when, in fact, they are not. As Ron Lindsay stated in a recent post on A+:

Social justice is great. After all, who’s against social justice? It’s when one starts to fill in the details that disagreements arise.

And it’s the details that matter here.

There are some who argue that, because minorities are at a disadvantage due to a history of oppression, they require special protection in order to reach equality. There are others who argue that such protection is both unnecessary and racist/sexist/___ist in and of itself. And there is a full spectrum of positions in the grey area in between these two views. 

What Stephanie claimed is that science tells us that the first view is “right”. Her certainty in that conclusion is clear from the video. Yet, she is wrong – sort of.

There are three details that we should consider. I am going to ignore one which comes from that grey area because it is extremely complicated, and that is the question of whether equal opportunity or equal outcome should be the goal. In other words, what “equality” means [If you claim that the answer to that question is no-brainer, you are making my point]. The other two major issues are the evidence for the claim and the evidence which suggests the best courses of action to correct injustices, which is the whole reason for asking the question in the first place.

We all know that stereotypes exist and that racism, sexism, any-ism, are alive and well in our society. And there is plenty of evidence that implicit biases exist. In fact, they are impossible to eliminate. We favor people whom we view as “like us” in many different ways. Depending on one’s definition of “ingroup” in a given context, we favor those who fit it. However, we are capable of making choices and taking actions which render such favor powerless. We are capable of overcoming these biases just as we are capable of overcoming other cognitive biases. Not eliminating, overcoming.

So science tells us that we have implicit biases which require a special effort on our part to overcome. Stephanie is right, no?

Not so fast.

Science may be able to tell us if affirmative action has contributed to the huge reductions in racism and related outcomes which have occurred in recent decades, but it cannot tell us if affirmative action is a good idea today simply based on the knowledge that we need to make a conscious effort to overcome biases. Even the first question is difficult to assess confidently, but I suspect it can be done and I suspect that the answer will be, “Yes. Yes, it has.”

But this is an extremely complex issue and it is further complicated by the fact that we all have dog in the race. We all care about it because we all identify with one or more of those man made categories we sum up as the variable “race”.

My personal views about special protection are like most of my political views (this IS a political issue, after all): very centrist. I believe that we need to pay attention to things like gender parity if we are interested in decreasing it. I am not convinced, however, that quotas are entirely appropriate in all situations. And if you think that science has the answer to whether my views are “correct”, I challenge you to prove so.

And here’s where I say that in my view, both Stephanie and Debbie are wrong. What I won’t do is reject their views outright and wonder why they can’t just see the truth that I think is written in “mountains of evidence”. I won’t do that because, although I am confident in my own conclusions, I am open to the possibility that I am wrong about this very complex, emotionally-charged issue.

Why I think they are wrong:

The goal is not to place blame for disparities, but to reduce them. If the major source of disparity is discrimination, then the act of discriminating needs to be reduced. Science does provide us with information which is useful in efforts to reduce interracial and other inter-group tensions. What the evidence suggests is not the multiculturalism approach that Debbie believes is best, but what she rejected: color-blindness (and gender-blindness, etc.). Or perhaps a better term would be color-not-noticing, but that doesn’t roll of the tongue very well.

We all have multiple identities. I am a woman, a scientist, an educator, a skeptic, an activist, a blogger, etc. There are always people with whom I share some identities and not others. When the context focuses on a specific value or identity, those with whom I share that value or identity are part of my ingroup. Ingroup/outgroup classification changes with context, but some are more flexible than others.

Decades of applied research has failed to demonstrate that interracial tension in schools can be reduced by increasing discussions of cultural differences and celebrating diversity. This should not be surprising given the mountains of research that Stephanie mentioned about ingroup/outgroup mentality. Attention to differences increases that tension.

What reduces the tension? Focus on similarities, seeing people as part of the ingroup and ignoring the differences which are present in a given context. Reducing the amount of “othering” we engage in. The best way to do that is to focus on commonalities. For example, the work that Chris Stedman, author of a soon-to-be-release book entitled Faitheist: How an Atheist Found Common Ground with the Religious  does has been criticized by PZ Myers and others because it brings people of different religious affiliations (and none) together to work toward common prosocial goals. Just yesterday a group of interfaith activists (as they call themselves) spent the day picking up trash on a beach to make it safer and cleaner.

Am I suggesting that people suppress parts of themselves about which they are proud? Let me make this clear: Hell, no. 

If that is what you’re taking from this post, you need to look outside of yourself and try to see the bigger picture. What I am saying is that my gender identity should have zero bearing on whether I am hired for a job or asked to speak at a conference or viewed as a sexual object in a professional context. Does that mean that I should not be proud to be a woman? Of course not.

Interfaith work does not suggest that people ‘check their religion at the door’, either. The work benefits more than just the likelihood that they will accomplish common goals.  Working together exposes each participant to people with whom they both share ideology and differ in ideology. Focus on the common ideology reduces the tensions caused by differences in other views and that reduction spreads to the differences themselves.

For example, a 2009 Gallop Poll result which most will find unsurprising is that people are much, much less likely to oppose same-sex marriage if they know someone who is gay/lesbian. There are certainly problems with drawing causal conclusions from such a study, but the effect is large and the findings are consistent with many lines of research which converge.

As I stated before, this is a complex issue. You may completely disagree with my argument, but to dismiss it altogether would be ludicrous, not to mention closed-minded and, dare I say it?, anti-intellectual.

I prefer to be recognized for my work rather than patronized because I am female. You may not see the issues the way I do, but calling me a misogynist for that disagreement is not only outrageous, it’s insulting and wrong.

When you speak with such certainty about how right and moral you are in relation to your critics without considering the possibility that you may be missing a nuance or two, you cannot hold any sort of moral or intellectual high ground.

My purpose here is not to argue about the topic of social justice, but to make the point that certainty, particularly about moral questions, is something we all need to be careful about. Too much (more than what is warranted) and it gets in the way of rational discussion. Too much and it divides people when no division is necessary. Too much and it is counterproductive. Too much and it is not confidence; it’s arrogance.

NOTE: Before you start commenting that Atheism Plus is about “allowing these discussions” because nobody else will, let me remind you that nobody ever said that discussions about evidence were outside the scope of Skepticism (one of the primary reasons put forward for the founding of A+) just because they relate to issues of social justice. In fact, quite the opposite is true and I think that this post is a good example of how science and skepticism can be applied to those areas.

*For those not following the ‘rationalist’ blogosphere, I apologize for my lack of links to the incidents I mentioned here. Frankly, there are too many and it’s difficult to know where to start or to choose one link which clearly demonstrates what’s happened. It seems to me that one does not need the background information to understand the example, but I cannot tell for certain.

The highlight of TAM2012 was an easy pick. That does not mean that the talks were bad by any stretch of the imagination. In fact, despite what some felt was a scarcity of “big draw” speakers (e.g., high-profile science communicators like Neil deGrasse Tyson and Bill Nye or high-profile atheists such as Richard Dawkins), the talks were as excellent as always. This was no surprise to me, though, because I have come to expect that kind of quality from those in the line-up.  I could list the talks I particularly enjoyed, but that would be far too long a post and my Twitter feed recorded some of the highlights. Many will be also posted by the JREF in coming months.

The meeting was smaller than last year (~1200 vs. >1600), but this is a good turnout considering that last year the line-up included both Tyson and Nye. Sizable, also, despite the hubbub that led some people to ‘boycott’, the economy, the growing number of skeptic, secular, and atheism conferences offered each year, and (probably the biggest factor, but the one that everyone seems to forget) the fact that Comicon was held in San Diego the same weekend!

For my part, I was honored to participate in a discussion on the main stage on the Future of Skepticism with an impressive panel: SkeptiCamp creator Reed Esau, skeptical IT guru Tim Farley, and long-time activist Jamy Ian Swiss (moderated by D.J. Grothe). I also presented a workshop on skepticism in classroom settings for a third time, along with Matt Lowry,  and I would like to thank the wonderful panel of educators (Dale Roy, Dean Baird, Ani Aharonian, and Sachie Howard) who took the stage for a round table-style Q & A with only a couple of hours’ (or less) notice.

So, the weekend was a good one. And the video embedded here was the stand-out highlight of it. If you have read more than a post or two on this blog, it will be immediately clear to you why it was the highlight and why I found it important enough to urge you to watch it. I should also note that almost everyone I spoke with at TAM found this talk to be, far and above, the best of the weekend if not more. Please watch it before continuing.

Jamy spoke clearly about the difference between discussion of and battling over issues such as scope, definitions, and goals. What he hinted at, but did not say, is that discussion can only happen among those who are educated about those issues (or those who are trying to educate themselves about them). With a few exceptions, it is usually when people who do not fully understand the nature of what we do insist on being allowed to redefine our work that distinctions become battle lines.

One sign that someone does not fully understand scientific skepticism is something Jamy hit hard – that skepticism, secularism, and atheism are different things. When we all understand this (good fences), we can identify our common goals and work together (good neighbors). The differences are complex, but as Jamy noted, we have general rules for practical purposes that allow us to operate while the philosophical discussions can continue among those interested. However, shallow treatment of the issues (or outright dismissal of the ‘rules’) is an ironic form of anti-intellectualism.

When Elizabeth Cornwell’s TAM2012 talk is posted, I hope you will revisit this post. She discusses the characteristics and behaviors of cyberbullies and it should be clear how it fits here. You might notice the enormous overlap in the sets of people who conflate atheism/skepticism and those who argue for verbal aggression (A.K.A., bullying and ridicule) as a means of outreach (and, apparently, as a general communication style).  It does not need to be this way.

I attended a couple of workshops on Thursday and one was interesting as well as relevant. “Coalition Building for the Skeptical Activist” was lead by the most qualified person I can think of to lead such a thing, Doubtful News founder Sharon Hill. Also on the panel were Kitty Mervine, whose website helps those who believe that they were abducted by aliens connect with other possible abductees and learn about alternative explanations for their experiences, Chris Stedman, an interfaith activist and author of the upcoming book “Faitheist“, American Atheists president Dave Silverman, and vice president of the Secular Coalition for America David Noise. What an interesting combination.

Sharon, Kitty, and Chris are all known for their bridge-building style. Chris’s efforts center around coalitions with diverse groups to work toward common goals. Chris is not a skeptical activist, yet his work and ours overlap in several areas. Chris is the kind of “good neighbor” that Jamy discussed in his speech.

Silverman and Noise, on the other hand, seemed odd choices for a workshop on coalitions. Noise seemed to echo a lot of what Silverman said; he seemed more of an activist for atheism than secularism. During the panel, the language and content both Silverman and Noise provided was related to ingroup-outgroup thinking. They stopped short of discussing the kinds of militant strategies PZ Myers has talked about, but considering that Silverman describes his organization as the “Marines of the Freethought Movement”, it is cause for concern. I heard nothing about building coalitions from either of them, only unsupported assumptions and uncreative, brute-force solutions to problems.

On Thursday, before the main stage events even began, Silverman tweeted this:

Tabling at #tam2012. You can be a skeptic and you can be a theist. But if you’re both, you’re not very good at one of them.

— David Silverman (@MrAtheistPants) July 12, 2012

There are so many things wrong with this statement that it’s hard to know where to start, but I wanted to reply with this entire post. The next morning I gave in to temptation and tweeted:

Pondering #TAM2012 tweets. “If you believe X, you’re not good at skepticism” is poor skepticism (it’s Belief Bias; form of confirm. bias). — Barbara Drescher (@badrescher) July 13, 2012

While Silverman is not known for “waging war” with skeptics over where to draw lines, he has attempted to redefine skepticism (or perhaps simply shown his lack of understanding of it). Furthermore, this kind of insult (which, I will note once again, comes from a place of ignorance) to skeptics who are not atheists does not even remotely resemble an attempt at discussion. Neither did his reply to Kylie Sturgess when she dared to disagree:

#thetruthhurts “@kyliesturgess: Couldn’t disagree MORE: You can be a skeptic & a theist. If you’re both, you’re not good at one of them.”

— David Silverman (@MrAtheistPants) July 13, 2012

‘Infographics’ seem to be the hot thing lately and they really, really bother me. I am usually fine with funny ones, but too often they portray a warped view of the world which is designed for the advancement of an agenda. I may even agree with that agenda, but whenever I see summations with percentages and shocking titles, my skeptical senses tingle.

My example is not quite an ‘infographic’, but the problems are the same: where do the numbers come from and do they mean what they appear to mean? Campaigns rely on the fact that people, in general, are cognitive misers. We generally will not go out of their way to analyze information, especially if it speaks to our world view.

Today a friend posted this on Facebook with the comment that we should be able to trust the data because the source is UNICEF. As usual, the headline itself is grossly misleading, but this is not apparent unless you click through it. I did and found myself on another non-UNICEF page which included more details and a link to the UNICEF press release.

First, let me make the point that the accuracy of data are not usually the biggest problem. Yes, people make up stuff and that stuff gets quoted, etc., but you don’t need to make up numbers to mislead people. How the data are manipulated and framed are the more common problems with these kinds of reports. Looking at the report that my friend posted, although data may be accurate, the frame is questionable and should be insulting to someone who is actually living in poverty. It is not, in my opinion, a measure of the proportion of children “living in poverty”.

I was prepared to take on the information in the press release, but when I read it I discovered that there was a layer of ‘warping’ between UNICEF and the other reports. The release describes a combination of two measures of ‘poverty’. Of course, one needs to download and at least skim the full report to get the big picture, and who is going to do that? Well, I will, of course, but first let me address the report that was posted to my friend’s page.

Image provided in the UNICEF press kit for their "Report Card" on poverty.

This report cherry-picked one of the measures – the one they could most easily use to twist into an image of the U.S. as not-so-child-friendly.

I take issue with such manipulation. Science is too easily abused for the purposes of selling something, even with good intentions. Although evoking sympathy may prompt people to act, it also warps our views of reality. If we cannot view information objectively, we cannot make the best decisions about how to use the resources at our disposal to meet our goals. In the end, everyone loses.

The definition of ‘poverty’ used in this situation is “living in a household in which disposable income, when adjusted for family size and composition, is less than 50% of the national median income”. In other words, they took the income for each household in the country, subtracted an estimate of the cost of basic expenses and adjusted it for family size/structure to determine ‘disposable income’ (more on that later), divided all of the households in half according to the resulting value, then counted the children in each group. They did this for each country separately. Then they ranked the countries accordingly.

There was no standard for ‘poverty’ applied to all countries, so the comparison is severely limited in terms of what it can tell us. The UNICEF report notes a number of justifications for their choices, mostly related the problems associated with alternative measures. I agree with many of their notes about other measures, but that does not solve the problems associated with this measure. In addition, their methodology for determining disposable income and adjusting for family size and structure was very questionable and involved a complicated formula that I will not even attempt to explain.

They found that, in the U.S.A., just over 23% of the children live in households in the bottom half. Frankly, I was surprised by this number. I thought that it would be much higher. It means that more than 3 out of 4 children live in households with an above-average amount of disposable income!

The comparison states that there are 34 other countries in which the proportion of children in the top half of the country’s households in terms of disposable income is higher than the proportion in the U.S.

Poverty, in this analysis, does not mean what the dictionary or Wikipedia say it means (“…an economic condition of lacking both money and basic necessities needed to successfully live, such as food, water, education, healthcare, and shelter.”), yet people will still think of it as such.

This is poor perspective.

The report provided justifications for the 50% threshold, however, I cannot help but be reminded of how often someone is shocked when they hear a report that 50% of people are below average in some desirable measure. It seems to me that no matter how much is done to improve the lives of those children, we will always have a ‘bottom half’. This is one of the problems with framing things in relative terms.

Of course we want all children to prosper, but just as we want everyone to have an above-average IQ, we cannot achieve such a thing when we define our goals in relative terms. The only way to increase the number of children in the upper half of that distribution is to increase the number of adults in the lower half. That’s tough when you consider that adults need to care for children, so nobody who cares for children can be in the lower half, either.

Perhaps that bottom half should be made up entirely of childless, middle-aged people? Or the elderly?

As the press release noted, the UNICEF includes two measures of ‘poverty’. The other measure defined poverty in a standardized manner which was more consistent with the traditional definition. They listed 14 items which were considered essential and considered a child to be “living in poverty” if they lacked two or more of those items. However, this analysis was limited to European countries. So, there is nothing in the report that tells us the proportion of children in the U.S. who are living in poverty as it is traditionally defined. Nothing.

Furthermore, the original post and the one it linked to suggest that we don’t take care of our children. Programs like National School Lunch Program (established in 1946), Head Start, and even basic public assistance are designed to provide those basic necessities, yet receiving them does not affect whether they are considered “living in poverty” by the UNICEF definition, which considers reported income.

Whether or not you believe those programs are sufficient is not something I am even qualified to argue about and not the point. I think that efforts to promote social programs, something which requires some political maneuvering and framing, have redefined what “poverty” means. That may have helped sell those programs, but in the long run we need to readjust in order to see things the way they are. My point is that losing perspective is never a good thing. In the end, we need to see reality if we are to determine how best to distribute resources and services to achieve goals like reducing economic disparity.

When we talk about poverty, I would like to see a more nuanced approach. For example, the most common measure of need used by public schools is whether a child qualifies for the National School Lunch program. Children who are homeless are lumped in with kids whose parents may struggle to make ends meet, but have enough to eat, clothing, shoes, a roof over their heads, and even cell phones. I do not mean to minimize the problems of people who are barely scraping by, but when their problems are not distinguishable from those who are literally going without essentials, that is shameful.

I hope I grabbed your attention with that title, but do not expect to find the answer to that question here. What I am going to discuss today is a study that many people seem to think answers that question, but it doesn’t.

As I noted in my last post, the study I’ll be discussing was grossly misreported, starting with its press release. Since the study itself appears to be behind a pay wall for most people, I’ll describe as much detail as I can in a blog post as I discuss the study’s validity and findings of the study, published in the Journal of Social Psychological and Personality Science and titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.

But for those who are not at all interested in the research methods or a breakdown of why I rate the quality of the study the way I do, I will give you the the bottom line so you can skip the rest or only read the sections that interest you (I’ve used headings to make it easier).

Summary

I think that the findings will hold up to replication, despite some issues I have with the way they did a few things. Overall, the research quality is quite high.

The groups they compared did not include atheists, agnostics, believers, non-believers, highly religious, or any other label that you can throw at it. In the studies they used raw religiosity scores and made some comparisons of “higher” and “lower” using values from the distribution. In a sense, the compared those who scored in the lower half of the sample to those who scored in the upper half.

They found:

• Differences in prosocial behavior cannot be dismissed as due to political affiliation, socio-economic status, or other factors often held up as responsible.
• Religiosity is correlated with trait compassion; the more religious, the more compassionate.
• Trait compassion is related to prosocial behavior in general. This relationship is stronger in the less religious than in the more religious. This does not mean that the less religious are more compassionate (see number 1) or that the less religious are more prosocial. It just means that compassion is a bigger factor in prosocial behavior in the less religious.
• The findings of the first study can be interpreted one way that isn’t discussed in the paper: when the relationship between compassion and religiosity is accounted for, the more religious are not more prosocial than the less religious.
• The findings in the second study, which involved inducing feelings of compassion, were similar for generosity, except that the more religious were more prosocial even after accounting for compassion.
• The findings of the second study also included a different pattern when the prosocial behavior was giving to charity. Compassion induced more giving, but the effect was weak and did not differ much across religiosity. Religiosity had a significant affect on charity. This can be explained by the guidelines provided by many churches for how much of one’s salary one should give.
• In the third study, in which state compassion (how compassionate the individual felt at that time) was measured and the prosocial behavior measure involved real-world cash, religiosity was not related to either compassion or prosocial behavior.
• In the third study, state compassion was positively correlated with prosocial behavior, but the effect was greater in the less religious than in the more religious.

What the findings as a whole say to me, and what I believe the press report tried, but failed, to express, at least with convincing support: We do not need religion to be prosocial. We need compassion.

This is great news for secularists.

However, it doesn’t say anything negative about religion or the religious, nor does it provide anything that should make atheists feel superior. It just shows that one can be good without God; that motivations can come from other sources.

Now on to the details…

NOTE: to keep this as short as possible, I’ve included a lot of links to terms and demonstrations. Where I describe problems in more detail I still water-down quite a bit. I will do my best to make it understandable without rambling on and on, but keep in mind that it takes many years to learn enough about research design and statistics to understand why some of these are problematic. Furthermore, not all researchers will agree on the consequences of some of these problems. I am still learning this stuff myself (probably always will be learning).

The Study (description)

The article reports three studies, each related to the relationship between compassion and prosocial behavior in less-religious individuals. I have created graphs using the information in the paper, but in some cases I did not have exact numbers, so while the relationships are visually accurate, there are only values where I could use exact numbers.

Theoretical Foundation

The introduction discusses research which documents that religiosity is associated with prosocial behavior. Specifically, religious people give more and volunteer more than nonreligious people, over and above what they give to and do for religious organizations. The researchers note that the nonreligious do give; when we compare groups, we do so using averages. However, it may be that the motivations for prosocial behavior vary in a way that interacts with religiosity. In other words, the more religious among us may be motivated to prosocial behavior by one set of factors and the less motivated by another.

The researchers hypothesized that compassion is a more influential factor in prosocial behavior for the less religious than for the more religious among us.

Study 1

The first study examined the relationships among religiosity and traits of compassion and prosocial tendencies. What this basically means is that situational factors were not involved; traits are a matter of personality or attitude. For example, “trait anxiety” refers to how anxious a person is in general, while “state anxiety” refers to how anxious that same individual feels in a given situation.

This study involved analyzing data from a 2004 “survey”. I put that term in quotes because it usually refers to a set of questions that do not measure more than what is apparent at face value. Established measures of latent variables (variables which cannot be measured directly such as feelings and attitudes) are usually called an “inventory” or “scale” and we refer to them loosely as “measures”. In this case, the survey involved such measures and I want to make that clear.

The sample was comprised of 1337 participants and covariates (variables other than those of interest which could explain differences among the groups) of gender, political orientation, and education were included in the analysis. The variables of interest were religiosity, compassion, and prosocial behavior. Religious identity (identification with a specific religion or no religion) was also considered.

Results

Correlations

• Covariates had little impact on the results.
• Trait compassion was positively correlated with religiosity* and prosocial behavior. On average, the more compassionate the individual, the more religious they were and the more the more prosocial they were.
• The relationship between religiosity and prosocial behavior was marginally significant (statistically).

Hypothesis Test (See Figure 1)

Figure 1: Study 1 results. "Higher" and "lower" are defined here as 1 SD from the mean.

• A regression analysis revealed an interaction of religiosity and compassion on prosocial behavior. What this means: The effect of compassion on prosocial behavior differed among levels of religiosity.
• More specifically, the level of trait compassion affected prosocial behavior less as religiosity increased.
• There was also a main effect of compassion, but that was apparent in the correlational analysis.
• There was no main effect of religiosity on prosocial behavior. This is interesting, because they found a marginally significant correlation, but it does not mean the there are no difference in prosocial behavior. I would interpret these findings, when put together, as suggestive of little or no difference between the more religious and the less religious in prosocial behavior over and above the differences accounted for by compassion.

The authors discuss the findings a little differently, though, focusing on the differences in the way that compassion affected prosocial behavior (the interaction in the first hypothesis test result) and ignoring the way that the effect of religiosity disappeared when compassion was entered into the equation. It seems more interesting to me to treat compassion as the moderator. It also makes more sense in the end.

Study 2

This study was experimental in that the researchers manipulated state compassion. In other words, they induced feelings of compassion in half of the participants and compared the amount of prosocial behavior those participants engaged in to the amount of such behavior in a control condition.

The sample included 101 participants and the study was conducted online, so the age range was exceptional (from 18 to 68 years). Participants were randomly assigned to one of the two conditions and each watched a short video under the guise that there would be a test of memory afterward.  The videos were established manipulations of feelings of compassion and neutral emotion (i.e., other researchers tested their effectiveness). Following the video, participants completed two tasks which are well-established measures of prosocial behavior commonly used in such research.

Results

• Again, covariates had little impact on the results.

Hypothesis Tests (See Figure 2)

Figure 3: Results of Study 3. Values are relative.

There were two tests since the participants completed to different prosocial tasks, one involving generosity and the other involving charity.

For the generosity task:

• This time there were a main effects of both religiosity and compassion on prosocial behavior. The more religious, the more prosocial. Those who watched the compassion-inducing video were the more prosocial on average than those who watched the neutral video.
• The interaction appeared again in the manner as in Study 1.

For the charity task:

• There were main effects of both religiosity and compassion on prosocial behavior.
• There was no interaction.

This is where they screw up, in my opinion.

The pattern of the moderation was in the predicted direction but failed to reach statistical significance.

This is not an acceptable statement unless the findings are marginal. This was not. The p-value was .408. This is not even close to meaningful. Still, they went ahead with the analysis of the interaction and reported an effect of compassion on charity for the less religious participants and no effect for the more religious.  The problem is that post-hoc analysis like this assumes that a significant interaction was observed. Their tests inflated alpha (the probability of a Type I Error) and can only mislead. They stated that they had found “partial support” for their hypothesis, but they did not in this case.

The relationships in the generosity task are very clear when we look at a Figure 2. The interaction is the interesting finding. Compassion had little effect on the more religious, but a very large effect on the less religious, who gave practically nothing when compassion was not induced. There is no analysis to tell us if the less religious surpassed the religious by a statistically significant amount when compassion was induced, but they were clearly out done by the more religious when not made to feel compassion.

The charity task showed no such interaction and the authors did not include a graph of this effect that I could recreate, nor did they provide the information to make one.

Study 3

For this study, the sample of 120 completed a state compassion inventory (a measure of their feelings of general compassion at the moment) and a series of “economic tasks designed to measure their generosity, trust, trustworthiness, and motivation to reward others’ generosity.” What differed in this study, however, was that the ‘points’ they earned in these tasks could be exchanged for cash at the end of the study. Participants did not know how much cash, but they knew that the more points they earned, the more cash they would receive.

Results

Hypothesis Tests (See Figure 3)

The findings of this study were very different from the other two.

• State compassion was not related to religiosity.
• Religiosity was not related to prosocial behavior.
• There was an interaction of religiosity and compassion on prosocial behavior. The amount of compassion felt had more of an effect on the behavior the less religious than it did on the more religious.

The graph of these findings, a reproduction of their graph since they did not provide information to create one that would make more sense (to me anyway), is a bit misleading. The values are z-scores, so they are relative to one another and not actual values. What is interesting, though is how little the prosocial score varied in the more religious group and how that line barely dips below the mean value (represented by 0).

There is also a problem with the press release in that it makes the claim that the high state compassion/less religious group out-performed the others. There is no statistical analysis comparing the groups in that way, so this is a misstatement. We do not know if less religious individuals are more generous than more religious when motivated to act prosocially. We just know that they are more generous when motivated by compassion than when compassion is low.

The Study Overall

As I noted, my opinion of the studies as a whole is relatively high, but I do have some major criticisms. Some of the language makes me cringe (e.g., results are the product of statistical tests, so “We tested our results”…), but I have seen more and more of this as scientific reports in general have grown sloppier.  Study design and method is much more important, as is the quality of the reporting beyond language.

The authors also throw around the term “robust”, claiming in the first study that the relationship between compassion and prosocial behavior is “particularly robust” for less religious individuals. That term refers to findings which are “sturdy” and will stand up when some supports are removed – effects which appear to hold up in different situations. Since this was one analysis of one data set, that term just doesn’t work. It does not fit in any of their uses of it.

In fact, they err in Study 2 by saying that the effect was “attenuated” for the more religious. That term is relative; attenuated compared to what? The effect was not “robust” in one condition and “attenuated” in another; they can only be compared to each other. The effect was greater in the less religious than the more religious.

Missing Information

There are a number of bits of information which are considered to be, at minimum, required for a good research report. A general rule of thumb for methods and results sections is to include enough (without being redundant) information to allow other researchers to replicate (in a strict sense) the study and to confirm that the statistical findings are properly interpreted.

I am not sure that this article meets that criterion. The methods are pretty well fleshed out and the paper is full of statistics, but some descriptive statistics are missing that I would have liked to have seen (e.g., means reported overall for measures, but not by group) and there was not enough of the right information to recreate them.

Grouping the Data and Errors of Generalization

One overall criticism which warrants discussion is in the grouping of data. There are some problems with this and they are related. The sensitivity of the religiosity measure is one problem that, by itself, is not a big target for criticism. Combined with the second problem of grouping participants, though, it becomes more serious.

The practice of comparing groups of people based on a variable which is distributed on a spectrum is a common one. The question the researcher wants to answer is important in deciding whether to group and, in this case, I do not disagree with that choice, but I question how they grouped and how it was communicated. If the data are clustered (the distribution is multi-modal), grouping is simplified, but if the data are distributed more loosely, it can be tricky and dangerous.

First, the researcher loses information, therefore they lose sensitivity and usually lose power. The sensitivity problem is relevant in the first study, but mostly because it makes the findings difficult to interpret.

Second, if the way that the grouping is communicated is not consistent and clear, it is likely to be misinterpreted, compounding any existing problems with the method. I discussed this problem in my last post. Most of the reports referred to the groups compared as “highly religious” verses “atheists and agnostics” or something like that. However, where are all of the people in the middle (i.e., most likely the bulk of the sample)?  Within each group there was variation in religiosity and comparisons are made using averages. Generalizing only works when the samples are representative of the population of interest and this applies in either direction of the generalization (i.e., specific to mixed or mixed to specific).

Third, researchers must decide where to draw the lines between high and low (and anything in between). Since the majority of variables in psychology are normal distributed (therefore symmetrical), the lines are usually drawn using rankings of sample values and the most common way to split a sample in half is to put all values above the median into “higher” and those below into “lower” (called a “median split”). However, ease is not a good reason to use this technique.   Here is an interesting demonstration of the dangers of dichotomizing normally-distributed variables.

But… religiosity is not usually distributed normally; it’s usually skewed. Skew means that it’s not symmetrical, so a median-split would make even less sense.

In this case, it seems that the authors tried to have the best of both worlds by treating religiosity as a continuous variable, but doing post-hoc analysis on it, discussing it, and graphing it as if it were dichotomous, choosing values which were one standard deviation from the mean in both directions as the central tendencies of each group. The biggest problem with this is the assumption of normality. If the variable is not normally-distributed (and I suspect that it is not), this grouping is a bit tough to swallow.

When this problem is mixed with a limited range as it is in the first study (the religiosity scale only had four points), it’s a problem. The four possible values were 1 = no religion, 2 = not very strong (religious identity), 3 = somewhat strong, and 4 = strong. Since the mean was 2.99, the bulk of the sample were fairly religious. one standard deviation (1.03) below the mean is not exactly in non-believerland and one above is off the scale (literally). It is just very difficult to see where “higher” leaves off and “lower” takes over.

Although the range is adequate in the other two studies, the problem of discussing groups which do not actually exist and have fuzzy definitions remains. In my opinion that is one of the reasons it was so misreported.

But, overall, the research is of a relatively high quality and interesting. I would like to see more variation in the prosocial tasks, given that the outcome of the charity task was so different from the tasks of generosity.

It seems that the less religious are at least as generous as the more religious, but their reasons for acting prosocially differ. I would like to see the day when, as a group, we are generous and prosocial consistently, without the need to be provoked and without needing to feel an emotional connection to the receiver.

Saslow, L., Willer, R., Feinberg, M., Piff, P., Clark, K., Keltner, D., & Saturn, S. (2012). My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals Social Psychological and Personality Science DOI: 10.1177/1948550612444137

I had promised myself that I would spend less time ranting about the problems of the activist community, but I was so disappointed and frustrated during a Twitter exchange with Melody Hensley (of CFI-DC, caveat: she was speaking for herself, not necessarily CFI) the other night that I felt it prudent to bring it up once again, or at least a part of it.

First, I want to address the tired complaint that traditional skeptics exclude “the god question”.

Yup, we do. But before you roll out the silly paragraphs in which you substitute “God” and “religion” with “ghosts” and “the paranormal”, understand this: we also don’t address “the ghost question”.

Or “the psychic question” or “the Bigfoot question” or “the angel question”.

Statements such as “There are no ghosts” with claims that this is more than a personal conclusion are not good scientific skepticism*. Neither is “All psychics are fakes”. Neither is “there is no God”.

I can’t tell you if ghosts exist. I can tell you that I don’t believe in ghosts. I can explain why I don’t believe in them. I can give you alternative explanations for the noises coming from your attic. I can discuss reasons that you might ‘feel’ that ghosts exist. But I cannot prove to you that there are no such thing as ghosts.

I can devise an experiment to show that your dog is not psychic, but I can’t prove that psychic energy doesn’t exist.

I can explain the mechanics of sleep paralysis and the nature of memory, but I can’t say for certain that aliens did not abduct you if you remove the testability of your claim by adding things like “they reset the clocks”.

I cannot prove that there is no dragon in your garage if it does not interact with the world in measurable ways. I can only say, “I am not convinced.”

What I personally believe about these things is irrelevant. It is poor skepticism, poor science, and poor reasoning to include my beliefs in a discussion of your claims. (NOTE: “Belief” is defined in my posts as “that which one holds to be true”.)

I will not speak for everyone who has “harped” about this issue, but I can tell you that this has always been my bottom line in these arguments, so those who would take it out of context and build straw men like “she says that religion is off-limits”, don’t bother.

What I really want to talk about is about here is why this isn’t good skepticism. I’d also like to refute the tired argument that only atheists are good skeptics.

Since there are several versions of this argument and I acknowledge that they carry different meanings, I am also arguing against the following claims:

• Only atheists are rational.
• Theists/Deists may be good skeptics when it comes to other areas, but they are not skeptical about religion.
• Agnostics and theists/deists do not ‘go far enough’.
• There are no reasons to believe in/is no evidence for the supernatural.

The problem with these claims is that they are based almost entirely on a conclusion – the conclusion that there is no god (atheism). It is human nature to judge the validity of arguments by the believability of the conclusion. For example, consider the following syllogisms and decide whether each is valid or invalid:

Some students are tired.
Some tired people are irritable.
Therefore, some students are irritable.

All dogs have four legs.
Daisy is a dog.
Therefore, Daisy has four legs.

If I study, I will get a good grade on the exam.
I got a good grade on the exam.
Therefore, I studied.

If you are like most of my students, you identified the first and third as valid, but the second as invalid. Unfortunately, the opposite is true. An argument is valid if and only if the conclusion logically follows from the premises. Validity is not truth. This is important, because none of us actually knows with 100% certainty what is and is not true.

When we assume that we know what is true, we fail to evaluate arguments on their own merits. If it we were wrong, we perpetuate and strengthen our misguided beliefs instead of discovering our errors.

To know how strong a conclusion is, we must examine two things: 1) the validity of the argument that produced it and 2) the strength of the premises.

The validity of the argument lies only in its logical progression, so we can evaluate this without going beyond what is presented. However, the strength of the premises is another matter. To evaluate those, we must consider their sources. In science, some are conclusions of other arguments (often previous research) and we must evaluate that research to know how strong the premise is. Others are a matter of observation, which is subjected to interpretation and induction. For example, in the famous syllogism about Socrates mortality, the strength of that conclusion relies on the assumption that the premise “All men are mortal” is accurate. Since not all men have died, we don’t actually know with 100% certainty that all men are mortal. We accept it based on a large number of observations and converging evidence, but certainty is not possible.

In my examples, the second syllogism is logically sound, but most people reject it because they “know” that not all dogs have four legs. Perhaps you’ve seen or heard of a three-legged dog (I met one once named “Tripod”) or you have knowledge of how it could occur. This does not make the argument invalid, but it does address #2, making the conclusion unsupported. We cannot determine if it is true from this argument.

The first and third arguments are invalid because the logic is unsound. We may “know” that the conclusions are true, but we can’t know that based on these arguments, so if we want to convince others we need to come up with better evidence/arguments.

The tendency to judge conclusions based on current beliefs is a product of how our brains evolved and developed – a side-effect of what makes us successful organisms. It is human nature, it is wrong and must be overcome if one is to be consistently rational (This, by the way, is a bit of a pipe dream, but I think it’s a good goal).

This problem pops up in a host of cognitive tasks and is a manifestation of the most influential of human frailties: the confirmation bias. This makes it extremely resistant to correction, especially in real-world contexts. In my experience, the concept of “validity” is difficult for many people to grasp because of this problem.

So, going back a few paragraphs, note that I wrote, “The problem with these claims is that they are based almost entirely on a conclusion – the conclusion that there is no god (atheism).”

Reason is about the validity of arguments, so judging a conclusion as valid or invalid without examining the argument is itself an irrational act. Without the argument, your only yardstick is your own belief about the truth of that conclusion. Although we have reasons for our beliefs, so do the people whose beliefs we’re evaluating. Everyone thinks that their beliefs are well-reasoned and accurate. That’s why they believe them!

If you find their conclusion unbelievable, then by all means, be skeptical, but to call it “irrational” without evaluating the argument is to say that you are 100% certain that there is no rational argument. That is the very definition of arrogance and it is not scientific.

Science does not tell us what is (true). Science tells us what is likely (to be true) and, in most cases, how likely. It does so by making arguments. Science is shared knowledge, not because it tells us facts, but because we can discuss the evidence and logic processes behind why we should be reasonably certain of many things. Although science is both a process and a set of knowledge (I’ll call them ‘facts’), the facts in that set are the products of the process. This may include negatives such as “my dog is not psychic” and “vaccines do not cause autism”, but testing is required to make such conclusions scientific.

Science is not about those facts, though. It’s about the process of discovery. When scientists make arguments (by publishing papers), they cite previous literature by noting the findings and, in some cases, describing how those findings were produced. They do not list facts; they discuss evidence.

Scientists don’t judge conclusions. Scientists judge arguments. Scientists look at the whole argument – the assumptions, evidence, and methodology that make up the premises as well as the logic that holds them together – and judge if the conclusions logically follow from those premises.

Likewise, scientific skepticism is about testing claims, examining evidence, and providing natural explanations for the evidence. If there is no evidence to examine, there is nothing to discuss.

Because science ignores untestable claims and because some scientists (e.g., Carl Sagan) have discussed the reasonability of belief without evidence, many people oversimplify the issue (as Melody did in this Twitter conversation) by making the statement that belief in God is within the scope of scientific skepticism because “You don’t believe something without scientific evidence”.

This is misguided for several reasons, but the two main reasons are:

1. The assumption that most or all religious belief is completely blind faith is simply wrong. “Evidence”, scientific or otherwise, comes in all manner of forms. In a 1998 study conducted by the Skeptic Society, the most popular answers among believers for why they believe in God involved empirical evidence and/or reasoning (you can find this in Michael Shermer’s How We Believe). These people certainly think that they have good reasons to believe. How people interpret evidence varies a great deal. Most real-world questions are very complex and what seems an obvious conclusion to one person may seem ridiculous to another. Skepticism is about how we interpret evidence, how we reason, and how we consider alternative explanations, not about the conclusions we eventually draw.A good example of interpreting evidence is the study that prompted me to send a tweet to Melody in the first place. She’d first shared a link, then tweeted it again, this time directly to Daniel Loxton, editor of Junior Skeptic, with the comment “More good news about atheists that you might find hard to believe:” Only Melody knows what she meant by the comment, but when I read the study I was disappointed to find that it is not “good news about atheists” at all. I will include details in a post to follow shortly, but the gist is that article’s spin is a very loose interpretation of the findings that fails to mention some sobering facts. Its overgeneralizations and assumptions are criminal. For example, the study reported doesn’t involve atheists. Researchers measured religiosity, then divided participants in half (“more religious” and “less religious”), a common practice in social psychology. Sure, there were atheists among the “less religious”, but we don’t draw conclusions about a part of a sample. Furthermore, taking the three studies as a whole, the ‘more religious’ were more compassionate and more prosocial than the ‘less religious’ half. In essence, anyone wanting to spin the findings another way could easily do so by noting that the less religious half onlyacted prosocially when moved to do so by compassion, whereas the more religious were consistently prosocial. This is not a finding that atheists are better people and, quite frankly, I am sick of people trying to prove such nonsense. Promoting the fact that one can be good without God does not require atheists to be morally superior and, as this study shows, it is a good thing that it doesn’t.Again, we all think that our beliefs are well-reasoned, but what’s more interesting is that people tend to assume that those who disagree do so because either they “need to” (In that same Skeptic Society survey, the most popular reasons believers gave for other people’s beliefs in God involved the stereotype of comfort and meaning to life.) or they are not as rational.
2. Even if there was literally zero evidence, the “null hypothesis” argument is an oversimplification of a concept borrowed from statistical rules and applied to the assumption that science makes that every hypothesis is testable. Science makes this assumption, but it also acknowledges that the assumption could be wrong by excluding the possibility of 100% certainty and by limiting its scope to testable hypotheses. You cannot invoke science as an answer to claims it cannot test, nor can you claim that someone’s conclusion is wrong because science cannot test it; that’s circular reasoning unless you’re saying that science is the only way to know something. By that logic, people cannot be scientific thinkers and also have morals**. Science (shared knowledge) may ignore the supernatural, but people (personal knowledge) do not and cannot use scientific processes to examine every question and still manage to function in the world, so if you want to attack person knowledge as wrong, you’ll have to do better than “it’s not scientific”.

So, what about the other versions of this claim?

Are atheists more rational than believers? Probably on average, considering the other variables which are correlated with atheism. However, given how poorly all human beings are at reasoning, that isn’t saying much.

Is atheism rational? I can’t answer that. Atheism is a conclusion. Whether it’s a rational conclusion depends on why the individual drew that conclusion.

Is religion rational? Again, I can’t answer that and neither can you. It’s a conclusion. Whether it’s a rational conclusion depends on the reasoning of the individual and the evidence they considered.

From what I know about how human beings process information, I can see a great many valid arguments for the existence of God that would be perfectly rational. They’d have to have some extraordinarily well-supported premises in order to convince me, but lacking support for those premises won’t make them irrational. Reasoning well does not require convincing others.

As I’ve said twice now, everyone thinks that their conclusions are the ‘right’ conclusions. So what makes your conclusion (that there is no God) better than someone else’s?

If your answer is “mine is well-reasoned”, that’s not a comparison. See two sentences back.

To know that you are “right” and they are “wrong”, you actually have to examine their argument/evidence. If you haven’t examined their argument, then who are you to tell someone that they are irrational? Who are you to tell them that they have no evidence when you haven’t even bothered to ask them what their evidence is? This is exactly what you’re doing when you claim that any belief in a god is “irrational” or make a blanket statement about the intelligence or cognitive abilities of those with religious beliefs. It’s elitist, arrogant, bigoted wishful thinking.

You cannot judge an argument by its conclusion, no matter how unbelievable the conclusion seems to you.

Finally, the following is a list of things that I have NOT said. In fact, I do not believe that any of the people accused most of making these statements actually has:

• “Religion is off-limits in skepticism.” There are plenty of testable claims related to religion, but test the claims and discuss the evidence rather than attacking belief in them.
• “Stop talking about ‘the god question’.”I have no problems with debates about the existence of God. What I have a problem with is criticizing conclusions as rational or irrational without examining the argument that produced them and calling such criticisms “skepticism”.
• “Stop promoting secularism.” I am a strong advocate of secularism, but promoting atheism is, in my opinion, no different from promoting any other set of conclusions. Freedom from religion requires freedom of religion. Removing religion from government does not mean taking it away from its citizens.
• “Atheism is not the only valid conclusion of properly-applied skepticism.” I did not say that it was, either. Hopefully my discussion explains why that question is not relevant.

I had planned to ignore Melody’s accusations that my criticisms are ‘mean-spirited’, but I cannot do that, either. It certainly is mean-spirited to attack people, but that is not what I have done. You won’t find a personal comment about Melody here. If it is mean-spirited to address (or even attack) what people say and do, especially when one finds what they promote to be harmful, then our whole business is ‘mean-spirited’.

*Given that all of the major organizations in skepticism have adopted scientific skepticism, this is what I’m discussing. If you would like to argue that activism should go beyond scientific skepticism, please do so elsewhere. For that matter, this post is not even about limiting scope for practical and strategic purposes, which is an entirely different cup of tea.

**Yes, I am aware that Sam Harris claims that science can tell us what is moral, but so far his arguments fall far short.

NOTE TO WOULD-BE COMMENTERS: Please do not comment if you have not actually read (not skimmed, READ) the post. Also, before you write a comment about how Dawkins and others (i.e., persons whose credentials you think I should not question) argue that science refutes the existence of God or should include “the god question”, I recommend a thorough review those arguments (that means more than reading a couple of blog posts by bystanders or comment threads). Their treatment of the subject is much more considered than the oversimplification I’m addressing here and their arguments are not as shallow. Finally, if your plan is to quote from Sagan’s Demon-Haunted World, you might want to read the whole book first.

NOTE: this post also appears on the wonderful site about crazy coincidence, theoddsmustbecrazy.com.

Monty Hall

I have always been amused and intrigued by responses to “The Monty Hall Problem”, especially when I talk about it to audiences with a high concentration of engineers and mathematicians. If you are familiar with it, but you’ve always struggled with an unsettled feeling of “this can’t be right”, read further and let me know if my explanation of the solution helps to alleviate the discomfort. If you are not familiar, I guarantee you will give your brain a workout by reading on.

First posed to statisticians in 1975, “The Monty Hall Problem” is well-known among academics because it still sparks debate. Many seem to think that disagreements about its solution stem from issues in the clarity of the problem, but I contend that it really stems from human flaws in the way that we process information.

I often discuss this problem in statistics and cognitive psychology courses for several reasons. It is a great exercise in probability calculation and it can be used to teach basic mathematical modeling (and its purpose). An added benefit, since almost all of my students were psychology majors, is that it also illustrates a flaw in human cognition as well as a pattern of problem solving. Even a knowledgeable statistician feels the need to run simulations to see the solution in action. Even then, fully grasping the mechanisms behind the answer often requires brute force cognition.

In general, human beings have a very difficult time wrapping their brains around concepts of probability. It is much like a visual illusion; we know that the lines are parallel/the circles are the same size/there is no motion, but we can’t make our brains process it in a way that represents that reality. It’s just not how our visual system works. I hypothesize that one of the reasons that probability is such a difficult field for most people is that it involves theory and models, which are distinct from observations and we must represent them differently in our minds to properly deal with them. Applications of probability often involve switching gears from the realm of models to data or vice versa and this is where I think most mathematicians get side-swiped in The Monty Hall Problem.

The Poser

In essence, here’s the problem:

You are a contestant on Let’s Make a Deal!and Monty loves your creative costume (a teddy bear carrying a human doll), so he calls on you to make a deal. Monty says, “There are three doors – Door #1, Door #2, and Door #3. Pick one and you get to keep whatever is behind it.”You’ve seen the show (you weren’t just walking down Ventura Boulevard in a teddy bear costume for fun), so you know that it is highly likely that there is a coveted BRAND NEW CAR! behind one of those three doors. If you choose wrong, however, you might end up with an ostrich…

Everyone hopes for a car. Some get donkeys or other animals.

You choose Door #3.

Monty then says, “Let’s see what’s behind Door #1!” and the door opens to reveal one of the many consolation prizes (and product placements), a lifetime supply of Rice a la Roly.

Cool! You might get that car after all!

Well, the show was successful because the shell-game-huckster-style of Monty Hall rarely stopped there. In this case, he does what he often does, offers to let you switch from your first choice (Door #3) to the only remaining option, Door #2.

Should this woman switch?

Should you? Does it matter?

Not the Problem

Before I get into the solution, let me first deflect a common complaint from mathematicians. The most well-known version of the problem, from its Wikipedia entry:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1 [but the door is not opened], and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

This version does not specifically state the name of the show or indicate the way that game shows of its era worked. If you have never seen the television show (i.e., you are younger than 35), or any game show of its kind, let me explain. Monty is in control of almost everything that happens. The only thing “contestants” can do is make choices when Monty offers them. As you will see, they had more control over their odds of winning than once thought, but Monty manipulates some of the build-up by choosing which items to reveal at different steps in the game.

Unfortunately, many probability theorists and mathematicians took issue with the lack of clarity in the problem (context is important sometimes). This provides a face-saving ‘other version’ for the geeks who get it wrong the first time. But whenever I hear comments like, “Okay, given this version, that Monty knows where the car is.” I usually think, “Of COURSE he knows where the car is! There is no other way to play the game!” and wish that people were more able to accept that they are just as human as everyone else.

The problem itself is written clearly, though: it specifically states that a door without a car behind it is revealed before you are given the option to switch. If the situation was a fully-randomized, double-blind game (like “Deal or No Deal”), then the option to switch would not even be on the table if the car is behind the revealed door. There would be no problem in that case. Therefore, the problem is a question of whether you should switch in a controlled setting – one in which the only participant who doesn’t know the location of the big prize is you.

The issue of knowledge is a factor in our processing of the problem, but it’s not what Monty knows that matters. It’s what you (the subject of the problem) know.

So, let’s put that complaint behind us and get back to the problem.

The Answer, and How to See it for Yourself

Hopefully, if the problem is new to you, you’ve spent some time trying to solve it instead of going with your first gut feeling, which was probably, “It doesn’t matter.”

It does. You should switch.

If you don’t believe me, try running some simulations. You’ll have to run a lot in order to get a large enough sample to be certain to see the trend, but here are a few ways to do it:

• Use your favorite program (MATLAB, R, etc.). There is a good database of pre-written simulators for this here. I am partial to Excel myself, even though it’s a bit more cumbersome. I just don’t remember enough code to use another program.

• Use a web-based simulator. Do it at least a hundred times, choosing to switch for half of the trials, and keep a tally your results.

• Use a die to simulate the outcome, assigning 1-3 to “Door #2″ and 4-6 to “Door #3″ (e.g., if you roll a 5, Door #3 is the one with the car). Roll at least a hundred times, choosing to switch for half of the trials (before rolling!). Keep a tally of the results.

What you will see is that switching will result in winning a car in approximately 2/3rd of the trials while staying will only provide a win in 1/3rd of them.

I know what you’re thinking. “But, there are only two doors left, so it should be 50/50!”

Why it is so Difficult to Accept

Human cognitive development is an interesting process. We learn to interpret information from the environment very quickly so that we can respond to that environment, but learning to reason hypothetically takes more time. Even adults with scientific training have a difficult time separating the concept of variables (each has a set of possible values) and data (values which are known).

In practice, hypothetical situations are often conditional (e.g., “If A, then B”). We tend to use information about what is to reason about what could be. We do this because it often works, but it is one of the ways in which our brains can lead us astray. For example, given the premise, “If I study, I will get a good grade on the exam”, what is the most sound conclusion when presented with a good grade? The most common response is, “I must have studied”, but that is not sound. In this premise, studying provides a guarantee for a good grade, but there is no statement that studying is the only way to get a good grade. It does not, for example, read, “If and only if I study…”

In the case of the Monty Hall Problem, the probability of winning is set before you pick a door. No matter which door you choose, the probability is 1/3rd. This is because there is a 1/3rd probability that the car is behind the door you chose given the information you had when you chose it. In reality, the car is behind one of the doors, so the probability it is behind Door #2 is 100% if it is there and 0% if it is not there. Probability is not a useful way to discuss what is or what happened; it is a tool for predicting what is likely to be true/happen.

The new information provided by revealing a loser changes the circumstances and this where we get trapped in our representations of models and data, possibilities and facts.

You had a 1/3rd chance of winning because there were three, equally-likely locations to choose from. It seems as if cutting the choice down to two should change the odds of winning to 1/2. It seems that way because we are focused on the probability that a given piece of information is true (e.g., that the car is behind Door #1) and not the probability that an event will occur (e.g., that we will win the car). The probability that we will win the car relies on the number of possible states of reality. This, in turn, initially relies on the number of locations for the car. When the situation changes, we try to adjust probabilities based on possible locations (which has changed) rather than on the number of possible states of reality (which has not).

Basically, when Monty makes the second offer, the offer is to switch from the door we have (#3) to the door we don’t have (#1 or #2). It does not matter that only one of those doors is left; there is still only a 1/3rd chance that our door has the car and a 2/3rd chance that the set of the other two contains the car.

Getting Un-Stuck

If you change the way you represent the problem from the beginning, the solution might seem more reasonable. Specifically, instead of thinking in terms of assigning probabilities to doors, think in terms of assigning probabilities to outcomes: winning verses losing.

Three possible states of reality, each with one winner and two losers

Let’s go step by step…

Monty asks you to pick a door from three choices. Behind one of those doors is a car. There are three possible locations and it must be in one of them, so there are three possible states of reality.

You choose to bet on Door #3; there is a 1/3rd chance that you will win the car.

There is a 2/3rd chance that you will not win the car.

This would be true no matter which door you chose.

Monty reveals that one of the remaining doors is a loser. At least one will be a loser since there is only one winner and you can choose only one. The car, however, does not move. Even though there are only two locations left, so there are still three possible states of reality. What’s changed is that we now know more about each of those possible states (there are fewer locations for the car to be):

So, if we model the problem in terms of the probability of winning with Door #3, the model itself does not change after the losing door is revealed. What changes is that we would no longer want to choose that door, so it is no longer among our options. This leaves us with only two options: keep the door we have or switch to the remaining door. The odds of winning/losing with Door #3 have not changed, but eliminating an option allows us to make a better choice – switch.

In an age in which many teens and young adults seem to equate criticism and not getting what they want with disrespect, it should not be surprising that some argue for unconditional positive regard in education and elsewhere. However, criticism is essential to gaining knowledge.

I am sure that most people have not thought much about why it’s called “critical thinking” and, when asked, would probably assume that “critical” refers to its importance (especially skeptics). However, they would only be partially correct. “Critical” means “crucial”, but it also refers to discriminating judgments. Critical thinking is a process in which we distinguish accurate information from inaccurate information and/or determine the best course of action given a set of possibilities.

The reason criticism is so important is a bit abstract, but it can be at least partially illustrated through the principle of falsification. Essentially, attempting to falsify a hypothesis is the best way to determine if it is likely to be true.

Most skeptics are familiar with the confirmation bias, which is the very strong human tendency to favor confirming information. We notice, remember, believe, and assign more weight to information that confirms what we already believe to be true than other information. What is discussed less often is how this bias extends to how we seek knowledge and test hypotheses. Humans tend to experiment to determine relationships in the world, but we do so in a confirmatory manner when a better approach is to attempt to falsify. That is, we tend to try this or try that to see what happens. However, the best test of a hypothesis is one in which the outcome tells us what is likely to be true by eliminating other possibilities. This is how science works most of the time and I think that we can agree that science works better than any other method.

A good illustration of this human tendency is the Wason 4-Card Task. Fortunately, it is also a good way to explain why falsification is ideal.

In the Wason 4-Card Task (A.K.A. the Wason Selection Task), participants are given the following problem:

There are four cards in front of you, each with a letter on one side and a number on the other side. The rule is: If there is a vowel on one side of a card, then there must be an even number on the other side.

Which cards must be turned over to determine if the rule holds true? (How many cards, minimum, and which ones?) 

The most popular answers in this task are the E and 4 cards or just the E card. This demonstrates an approach to hypothesis testing which is confirmatory in nature. That is, we search for examples of the first condition, then confirm that the second condition is met. This is similar to how most people deal with other types of questions, such as whether or not a pill reduces headache pain. How would you find out? Should you take the pill when you have headache pain, then judge the pill effective if the pain is gone an hour later?

Unfortunately, this approach is not a very effective means of gaining knowledge. To understand why, let’s look at what we can learn from turning over each card.

Only the bold information tells us anything about whether the rule holds true or not and only in the cases in which it is violated do we learn something meaningful enough to draw a conclusion. The correct answer to the basic Wason Task is that both the E and 7 cards are required. The E card can confirm the rule, but what is important is that it, and the 7 card, have the ability to falsify it. Turning over the 4 card is unnecessary because the rule does not even apply unless it has been followed, therefore, we learn nothing interesting.

If my statement about is confusing, you are not alone. This task is one of the most well-studied in the field of cognitive psychology, yet why humans are so bad at it is still a bit of a mystery. One of the hypotheses is that people have a difficult time recognizing that the rule is not bi-directional. In other words, the rule is:

If there is a vowel on one side, there is an even number on the other side.

it is NOT:

If there is a vowel on one side, there is an even number on the other side and vice-versa.

However, when conditional statements like this are made, we often assume that they are bi-directional unless we have context which tells us otherwise. For example, rules like these provide context which suggests more than what is stated:

If it is Monday, then the banks are open.

If I buy a ticket, I will win something in the raffle.

Because we have schemas that tell us that banks are open on weekdays (at least), we understand that the reverse of the first conditional (If the banks are open, then it is Monday) is not necessarily true. In the second conditional, our knowledge of how raffles work tells us that a raffle ticket must have been purchased if the author won something in it. In each case, however, we are not evaluating the statements on their own merit. We introduce information from our knowledge base. The problem with this kind of reasoning is that we assume that our knowledge is accurate.

When these statements are included in arguments, we draw conclusions based, at least in part, on our current understanding of the world, which could be wrong.

One example that I often used in class illustrates how strong this bias can be and how it affects our ability to determine cause and effect relationships. Given the following argument, students are asked to judge whether the argument is valid or invalid. They are reminded that “valid” refers only to whether the argument is logically sound and not whether the statements within it are true (truth is a separate property):

Major Premise: If I learn a lot, I will get a good grade on the test. 

Minor Premise: I got a good grade on the test.

Conclusion: Therefore, I learned a lot. 

On its surface, the argument appears valid. However, it is not. There is nothing in the major premise to indicate that learning is the only way to get a good grade on the test – that learning is required for a good grade. It merely states that learning will guarantee one. Unfortunately, some dedicated students who have successfully used flash cards and other rehearsal techniques in courses in which memorization is helpful – introductory courses with multiple-choice exams and so forth – will assume that this method of studying equates to learning. When they fail in a more rigorous upper-division course in which conceptual understanding is necessary, they will be unlikely to attribute that failure to their study methods.

Going back to the example of how to determine if a pill takes away your headache pain, the method most people follow is to turn over the E card and nothing else. If your headache pain does not go away (outcome: an odd number), then you will know that the pill doesn’t work. However, what if the headache does go away? Is it possible that headaches go away on their own? Could taking the pill induce a placebo effect? What you need is to design an experiment in which you can eliminate all other possibilities (headaches go away by themselves, placebo effects, etc.). What you need to do is set the pill up to fail. If the headache goes away when it should (turning over the E card and finding an even number) AND it does not go away when it should not (turning over the 7 card and finding a consonant), then we can conclude that the pill worked.

The confirmational approach is like asking a psychic to predict the flip of a coin, but only counting the times in which the psychic was correct. This is an excellent way to find out if they have supernatural powers, but only if you also know the rate at which they are incorrect. When skeptics test psychics, they compare the outcome to what would be expected by chance. If they can consistently outperform blind guessing, the only logical conclusion is that they must not be guessing. To eliminate alternative explanations, skeptics ensure that the psychic is not receiving information from other sources such as body language cues.

Now, what does all of this have to do with criticism?

What science does, both in single studies and as a general process, is eliminate incorrect hypotheses. In general, scientists come up with likely hypotheses and theories based on what we are reasonably certain is true (as many as humanly possible), then eliminate them until only one remains. We eliminate them by finding faults with them, by demonstrating that they can’t be true (or are highly unlikely to be true).

Critical thinking works this way, too. The most effective process of evaluating information is one which looks for faults, inconsistencies, and weaknesses. Applying critical thinking to plans and ideas, otherwise we waste resources and time as well as risk harm. After we have completed a project, even a successful one, analysis should not focus on “what we did right”. We can’t know why we succeeded without comparisons to similar situations in which we failed. Focusing on what we think is good is like counting the “hits” and ignoring the “misses”.

Many Skeptics, Daniel Loxton and Massimo Pigliucci (So you think you’re a skeptic, don’t you?) come to mind, have discussed the need to restrict one’s public discussions in the name of Skepticism to topics within one’s area of expertise. In the absence of such expertise, we should only convey to the public a scientific consensus, if one exists. So how is a non-scientist or someone working in a different field supposed to know whether a scientific consensus exists and/or what that consensus is?

Well, that is what I had initially intended to write about today. I am afraid this post goes a little off-track, but it still covers important ground.

This post began as a set of corrections to some of the misleading statements in a recent post by Rebecca Watson in which she points out what she believes is wrong with Ben Radford’s use of an article to support his argument that images of thin women in media do not “encourage” eating disorders such as anorexia nervosa. A friend alerted me to this post, knowing that I participated in a discussion on Facebook on the matter.

In her post, Rebecca accuses Ben of cherry-picking and quoting out of context. In a response to Rebecca , Ben defends himself quite well regarding the accusations that he misrepresented the authors (there were two), but I think that there is a lot missing from the discussion that is important and, in fact, this is shaping up to be a great example of why organized skepticism is needed and why more working scientists should get involved.

First, I share Rebecca’s concern that readers will assume that Botta’s findings support Ben’s claim. Ben writes,

Rebecca is assuming that the quotes were selected as representing the conclusions of those particular studies from which they were cited. I made no such claim.

This is true, but Rebecca assumed what most laypersons are likely to assume. It is far too easy to mislead when discussing complex topics such as this one. Evidence to support one’s assertions is difficult, too, when the assertion is that a claim is untrue. I think that Ben’s argument would have been better served by a more detailed discussion of the complexity of the issue and, perhaps, a quote from a recent review of the literature. Those are difficult to come by, but they provide the “big picture” view to which he referred.

Quoting from the introductions of research reports as Ben did is never a good idea, in my opinion, because it is hearsay. However, it is also problematic when people assume that a researcher’s conclusion can support an argument; it can’t. What counts as evidence is a finding.

For example, what prompted me to participate in the Facebook conversation was Rebecca’s comment:

Study after study shows that straight women and gay men develop eating disorders because our society tells them that they must be thin and pretty in order to attract a man (eg: [link to study by Siever]), and that attracting a man should be their #1 goal in life.

The study to which she linked found correlations among disordered eating, body dissatisfaction, the importance placed on physical attractiveness, and sexual orientation by gender. How that was translated into a causal chain involving societal influences and personal goals is quite alarming, but the paper itself provides some clues. The discussion is longer than any other section of the paper, less than parsimonious, and somewhat speculative. This is one of the reasons that researchers cite and discuss findings, not the conclusions other scientists draw.

As I explained to Rebecca,

Correlation does not equal cause.

Causal conclusions are extremely difficult to draw when you cannot randomly assign subjects to conditions and you cannot randomly assign people to be a straight women or gay men. It takes massive amounts of converging evidence from a variety of studies which eliminate rival hypotheses.

So to say that “straight women and gay men develop eating disorders because our society tells them [anything]” is a very bold statement that is not supported by the evidence. It is certainly not supported by the study to which you linked.

But the confirmation bias leads people to follow their current beliefs and demand to be proven wrong. For example, Rebecca ended her post with,

In the Facebook thread, I genuinely wanted to see what evidence actually supported the idea that the link between media and body image is a myth, but I got nothing from Radford or anyone observing the thread. So, I’m forced to continue to side with what appears to be the consensus opinion: the media’s portrayal of the thin ideal most likely negatively impacts the body image of those who process those images poorly.

First, the “who process those images poorly” part is a new assertion and one that reveals her understanding (or lack of understanding) of the findings. What does it mean to “process images poorly” anyway? She initially wrote, “processing them the wrong way”. Is there are “right way” and a “wrong way”? Botta’s variable of “image processing” involves whether the individual questions the weight of characters as realistic, whether they believe they are realistic, and whether they compare themselves to the characters. There’s no right or wrong here.

What’s more important is her comment that she wanted evidence that the claim is untrue. As I explained to Rebecca on Facebook, that’s not how skepticism works. The burden of evidence is on those making the claim, not those refuting it.

This post is becoming much longer than I’d intended, so I will skip criticism of Posavac’s article and Rebecca’s misrepresentation of the findings. Instead I’d like to discuss the Botta article as it seems to be at the center of this battle. My interpretation of it is quite different from Rebecca’s.

Some background that I found interesting: Botta is not a psychologist, but a professor of communications. Now, expertise can come from many places. Formal education in an area is not a guarantee that one is an expert (although, given no other information, it is much more likely that someone with a PhD in a given area knows that area well than someone without a PhD in that area). Likewise, one can gain expertise through years informal study and practice. My point, though, is that we should not assume that Botta is an expert on eating disorders.

What is more telling about this particular article is the journal in which it was published: the Journal of Communication. It is not published in the psychological literature where psychologists are likely to see and criticize it. A communications journal is not an inappropriate venue for this article, however, because it is not about eating disorders.

This 11 year old study is about body image.

In fact, Botta used the EDI (Eating Disorder Inventory), a well-established, valid, reliable inventory of disordered eating. However, she chose to use only those sub-scales related to body image disturbance, thus making generalization to disordered eating impossible. She did include a measure of “bulimic behaviors”. It’s really very puzzling. I cannot think of an explanation for this, but one would not be relevant. What is relevant: she did not measure disordered eating with the exception of “bulimic behaviors” and references to other eating disorders as a conclusion to this study would be a GROSS overgeneralization.

There are many, many studies regarding both immediate and long-term effects of media exposure on body image and the findings are not consistent, particularly those examining television viewing. Of course, the quality, methodology, and generalizability of those studies are not consistent, either. As always, null findings are likely to be underreported.

There is no scientific consensus that “ideal thin” media is a direct causal factor in body dissatisfaction. Skepticism of any claim to this effect is certainly warranted.

This area of research is fraught with methodological problems similar to the study of acupuncture and psychotherapy. Body image measures rely on self-report and studies involving media imagery make hypotheses obvious to the participants. What’s more, most studies involve short experiments from which generalizations about real-world effects are extremely difficult.

What the massive body of literature on body image suggests: person variables (characteristics of individuals) such as self-esteem (Jarry & Kossert, 2007), attachment orientation (Greenwood & Pietromonaco, 2004), neuroticism (Daily & Buunk, 2009), and even other components of personality (Roberts & Good, 2010) have direct effects on body image and/or mediate/moderate the relationship between media and body image. In other words, any relationship which exists between media exposure and body image is complex and, at the very least, indirect.

But that is not even what Ben was talking about in his original piece, nor is it what Rebecca started to discuss (given the title of her post). The topic was the question of whether the “popular view” that exposure to images of thin women promotes eating disorders is myth.

Even a surface treatment of that question has me scratching my head a bit given the nearly equally accepted “popular view” that there is an epidemic of obesity in this country.

While body image is highly correlated with eating disorders – it is even one of the diagnostic criteria – there is no evidence that poor body image, or even the desire to be thin, causes eating disorders. Stating that eating disorders are a direct result of wanting to be thin is a lot like saying that obsessive-compulsive disorder is the result of wanting to be organized or clean. Assuming a causal link exists, the direction of cause could easily be the opposite of what people think.

But let’s assume that what we’re really talking about is body image and not eating disorders. Do Botta’s findings really refute Ben’s assertions? Given that he did not discuss the study in detail, I am not sure if Ben knows, but let’s take a look.

Botta’s study is a complex analysis of a number of self-report measures which involves a large number of statistical tests. As such, some significant relationships are likely to occur by chance and it is important to consider the specific hypotheses when interpreting them. I’ve noted where Botta reported significant relationships that I believe should be considered with caution. That said, here is a list of what she found:

Factors in whether participants endorsed a thin ideal

What did NOT predict endorsement of a thin ideal:

• total television exposure
• exposure to “thin dramas” [shows like Melrose Place and Beverly Hills, 90210 – did I mention this study was more than decade old?]
• the extent to which participants reported questioning characters’ bodies when watching television

What DID predict endorsement of a thin ideal:

• the extent to which participants compared themselves to the characters
• the extent to which participants viewed the characters as realistic

Factors in body dissatisfaction

What did NOT predict body dissatisfaction:

• total television exposure
• exposure to “thin dramas”
• the extent to which participants reported questioning characters’ bodies when watching television
• the extent to which participants viewed the characters as realistic

What DID predict body dissatisfaction:

• Body Mass Index
• endorsement of the thin ideal
• the extent to which participants compared themselves to the characters

Botta also reported a significant interaction of endorsement of a thin ideal and total television exposure, but with a p-value of .03. Given the large number of tests produced in this kind of analysis, it pays to be a little more conservative and consider only those less than .01 as significant. Likewise, she reported that ethnicity predicted body dissatisfaction, however, she only reports the p-value as less than .05.

Factors in drive for thinness

What did NOT predict drive for thinness:

• total television exposure
• exposure to “thin dramas”
• the extent to which participants reported questioning characters’ bodies when watching television

What DID predict drive for thinness:

• Body Mass Index
• endorsement of the thin ideal
• the extent to which participants compared themselves to the characters

Again, Botta also reported that viewing the characters as realistic was related to drive for thinness, but with a p-value of less than .05.

Factors in bulimic “action tendencies”

What did NOT predict bulimic tendencies:

• total television exposure

What DID predict bulimic tendencies:

• Body Mass Index
• endorsement of the thin ideal
• the extent to which participants compared themselves to the characters

Again, Botta also reported that exposure to “thin dramas” predicted bulimic tendencies, but not in the direction you might think. The more they reported being exposed to “thin dramas”, the LESS they reported engaging in bulimic behaviors. Regardless, with a p-value of less than .05, I don’t believe it needs to be explained as anything more than an odd finding. In addition, there was an interaction of thin ideal endorsement with the questioning of characters that was difficult to fully interpret or explain.

So, do Botta’s findings support the claim that media images cause eating disorders?

Not. At. All.

Do they suggest that media images cause eating disorders?

Nope.

Do they suggest that media images cause body dissatisfaction?

Again, nope.

Do they suggest that media images teach women that thin is best?

No.

What do her findings suggest?

The amount of media exposure, even specifically to thin characters, is not directly related to whether women think that thin is best, body dissatisfaction, drive for thinness, or bulimic tendencies.

If her findings are similar to those of other researchers (and they are), they actually kinda support Ben’s assertion [we’re still in the “what if we were talking about body image” state].

What Botta’s findings also suggest: Women who compare themselves to characters on TV are more likely to think that “thin is best” than women who do not compare themselves to those characters. Also, thin ideal endorsement and current BMI are related to body dissatisfaction (not a surprise), drive for thinness, and bulimic behaviors.

So, it seems to me that what these findings tell us, beyond “media is not the problem”, is that women who are unhappy with their bodies engage in behaviors which are likely to make them even more unhappy. They have warped views of what is ideal and compare themselves to people who, in their judgment, fit that view. How this equates to “every word of it disagrees with Radford’s assertion that media images have no relationship to body image” I don’t know.

I think it is plain, though, that the issue is complex and so is the literature about it. It is fairly easy for the average human to view this kind of literature as supporting their current view of the world. It is also human to defend that view, even when it is not supported, and to ignore explanations of why they should be skeptical. That’s one of the reasons we need Skeptics (like Ben Radford).

Some References

Botta, R. (1999). Television images and adolescent girls’ body image disturbance Journal of Communication, 49 (2), 22-41 DOI: 10.1111/j.1460-2466.1999.tb02791.x

Dailey, S.E., & Buunk, A.P. (2009). Female body dissatisfaction after exposure to overweight and thin media images: The role of body mass index and neuroticism Personality and Individual Differences, 47 (1), 47-51 : 10.1016/j.paid.2009.01.044

Greenwood, D.N., & Pietromonaco, P.R. (2004). The interplay among attachment orientation, idealized media images of women, and body dissatisfaction: A social psychological analysis in The psychology of entertainment media: Blurring the lines between entertainment and persuasion. Shrum, L. J. (Ed.), 291-308 Other: 2003-88226-016

Jarry JL, & Kossert AL (2007). Self-esteem threat combined with exposure to thin media images leads to body image compensatory self-enhancement. Body image, 4 (1), 39-50 PMID: 18089250

Roberts, A., & Good, E. (2010). Media images and female body dissatisfaction: The moderating effects of the Five-Factor traits Eating Behaviors, 11 (4), 211-216 DOI: 10.1016/j.eatbeh.2010.04.002

Siever, M. (1994). Sexual orientation and gender as factors in socioculturally acquired vulnerability to body dissatisfaction and eating disorders. Journal of Consulting and Clinical Psychology, 62 (2), 252-260 DOI: 10.1037//0022-006X.62.2.252

Dragon*Con is now just over three weeks away and the schedule is as solid as these things get, so I’m giving you the highlights as promised. I will be one BUSY Skeptic! [Edit: By the way, Dragon*Con will take place over Labor Day weekend, September 3rd through 6th.]

For those who have not heard of it, Dragon*Con an enormous SciFi/Fantasy convention held in Atlanta over Labor Day weekend. Skeptrack, the skeptic fan track, is the brainchild of Derek Colonduno and Robynn McCarthy (A.K.A., “Swoopy”), hosts of Skepticality. As usual, they work their butts off and many skepticism activists do their best to make that work count by participating in talks and panels designed to spread the critical thinking bug to geeks everywhere.

Dragon*Con is very different from conventions like The Amaz!ng Meeting, and not just the costumes. TAM is, at heart, a Dog-and-Pony-Show for skepticism. At Dragon*Con activists do the work of activism and education. There are a number of fan tracks, including a Podcasting Track directed by Swoopy, and Space & Science Tracks, who work closely with Derek and Swoopy. In addition to the obvious Star Wars Track, Trek Track, and Whedonverse Track, there are literature, costuming, and writing tracks. There are also two tracks related to Skeptrack: Paranormal Track and X Track.

The former is self-explanatory, I hope, and this year I will be attending a workshop on that track by Ben Radford on investigating the paranormal (details below). I am very interested in the methods used for this kind of thing as they differ a great deal from the kind of work that I do.

Last year most of Skeptrack was streamed live, but there were some problems. I believe they will try this again this year with, hopefully, fewer headaches. If so, I will provide a link on Facebook and Twitter as soon as it is available, which might be as late as the first day.

If you are attending or planning to stream it live, here is where and when you can find me:

Skepticism, Scams, & Consumerism: Ranging from psychology behind sales to consumer rights, we discuss how as skeptics we identify and challenge dodgy products and pseudosciences.
Time: Friday 2:30pm – 3:30pm
Location: Hilton 205/206/207
I will be moderating this panel and talking about the psychology of purchasing behavior. Panelists include Matt Lowry, Rachael Dunlop, Richard Saunders, & Tom Merritt.

The Calculus Diaries – Lose Weight, Win in Vegas, Survive a Zombie Apocalypse: Fun examples of math applications in the real world and why it’s important to understand even just the basic concepts.
Time: Friday 5:30pm – 6:30pm
Location: Hilton 202
This panel is on the Science Track and will mostly be the work of Jennifer Ouellette, whose book with that title will be released August 1st, just in time for me to read it on the plane. As a statistician, I hope I can contribute something worthwhile to the discussion.

How Your Brain Works, and How to Fool It: Our perception of reality is driven more by expectation, belief, and desire than by sensory input. An examination of how we fool ourselves.
Time: Friday 8:30pm – 9:30pm
Location: Hilton 202
NOTE: This is not the original title and I think it is a little misleading. The original title was “What You See Is [not always] What You Get (or WYSInaWYG)”. The topic is not about how to fool your brain, but how your brain fools you. I believe Jason Schneiderman plans to join me and would be a welcome addition.

Skepticism and Sexuality: When do we get skeptical about sex, the media? When alien cults want to save African clitorises, this panel is here to discuss the facts.
Time: Friday 10:00pm – 11:00pm
Location: Hilton 205/206/207
I will bring with me some of the most interesting myths about the psychology of sex, courtesy of my friend a colleague who teaches the best course on the topic evah. My fellow panelists include Heidi Anderson, Ben Radford, Kylie Sturgess, & Ginger Campbell. Desiree Schell will moderate.

Naturally Skeptical? The Psychology Behind Being a Skeptic: A round table discussion on the factors that do (and don’t!) contribute to becoming a questioner of the paranormal and pseudoscientific.
Time: Saturday 10:00am – 11:00am
Location: Hilton 205/206/207
I will once again take the reigns as moderator and lead a discussion of the psychology of critical thinking and open-mindedness. I am very pleased that Scott Lilienfeld has decided to join us. He has written quite a bit about pseudoscience in our field, including his latest collaboration, 50 Great Myths of Popular Psychology: Shattering Widespread Misconceptions about Human Behavior. Other panelists, who are not exactly chopped liver themselves, include D.J. Grothe, Matt Lowry, Kylie Sturgess, & Pamela Gay.

Mathematical Modeling Pitfalls: Model don’t always accurately reflect messy reality, particularly where human behavior is concerned. Algorithms can only mimic human behavior, and there is a lot of room for bias and error as a result.
Time: Sunday 11:30am – 12:30pm
Location: Hilton 205/206/207
I am not actually on the schedule for this, but Jennifer Ouellete invited me to contribute when I mentioned my love-hate relationship with modeling and my frustrations in teaching that models are not what they model. If I get it together in time, I will present what I think is a cool visual of what you can do with a simple model that shows how they can be effective science. Well, I think it’s cool, anyway!

Raising Skeptical Geeks: A few known skeptical parents talk about issues and how they a raising their kids to be better rational thinkers.
Time: Sunday 2:30pm – 3:30pm
Location: Hilton Crystal Ballroom
This is sure to be the highlight of my Dragon*Con experiences as I love to talk about my kids. I am collecting stories and funny quotes to share. I am also thrilled that fellow panelists will be three good friends and one of the geekiest dads (by all appearances) around: Daniel Loxton, Heidi Anderson, Desiree Schell, & Adam Savage.

Skepticism and Education: JREF now has a Director of Educational Programs – what else is being done out there and how can skeptics help educate the next generation?
Time: Sunday 4:00pm – 5:00pm
Location: Hilton 205/206/207
Of course this is on my list of favorites as well as one of the most important of the panels on which I will serve. Other panelists include D.J. Grothe, Michael Blanford, Daniel Loxton, Pamela Gay, & Matt Lowry. Kylie Sturgess will moderate.

Women: Myths, Feminism And Skepticism: Puzzled by feminine mystique? Searching for your ‘Inner Velma’? Join investigators on gender, pop-culture and what science REALLY tells us!
Time: Monday 2:30pm – 3:30pm
Location: Hilton 205/206/207
Oh, so many myths, so little time! This topic is rich and Heidi Anderson, Desiree Schell, Pamela Gay, Kylie Sturgess, and myself are looking forward to a fact-packed dicussion.

Skeptrack Sign-Off Wrap Up and Feedback: Join the skeptrack guests and speakers for a discussion about how things went, last minute news, and how we can make things better next year!
Time: Monday 4:00pm – 5:00pm
Location: Hilton 205/206/207
Unfortunately, I will not make this discussion and also make my flight home. I am sure, though, that there will be much greatness in the room.

That wraps up my obligations. Here are just a few the events I am looking forward to attending as an audience member (if possible – the * indicates an event I cannot attend due to a scheduling conflict, but recommend):

CSI: Paranormal
Time: Friday 1:00pm – 2:00pm
Location: Hilton 205/206/207
Presenters/Panel Members: Joe Nickell

*Skeptically Speaking Live!
Time: Friday 8:30pm – 9:30pm
Location: Hilton 205/206/207
Presenters/Panel Members: Desiree Schell

Skeptical Coffee Talk
Time: Saturday 8:30am – 9:30am
Location: Hilton 205/206/207
Presenters/Panel Members: James Randi, D.J. Grothe, & Joe Nickell

Paranormal Investigation Workshop
Time: Saturday 1:00pm – 3:30pm
Location: Sheraton
Presenters/Panel Members: Ben Radford

*I Very Much Doubt That!
Time: Saturday 1:00pm – 2:00pm
Location: Hilton Crystal Ballroom
Presenters/Panel Members: James Randi

*Mystery Investigators Children’s Show
Time: Saturday 2:30pm – 3:30pm
Location: Hilton 205/206/207
Presenters/Panel Members: Richard Saunders & Rachael Dunlop

Monster Talk Podcast Live
Time: Saturday 4:00pm – 5:00pm
Location: Hilton 205/206/207
Presenters/Panel Members: Blake Smith & Ben Radford

Skeptic Zone Live!
Time: Sunday 7:00pm – 8:00pm
Location: Hilton 205/206/207
Presenters/Panel Members: Richard Saunders, Rachael Dunlop, Kylie Sturgess, & Brian Brushwood

Skepticism 2.0: Blogging
Time: Monday 11:30am – 12:30pm
Location: Hilton 205/206/207
Presenters/Panel Members: Daniel Loxton, Rebecca Watson, Brian Dunning, Rachael Dunlop, & Kylie Sturgess

Martial Arts Mysticism
Time: Monday 1:00pm – 2:00pm
Location: Hilton 205/206/207
Presenters/Panel Members: John Clements

You can find the full schedules as they become available on the track websites. I hope to see you there!