Note: This post also appears on Insight, the official blog of the Skeptic Society.

A few months ago I wrote about the psychology of vaccine denial. In the post I discussed two publications, one of which (Nyhan, et al.) found:

Corrective information reduced misperceptions about the vaccine/autism link but nonetheless decreased intent to vaccinate among parents who had the least favorable attitudes toward vaccines. Moreover, images of children who have MMR and a narrative about a child who had measles actually increased beliefs in serious vaccine side effects.

None of the interventions increased parents’ intent to vaccinate.

Then, a couple of weeks ago, a friend sent me a link to this piece describing research which seems to contradict that finding. The authors (Horne, et al.) concluded that

…highlighting factual information about the dangers of communicable diseases can positively impact people’s attitudes to vaccination.

These two conclusions seem to contract each other. Which should we believe?

Many times this question comes down to the quality of the research. In this case, I believe these are both fairly well-designed studies. One, however, is more precise than the other in several ways. I believe that precision highlights the complexity of the issue as well as giving us a better idea of the direction that vaccine promotion should take.

Let’s look at the differences in sampling and method between the two studies.

The Horne study sampled 315 men and women. In the Nyhan study, the final sample was 1759 parents with children under the age of 18. In most research, 315 subjects is more than sufficient and more is not always better. The danger in larger samples is to find effects that are statistically significant, but not practically significant. However, when comparing conflicting findings, it is best to bet on the side of the larger sample.

Then there’s the question of limiting the study to parents. Although Horne compared parents to non-parents and found no significant differences in attitudes or effects, noise is noise. These two groups of people vary, and the attitudes of non-parents are not particularly relevant. Limiting the study to parents would give me more confidence in the robustness of the findings and their application in real-world activism.

Still, if both are reasonably well-designed studies by competent researchers, the end results should not contradict each other. So there must be more going on. And there is.

For one thing, this is a great example of how complex social sciences are. We should never make policy decisions based on a single study and this demonstrates why. Replication, especially with variants of measures and materials, is essential to learning the best methods of persuasion.

For another, these studies differ in more than just sampling techniques. The Horne study is much simpler and, in fact, oversimplifies. Nyhan, et al. included three outcome measures, each addressing a specific attitude:

1. The belief that vaccines cause autism.
2. Perceived risk of side effects from vaccines.
3. Intent to vaccinate one’s child/children.

By contrast, the Horne study involved a single measure which combined answers to five specific questions (such as “I intend to vaccinate my child.” and “Doctors would not recommend vaccines if they were unsafe.” to come up with a more vague “vaccine attitudes” scale. Even if the answers to these questions are highly correlated, how interventions affect those answers may be very different. They certainly were in the Nyhan study. And if “effective” is defined as increasing intent to vaccinate, then the Horne study does not answer the question it purports to answer. Personally, I am more interested in intent to vaccinate than I am in any other aspect of “vaccine attitudes”, so the Nyhan study’s findings are much more meaningful to me.

In general, it is best to measure outcomes of interest as specifically as possible, but of course the more outcomes a researcher studies, the larger the sample must be.

Finally, and perhaps the most important difference between these two studies, is the timing of the experimental portion. When measuring the effect of treatments or interventions on attitudes, an experiment should be spaced over time. A researcher will measure the attitude, then wait before applying a treatment and measuring the attitude again. When polled about attitudes, those attitudes are brought to mind. This affects our receptiveness to relevant information in complex ways, ways that vary based on a number of other factors such as the strengths of our attitudes and the way the questions are worded. However, allowing subjects to forget about the initial survey provides a more accurate picture of how people confronted with information in the real world may respond to it.

The Horne experiment was conducted a day after the initial screening while the Nyhan experiment occurred about two weeks after initial screening.

My conclusion? I think the issue is complex, but while Horne’s findings appear easier to understand, Nyhan’s findings are more specific, answer more interesting questions, and can be more easily viewed within the framework of well-established knowledge about human decision-making (e.g., cognitive dissonance).

That, and we need more research if we are to develop effective ways of increasing vaccination rates.

Horne Z, Powell D, Hummel JE, & Holyoak KJ (2015). Countering antivaccination attitudes. Proceedings of the National Academy of Sciences of the United States of America, 112 (33), 10321-4 PMID: 26240325

Nyhan B, Reifler J, Richey S, & Freed GL (2014). Effective messages in vaccine promotion: a randomized trial. Pediatrics, 133 (4) PMID: 24590751

I don’t know if this could really be called “new”, but it’s a form of anti-intellectualism that usually goes unnoticed. I find it particularly frustrating because I so often see it often among people who claim to respect knowledge, education, and expertise. It is an ironic lack of respect for that same knowledge, education, and expertise.

The Psychology of Vaccine Denial

I’m sure you’re wondering what I’m talking about here, so I will get to the point. Rebecca Watson wrote a short piece, published in Skeptical Inquirer Online, that seems to question the potential effectiveness of a bill currently in the works in California which would eliminate non-medical exemptions for vaccination requirements to attend public school. I say “seems to” because it’s actually unclear.

My main criticism of the piece itself is not that much of what she says is blatantly wrong, but that the piece doesn’t go anywhere and the research cited doesn’t support the weak, barely identifiable thesis at all. It is disjointed and doesn’t flow well. The transition from the topic of education to that of the bill is a huge leap. Her conclusion makes little sense given the rest of the piece. It’s only a few paragraphs (very short for SI), but in those few paragraphs she manages to treat some important research shallowly and selectively, missing the valuable knowledge that a nuanced look at the findings would provide. I won’t make that mistake here.

She cites two articles, the first mention is this:

Researchers Brendan Nyhan and Jason Reifler have spent the past few years conducting studies that seem specifically designed to depress science communicators. Last year, they published a paper in which they showed that correcting myths about the MMR vaccination actually decreased a parent’s intention to vaccinate.

What’s missing is that this was only true among those “with the least favorable vaccine attitudes”.

Even showing participants images of sick children was counterproductive, increasing their belief that vaccines are connected with autism.

Yes, but the “even” part is very misleading. Emotional pleas such as describing disease risks and showing images of or telling stories about children with diseases all increased this belief, but education refuting a link successfully reduced that same belief. 

From the article:

“Autism correction” is most effective in reducing agreement with the autism misperception. Strong agreement declines from a predicted probability of 8.9% to 5.1% (and likewise for other response options). By contrast, the predicted probability of strong agreement increases to 12.6% for “Disease images.” Similarly, the predicted probability of believing serious side effects from MMR are very likely increased from 7.7% among control subjects to 13.8% in the “Disease narrative” condition.

This combination of results tells us a lot about what is happening when people are confronted with different strategies, yet nothing Watson wrote went beyond the few bits she selected from the abstract.

For the second citation, Watson writes:

Last month, they conducted a similar test using the common belief that the flu vaccine causes the flu. The results were the same: correcting the misconception only decreased the subjects’ self-reported intention to get vaccinated.

But this is what the article’s abstract actually says:

Corrective information adapted from the Centers for Disease Control and Prevention (CDC) website significantly reduced belief in the myth that the flu vaccine can give you the flu as well as concerns about its safety. However, the correction also significantly reduced intent to vaccinate among respondents with high levels of concern about vaccine side effects – a response that was not observed among those with low levels of concern.

After reading the article, I can tell you that the education measures worked across the board–in every concern level, educating people about the vaccine significantly reduced belief in the myth. However, those with the most concern about side effects dug in when it came to intent to vaccinate–not everybody, those with the most concern. (BTW, they didn’t conduct the test “last month”.)

These are finer points, but they are far from trivial. The details are what tell us what’s going on. I would not expect someone without an education in psychology to recognize the implications, although Abbie Smith, who reviewed the first study in a blog post last year, managed quite a bit of insight (which she wrote about with care, describing the findings in detail and not speculating beyond the what happened in the study).

And this is where the anti-intellectualism is most apparent in Rebecca’s piece:

At this point, we can only guess as to the reason why this happens.

No, Rebecca, at this point, you can only guess. So, if you don’t know why it happens, then nobody does?

To anyone who has studied decision making, reason, attention, or just about any area of social psychology for a few years, this statement is absurd. The pattern of results found in these two studies is exactly what I would have predicted. We have decades of research telling us why this happens.

Some brief explanations are provided right there in the articles’ discussion sections. The authors mention loss framing, danger priming, and other effects.

For a much more in-depth look, I will ask you to read Mistakes Were Made, But Not By Me by Carol Tavris and Elliot Aronson. The nutshell is that people do all sorts of mental gymnastics to reduce something we call cognitive dissonance–a tension between contradictory attitudes or an attitude and a behavior–in ways that allow us to avoid changing the behavior or strongly-held attitude.

In this case, when those holding strong anti-vaccine attitudes accept that their expressed reasons (e.g., autism) for those attitudes are invalid, they simply find another reason to maintain the attitude (e.g., side effects).

People are invested in the choice not to vaccinate, not the reason for the choice.

This would be especially true for those who have acted on that choice. The alternative is to accept that they have put their children at risk for no reason.

So, although Watson is not incorrect in reporting that some approaches backfired, she failed to see or report the nuances in these findings that tell us why and what we might do about it. And there’s more that I would not expect a layperson to recognize.

These studies only measured attitudes immediately following education–education that worked in dispelling myths about those vaccines. What I would like to see is follow up research examining attitudes months or years afterward. What happens, for example, when people are educated, then given time to change their attitudes without threat to their egos and identities? I predict that a large portion of them will change their minds. Much of the resistance is probably rooted in ego threat. Giving people time and space may allow them to save face while changing the attitude to reduce the cognitive dissonance associated with the conflicting ideas.

Furthermore, in these laboratory studies, parents are asked to report their attitudes prior to the exposure to materials. This is a form of declaration, committing people to a viewpoint that they then feel compelled to defend. That’s not what happens in real world situations.

So her next paragraph…

Do people hold their anti-vaccination beliefs so deeply that correcting a misconception only encourages them to spend time digging around for another reason to hate vaccines? If so, then the answer may be to address the underlying reasons for the belief instead of the scientific facts.

How are these two sentences connected? That people find other reasons to maintain a behavior or attitude is not evidence that there is some hidden reason. And expressed reasons are precisely what what are addressed in the studies she cited. She’s come full-circle with nothing at all to show for it.

Cognitive dissonance and the unconscious strategies people use to reduce it are human nature. We cannot “address” human nature so easily. We can educate people about human nature and how it does not always lead us to the best decisions to meet our goals (and by “we”, I mean people who have studied human nature, such as social scientists with years of training and knowledge), but of course that’s a much broader goal. Increasing vaccination rates is a public safety issue that must be addressed with more urgency and specificity.

Finally, all of this came down to this one guess of hers:

For instance, perhaps the belief is rooted in a fear of government control over individual choices.

Um, seriously? This came out of nowhere as if she just didn’t have an ending to her story, or perhaps couldn’t come up with a good segue to get to the one thing that she meant to talk about: the California bill that may eliminate personal belief exemptions for unvaccinated kids to attend public schools. Rebecca’s logic is that if fear of vaccine harm is actually rooted in fear of government control, then the bill might make matters worse.

This is a huge leap. For one thing, she cites no research showing that fear of government control has anything to do with the average vaccine denier’s choices. Even if it did, the very research she cited shows that removing all government involvement in vaccination would not change intent to vaccinate (those with the most concern would simply find another thing to worry about). But more importantly, she begs the question:

But will the law (which already exists in West Virginia and Mississippi) only encourage the anti-government anti-vaccine activists to band together and renew their efforts to fight for their freedom to harm innocent kids?

Well, now, why not take a few minutes and do a little research to find out how laws affect vaccine rates?

According to Pew,  “states with the strictest immunization laws tend to have the highest immunization rates” (they have a nice graph sorted by vaccination rates and Mississippi is at the top).

Not surprising. People tend to follow the law, and if they want to send their kids to public schools, they must vaccinate. But does this change attitudes? I think a lot of people would say that they don’t care, as long as it changes the behavior, but I think we can all agree that changing the attitude would be best.

And stricter immunization laws will change attitudes and beliefs about vaccines. How do I know this? Simple: cognitive dissonance.

One early finding in studies of cognitive dissonance theory is that it is often easier for people to change an attitude than it is to change a behavior. We have seen numerous examples of this, not only on laboratory studies, but in real-world behaviors such as smoking and exercise habits. Once invested in a behavior, the attitude follows as a matter of reducing the tension because we are invested in the behavior, not the reason for the behavior.

In this case, the biggest thing currently in the way of attitude change is the personal belief exception. Remove that, and behaviors must change. Once behaviors change, attitudes will follow, especially with education, which will pave the way for attitude change by giving parents a way to engage in the behavior (of sending their child to public school) without dissonance. This is especially true when the parent has not declared their attitude prior to education, as they do in a laboratory study.

What the research cited suggests, when included in the context of decades of psychological research about the relationships among attitudes, behaviors, and values, is that a combination of stricter laws and education correcting myths about vaccines is not only highly likely to increase vaccination rates, it will also decrease perceptions of risk of harm from vaccines.  Giving people facts does indeed work. It works to educate people about facts. If you want them to change their attitudes, however, you need to dig a little bit deeper.

To head off what will surely be a the first thing Watson’s supporters will point out: what’s the difference we came to the same conclusion? Her argument is “I think we should try X because nothing else seems to work” and mine is that X is what the science suggests. Only one of these is a valid argument. The ends do not justify the means.

The New Anti-Intellectualism

The implication that just anyone can write about this stuff with authority is the kind of anti-intellectualism I’m referring to in the title.

And before you assume that I am saying that skeptics have nothing to say, think again. Pseudoscience and fraud, the core of skepticism, are not science. Skepticism is a field in and of itself, very distinct from science. It includes scientific thinking and it benefits, as every field does, from the products of science, but it is not science.

This piece is poorly researched, weak, and reads like a book report that someone started, put away, then suddenly realized it was due and wrote the rest while the other kids were watching a film in class. A big part of that is the fact that Watson simply does not know the field, something she has demonstrated repeatedly. Yet, knowing her track record in this area, CFI decided to commission and publish this. The poor quality of the piece is a side effect of overconfidence coupled with a lack of expertise, but it further points to a huge drop in standards by SI Online. Not that SI hasn’t published some misses in the past, but this sad little piece is just one of many lesser-quality articles recently appearing there, including one in which the author describes party/county fair psychics as harmless fun.

So, am I saying journalists and other non-scientists (e.g., skeptics) should never write about science? No, I am not.

It is fine for non-experts to write on topics when they do so with great care. I cannot stress this enough.

A non-expert can do a great job when they do a proper amount of research by talking with experts (rather than spending a few minutes Googling and picking sentences out of abstracts that one believes supports one’s already-formed opinion), when they discuss experiments and studies accurately without omitting important details, when they properly credit the sources of ideas and opinions, when they follow what they find rather than start with a conclusion and attempt to support it, and when they refrain from stating their personal opinions as authoritative. Watson rarely appears to do any of those things when she writes about science. She simply writes and speaks with an air of confidence and that seems to be enough to make some people think that she is clever and knowledgeable.

Good science journalism allows the researchers’ voices to be heard, not the author’s. Think about that.

I have written before about the dangers and hypocrisy of speaking and writing on topics which require expertise one does not have (here, here, and here, for example–two of which are also about Rebecca Watson). It’s actually a topic that has received a lot of coverage, from Massimo Pigliucci’s talk at TAM8 to articles by Daniel Loxton (yes, I’ve linked to them both before and for good reason). I expect to see this kind of thing all over the blogosphere, but to see it on the Skeptical Inquirer’s site is disheartening, especially on the heels of other pieces that fall far below their old standards.

As I stated in this post more than three years ago, we (skeptics in general) criticize Jenny McCarthy and Bill Maher because they don’t have the expertise to make the statements they make. We criticize “the Food Babe” and many, many others for the same reasons. We tell people not to take medical advice from a Playboy Bunny and a talk show host, yet we (skeptics again) give a microphone to a blogger to talk about the psychology of vaccine denial simply because she calls herself “Skepchick”? How is this justified?

Now, I am perfectly aware that many people don’t believe that psychology is a science or that expertise in the field is actually a thing. I deal with that kind of anti-intellectualism every day. But I am still stunned when I see such blatant disregard for it among people and organizations who wave the flag of “listen to the experts” when it suits their purposes. CFI, you should be ashamed.

This is an updated version of a post that originally appeared on the Woo Fighters website in 2010.

In what is now considered “the golden days” of skepticism, I experienced first-hand the power of grass roots activism. I will never know if or how my view of the world would differ if I had never taken that psychology class in my junior year of high school, but I am very, very glad that I did.

I was very clearly excited to be talking about skepticism in education in 2011 at TAM9. I brought three students that year and one of my former students (Dylan Keenberg) spoke as part of the Sunday Paper session.
– photo by Dean Baird

When people ask “When did you become a skeptic?”, I have to answer that I have always been one. I never blindly accepted claims and I always looked for evidence. I held my beliefs tentatively. Where I went wrong was in the assumption that the “default” conclusion should be to consider a claim true unless the evidence refutes it. I thought that a lack of evidence meant that I could not draw a conclusion. I was naive and ignorant.

From a very young age, I was fascinated with psychic phenomena. I thought that ghosts were silly; Houdini made that clear. I had seen The Amazing Randi on The Tonight Show, so I knew that Uri Geller was a fraud and I never really bought into the typical magic tricks, anyway. But I was obsessed with ESP (extrasensory perception) and numerology. I had many obsessions, but these were different because I was never satisfied. I read about “cosmic twins” and the predictions of Nostradamus. I studied palm lines. I tried to move things with my mind. I made a set of Zener cards and did my best to test myself and my friends. Nothing ever panned out. Yet it did not occur to me to seek alternative explanations.

A more appropriate question than “When did you become a skeptic?”, I think, is “When did you stop believing?” And my answer to that is in October of 1982.

My high school in the greater Sacramento area did not offer Latin, but it did have an introductory psychology course and Mr. Tamblyn (now Dr., I see) managed to cover more than I see in most college-level courses. We recreated Asch’s conformity trials with students from other classes. We learned about the Stroop Effect. But what he and a grass roots skeptic taught us about critical thinking was the most valuable of gifts.

In early October, we had a guest speaker. She was a psychic. She gave several cold readings, including one of me. She said that she saw me sitting at a piano. Now, I didn’t play piano at the time, but I had wanted to learn since I got my first organ (they were very popular in the 60s and 70s) at the age of four. My parents hinted that we might finally have space for piano (they gave me an electronic keyboard that year). I was convinced that she was tapping into some unseen energy. She read several other people and we were all suitably amazed.

About a week later, another psychic visited us. He surveyed the class, asking how many of us believed in psychic phenomena, and about 3/4th of the students raised their hands.

He did several cold readings, some amazing mind-reading card tricks, and a few other feats. He entertained us us for about an hour. Then he polled the class again. Only a few did not raise their hands this time.

At this point he stopped cold and said, “I am not psychic. I am a magician. Everything I have done today has been a trick.”

He showed us how he did a few of the tricks. He explained the method of cold readings. We discussed the way the psychic the week prior may have accomplished what she did. At one point, I looked down at the books sitting on my desk and noticed that I had doodled all over one the paper covers – a piano keyboard. I also carried a key ring with a note-shaped fob. I don’t know if either was visible when she was there, but it was not inconceivable. Together, we produced an explanation just about everything that we’d been amazed by the week before.

What he had to say next had a much greater impact, though. In fact, it was the end for me. It was the information I needed to finally let go of the nagging question about whether supernatural abilities were real.

He told us that he and a few others had founded a group called Bay Area Skeptics.

He told us about the challenge.

Bay Area Skeptics had been founded in June and operated, at that time, as a local chapter of The Committee for the Scientific Investigation of Claims of the Paranormal (CSICOP), now known as the Committee for Skeptical Inquiry (CSI). Bay Area Skeptics offered a $1,000 reward to anyone who could demonstrate supernatural powers. Although this group and challenge was new, James Randi had been offering a reward (which by that time was $10,000) since 1964. The fact that nobody had claimed this money after nearly two decades told me everything I needed to know. The money was there. All they had to do was show their powers.

This seemed utterly ridiculous to me and still does to this day. I concluded that the odds that psychic abilities existed were very, very low.

So I let go.

Some students were pretty angry about the ruse and the final poll revealed that a few (I think there were 2 or 3 out of about 45) remained believers, but many of us were amazed. Amazed at our own willingness to see what we wanted to see. Amazed at how skilled both the psychic and the skeptic were. Amazed at how little we knew about the evidence (or lack thereof).

It was not until well into college that I fully understood that the appropriate “default” conclusion was the null, but what happened on that fall day in 1982 was a foundation for that concept. It also taught me that what we do not see can be just as important as what we see.

Although I remained an ardent skeptic, over the years my participation in skepticism as a movement varied. At times I diligently maintained memberships in various organizations. I tried Mensa for a while, but was very disappointed to discover that their special interest groups for nonsense like astrology outnumbered the groups with a rational focus by about 5 to 1. Most other groups were either religious or game-focused. At times I paid little attention to issues of skeptical activism. Eventually, I kind of forgot about that day in psychology class.

Then while studying psychology as an undergrad and grad student many years later I had a mentor/professor whose office was filled with old Skeptical Inquirer magazines. Dr. Donald Butler’s courses in research methods and statistics were built around skeptical concepts. He reminded me that skepticism is the best lens through which to view the world.

In April of 2000, I attended the annual convention of the Western Psychological Association in Portland, Oregon. It was my first academic conference after returning to school in 1997. I found that it was not all that different from other types of conventions and conferences, but the talks were so much more interesting. I was thrilled to see Michael Shermer and Ray Hyman on the schedule and attended both of their talks. Shermer described the findings of his survey on religion (something I found particularly interesting since I had responded to that survey myself) and discussed his book How We Believe. Hyman’s talk was titled Science and Pseudoscience. As Dr. Hyman wowed the crowd with rope tricks and mind reading, all of the memories of that day in high school came flooding back, but I could not recall the skeptic’s name. Dr. Shermer and Dr. Hyman speculated that it was Bob Steiner, and James Randi offered the same guess later in an email. However, the internet eventually provided enough clues and I now know that it was Terence Sandbek, a clinical psychologist and professor at American River College.

The list of people who have taught me to appreciate skeptical thought and who have fine-tuned my philosophy is not a short one, but I do not know if I would have come to appreciate these people if it were not for the work of a skeptic and a high school teacher back in 1982. So, thank you Dr. Sandbek, for showing me that what appears to be an extraordinary feat is usually simply a practiced one and to Dr. Tamblyn, for showing me how easily we accept extraordinary claims without evidence. Oh, and for teaching me to drive! (Yes, he taught driver’s ed, too.)

I can think of no pursuit as rewarding and valuable as the study, promotion, and teaching of critical thinking, science, and skepticism.

Let me cut to the chase: No.

Full disclosure: I own a Fitbit Flex™. I haven’t worn it in months, but I do like the thing. It’s a long, boring, irrelevant story why I’m not wearing it, so I won’t get into that. Just know that I am sincere when I say that I like my Fitbit.

I don’t think my Fitbit is telling me exactly how far I’ve walked or how many calories I’ve burned. I do think that it gives me a ballpark figure that can be used to compare one day to other days. I don’t think that it’s telling me exactly when I am sleeping, either, but the information about how restless I am some nights compared to others is interesting and even sometimes useful.

I also like that it buzzes on my wrist to wake me up in the morning.

Recording some basic information about my daily activity, giving me a reason to record what I eat, waking me up in the morning, and reminding me to get off my ass. These are not big goals and perhaps I don’t need a $100 gadget to accomplish them, but is the device bullshit?

Mother Jones sure seems to think so. Last week the site posted a piece titled “Science Says Your Fitbit is a Joke“. When I searched for more reports on the topic I found a Jezebel piece titled “Your Fitbit is Bullshit, Says Science“, and one on RYOT titled “Science Says Your Fit Bit’s Full of It“.

Since I don’t particularly trust any of these sites as a source of accurately-interpreted science news, especially those which appear to be trying a little too hard to find clever, edgy headlines, I was skeptical.

The Jezebel piece is easily dismissed as simply lazy blogging. The author links to Mother Jones, paraphrasing one of the main points of the piece (bold mine):

Mother Jones points to a new study showing that your iPhone or device of choice does just a good a job[sic], if not a better one, at doing things like tracking calories and measuring activity.

Except that’s not what the Mother Jones piece said. It reads:

But, according to a new study published Tuesday in the Journal of the American Medical Association (JAMA), the apps on your smartphone do the job just as well, or even better—at least in terms of measuring your steps and your calories.

So Jezebel incorrectly paraphrased Mother Jones. And they aren’t the only ones. RYOT wrote this:

A new study came out showing that iPhone apps and other (much cheaper) devices do a better job than the bands at tracking your activity, steps, and even calories.

Even assuming Mother Jones is correct, these paraphrases are not. Walking is, of course, an activity, but it’s only one type of activity. Mother Jones was correct in noting that smartphones can do the work of pedometers, but Jezebel and RYOT are overstepping (pun intended). Wearables such as the Fitbit Flex™ and Jawbone Up™ can track your activity while you are swimming, bowling, or doing jumping jacks. Can your smart phone do that? Well, I suppose if you’re holding it in your hand, it can, but who does that? Also, if you don’t sleep with your smart phone strapped to your body, it cannot track your activity during sleep.

But is Mother Jones right? Might smartphones do a better job? Well, let’s find out.

Although Jezebel linked to the original source, a press release by University of Pennsylvania School of Medicine, it does not appear that the author read the press release, nor did the author of the RYOT piece, who linked only to the press release (failing to even give Mother Jones credit for the thoughts expressed). There is no reference to smart phones doing a better job at anything, much less a list of activities. The only study which is discussed is the one described in the Mother Jones piece, which examined the accuracy of several devices at counting steps and found all to be similar.

But let’s get to the bottom line. Is it a fair assessment that science says these devices are bullshit? A Joke? Full of it?

First, the study discussed involved counting steps. That’s it. There is absolutely no comparison of different devices in regard to tracking calories and no examination of other activity or reports such as distance walked. The study was fairly well-designed–something that does not come across in the reporting. Although there were only 14 participants, each wore every device at the same time and each walked both 500- and 1500-step trials twice.

Here are the results from the 500-step trials:

The results of the 1500-step trials are nearly identical. Now, it appears that the Fitbit One™ and the Fitbit Zip™ are spot-on, and the fact that the error bars are practically non-existent is pretty impressive. Unfortunately, the authors failed to do any sort of statistical analysis at all. Instead, they stated these results:

We found that many smartphone applications and wearable devices were accurate for tracking step counts. Data from smartphones were only slightly different than observed step counts, but could be higher or lower.Wearable devices differed more and 1 device reported step counts more than 20% lower than observed.

Given the error bars for the rest of the devices, including the Fitbit Flex™, I’d believe that most did not differ from one another significantly, with the possible exception of the Nike Fuelband™, which apparently sucks. However, there are two very clear winners, with the Digi-Walker™ close behind. A simple statistical analysis would have confirmed this.

Regardless, it is very clear that smartphones do not outperform most of the wearables. What is really in question is whether people use the devices and use them to their advantage. This may be questionable, but it has hardly been decided by science and it has certainly not been decided in favor of tossing one’s Fitbit™. The JAMA article concludes:

Increased physical activity facilitated by these devices could lead to clinical benefits not realized by low adoption of pedometers. Our findings may help reinforce individuals’ trust in using smartphone applications and wearable devices to track health behaviors, which could have important implications for strategies to improve population health.

So there’s hope that devices, whether they are wearables or just smart phones, will translate to improved health for at least some users.

To be fair to Mother Jones, their piece is much more fleshed out than the other two, covering more than this one study. In my opinion, the Jezebel and RYOT pieces simply plagiarized Mother Jones. However, I saw nothing in the rest of the piece to warrant such harsh criticism of wearables.

Now after writing most of this I came across this wonderful piece on Mashable that was posted yesterday. I could have simply linked to it, saying “read this” because it’s pretty much what I have said, right down to a comparison to the game of “telephone” that I just edited out of this post.

In the end, I have to agree with its author, Chris Taylor, who opens with:

There are poorly designed scientific studies, and then there’s poor reporting on scientific studies by journalists who should know better.

Unless you’ve been living under a rock, you’ve probably heard that some of the people who stood in line for hours to be the first to get an iPhone 6 Plus pulled those phones out of their pockets a day or two later to find them bent. If you haven’t, I suggest you go to Twitter and search #bendgate. Or Google “bendghazi”.

When it became clear that this was no hoax (a video surfaced showing that the phone can indeed be bent using bare hands; you can watch it below), a Facebook friend made a joke about this not being a case of Uri Geller resuscitating his career.

Then, this morning, I found this post on a site called Telepolis, claiming an exclusive interview with Geller on the topic.

TELEPOLIS: Mr Geller, shortly after the release of the new iPhone6 you started bending them. What’s the reason?

Uri Geller: I had a secret deal with a competitor of Apple.

Which is cute, but is it satire?

TELEPOLIS: The skeptics dispute the use of your special powers. Some people managed to bend the iPhone6 by using their hands. There is a video on YouTube … There is a video on YouTube …

Uri Geller: My answer to the skeptics is this: I don’t need to use the power of my fingers to bend metal because I only use the power of my mind, and I am doing very well with that. As a matter of fact please go to my website urigeller.com and please see how I can also bend footballs! And by the way, while you are on my website, please also watch the documentary by the BBC called The Secret Life of Uri Geller there you can see how I also bent the CIA!

Oh, yes, clearly satire. And not really all that funny, although I laughed at the original joke.

But then this afternoon a friend sent me a link to a MarketWatch post claiming to have talked to Geller.

“There are two possible explanations,” Uri Geller, the psychic illusionist famous for bending spoons with his mind, told MarketWatch. “Either the phone is so seriously thin and flimsy that it is bendable with mere physical force, which I cannot believe given the extensive tests Apple would have done. Or — and this is far more plausible — somehow the energy and excitement of the 10 million people who purchased iPhones has awakened their mind powers and caused the phones to bend.”

He even urged Apple to hire him, saying that he could explain that the problem is not Apple’s fault. He also said:

I don’t own an iPhone 6 — I’m loyal to my BlackBerry BBRY, +0.10% and would never change — but if I did I have no doubt I could bend it with my mind.

Well, pardon me for being skeptical.

Oh, I’m not skeptical that he could bend it. I do think it’s a bit unreasonable to expect a phone to be both light AND indestructible, although iPhone users have become accustomed to reasonably durable devices. But with his mind? Notsomuch.

James Randi, Banacheck, and many others have done a great job of showing that bending spoons, keys, and other things can appear to be magic when it’s really nothing more than a magician’s trick. Still, I’d like to see Geller try under controlled conditions.

If you are not an educator, please help spread the word about this wonderful opportunity.

If you ARE an educator, would you like to bring more skepticism and critical thinking into your classroom? Would you like to be inspired, energized, and informed? The Amaz!ng Meeting 2014 is a great place to meet other educators, gather materials (including printed copies of the JREF’s education modules for classroom use), pick up tips, and be inspired.

The Amaz!ng Meeting is attended by people from all walks of life and all over the globe. Speakers include scientists, philosophers, journalists, educators, activists, and even entertainers. Simply put, The Amaz!ng Meeting is James Randi Educational Foundation’s yearly celebration of science, education, and critical thinking. Educators who attend TAM will be able to bring what they have learned into the classroom.

In addition to three days of superb talks and panel discussions, TAM 2014 offers a full day of workshops, including two which will focus on incorporating skeptical thinking lessons into classrooms. This year’s theme, “Skepticism and the Brain” promises to be especially valuable to educators. And the JREF would like to help you join in!

In an effort to expand our promotion of education and the development of future critical thinkers, the JREF established the TAM Teacher Scholarship Fund in 2013. As many readers know, I am an educational programs consultant for the JREF and will be responsible for this project. The fund will pay the registration fees for a limited number (to be determined by donations received by June 15, 2014) of educators to attend The Amaz!ng Meeting 2014 in Las Vegas, Nevada, July 10-13.

The best news is that due to the generosity of the skeptical community, we are already on our way to awarding grants and we have a pledge from one donor, Brian Walker, to send at least TEN more teachers to TAM 2014!

Details regarding eligibility, how to apply, and what to expect can be found here.
If you would like to help send teachers to TAM 2014, you can do so here. Every little bit helps! Donations made after June 15, 2014 will be distributed to TAM 2015 grant recipients.

At The Amaz!ng Meeting 5, James Randi announced that future meetings would be held in the summer to ensure that more educators would be able to attend. However, the costs associated with large conventions are still a barrier to many. Teaching is a notoriously under-paid profession and, while many schools provide funds for professional development, those funds are often inadequate and their use is usually restricted.

The JREF would like to relieve some of that burden, and you can help.

The TAM Educator Grant is now accepting donations of any size (every little bit helps!). Like all donations to the JREF, these are tax deductable. However, these donations will only be used for the purpose of sending teachers to The Amaz!ng Meeting.

Please visit the website for more details about this very worthy cause.

If you are a teacher, or studying to become a teacher, and would like to apply for one of these grants, the details and a link to apply can also be found here.

Special thanks to volunteer Dale Roy of Granite State Skeptics for her efforts on this project.

I hope I grabbed your attention with that title, but do not expect to find the answer to that question here. What I am going to discuss today is a study that many people seem to think answers that question, but it doesn’t.

As I noted in my last post, the study I’ll be discussing was grossly misreported, starting with its press release. Since the study itself appears to be behind a pay wall for most people, I’ll describe as much detail as I can in a blog post as I discuss the study’s validity and findings of the study, published in the Journal of Social Psychological and Personality Science and titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.

But for those who are not at all interested in the research methods or a breakdown of why I rate the quality of the study the way I do, I will give you the the bottom line so you can skip the rest or only read the sections that interest you (I’ve used headings to make it easier).

Summary

I think that the findings will hold up to replication, despite some issues I have with the way they did a few things. Overall, the research quality is quite high.

The groups they compared did not include atheists, agnostics, believers, non-believers, highly religious, or any other label that you can throw at it. In the studies they used raw religiosity scores and made some comparisons of “higher” and “lower” using values from the distribution. In a sense, the compared those who scored in the lower half of the sample to those who scored in the upper half.

They found:

• Differences in prosocial behavior cannot be dismissed as due to political affiliation, socio-economic status, or other factors often held up as responsible.
• Religiosity is correlated with trait compassion; the more religious, the more compassionate.
• Trait compassion is related to prosocial behavior in general. This relationship is stronger in the less religious than in the more religious. This does not mean that the less religious are more compassionate (see number 1) or that the less religious are more prosocial. It just means that compassion is a bigger factor in prosocial behavior in the less religious.
• The findings of the first study can be interpreted one way that isn’t discussed in the paper: when the relationship between compassion and religiosity is accounted for, the more religious are not more prosocial than the less religious.
• The findings in the second study, which involved inducing feelings of compassion, were similar for generosity, except that the more religious were more prosocial even after accounting for compassion.
• The findings of the second study also included a different pattern when the prosocial behavior was giving to charity. Compassion induced more giving, but the effect was weak and did not differ much across religiosity. Religiosity had a significant affect on charity. This can be explained by the guidelines provided by many churches for how much of one’s salary one should give.
• In the third study, in which state compassion (how compassionate the individual felt at that time) was measured and the prosocial behavior measure involved real-world cash, religiosity was not related to either compassion or prosocial behavior.
• In the third study, state compassion was positively correlated with prosocial behavior, but the effect was greater in the less religious than in the more religious.

What the findings as a whole say to me, and what I believe the press report tried, but failed, to express, at least with convincing support: We do not need religion to be prosocial. We need compassion.

This is great news for secularists.

However, it doesn’t say anything negative about religion or the religious, nor does it provide anything that should make atheists feel superior. It just shows that one can be good without God; that motivations can come from other sources.

Now on to the details…

NOTE: to keep this as short as possible, I’ve included a lot of links to terms and demonstrations. Where I describe problems in more detail I still water-down quite a bit. I will do my best to make it understandable without rambling on and on, but keep in mind that it takes many years to learn enough about research design and statistics to understand why some of these are problematic. Furthermore, not all researchers will agree on the consequences of some of these problems. I am still learning this stuff myself (probably always will be learning).

The Study (description)

The article reports three studies, each related to the relationship between compassion and prosocial behavior in less-religious individuals. I have created graphs using the information in the paper, but in some cases I did not have exact numbers, so while the relationships are visually accurate, there are only values where I could use exact numbers.

Theoretical Foundation

The introduction discusses research which documents that religiosity is associated with prosocial behavior. Specifically, religious people give more and volunteer more than nonreligious people, over and above what they give to and do for religious organizations. The researchers note that the nonreligious do give; when we compare groups, we do so using averages. However, it may be that the motivations for prosocial behavior vary in a way that interacts with religiosity. In other words, the more religious among us may be motivated to prosocial behavior by one set of factors and the less motivated by another.

The researchers hypothesized that compassion is a more influential factor in prosocial behavior for the less religious than for the more religious among us.

Study 1

The first study examined the relationships among religiosity and traits of compassion and prosocial tendencies. What this basically means is that situational factors were not involved; traits are a matter of personality or attitude. For example, “trait anxiety” refers to how anxious a person is in general, while “state anxiety” refers to how anxious that same individual feels in a given situation.

This study involved analyzing data from a 2004 “survey”. I put that term in quotes because it usually refers to a set of questions that do not measure more than what is apparent at face value. Established measures of latent variables (variables which cannot be measured directly such as feelings and attitudes) are usually called an “inventory” or “scale” and we refer to them loosely as “measures”. In this case, the survey involved such measures and I want to make that clear.

The sample was comprised of 1337 participants and covariates (variables other than those of interest which could explain differences among the groups) of gender, political orientation, and education were included in the analysis. The variables of interest were religiosity, compassion, and prosocial behavior. Religious identity (identification with a specific religion or no religion) was also considered.

Results

Correlations

• Covariates had little impact on the results.
• Trait compassion was positively correlated with religiosity* and prosocial behavior. On average, the more compassionate the individual, the more religious they were and the more the more prosocial they were.
• The relationship between religiosity and prosocial behavior was marginally significant (statistically).

Hypothesis Test (See Figure 1)

Figure 1: Study 1 results. "Higher" and "lower" are defined here as 1 SD from the mean.

• A regression analysis revealed an interaction of religiosity and compassion on prosocial behavior. What this means: The effect of compassion on prosocial behavior differed among levels of religiosity.
• More specifically, the level of trait compassion affected prosocial behavior less as religiosity increased.
• There was also a main effect of compassion, but that was apparent in the correlational analysis.
• There was no main effect of religiosity on prosocial behavior. This is interesting, because they found a marginally significant correlation, but it does not mean the there are no difference in prosocial behavior. I would interpret these findings, when put together, as suggestive of little or no difference between the more religious and the less religious in prosocial behavior over and above the differences accounted for by compassion.

The authors discuss the findings a little differently, though, focusing on the differences in the way that compassion affected prosocial behavior (the interaction in the first hypothesis test result) and ignoring the way that the effect of religiosity disappeared when compassion was entered into the equation. It seems more interesting to me to treat compassion as the moderator. It also makes more sense in the end.

Study 2

This study was experimental in that the researchers manipulated state compassion. In other words, they induced feelings of compassion in half of the participants and compared the amount of prosocial behavior those participants engaged in to the amount of such behavior in a control condition.

The sample included 101 participants and the study was conducted online, so the age range was exceptional (from 18 to 68 years). Participants were randomly assigned to one of the two conditions and each watched a short video under the guise that there would be a test of memory afterward.  The videos were established manipulations of feelings of compassion and neutral emotion (i.e., other researchers tested their effectiveness). Following the video, participants completed two tasks which are well-established measures of prosocial behavior commonly used in such research.

Results

• Again, covariates had little impact on the results.

Hypothesis Tests (See Figure 2)

Figure 3: Results of Study 3. Values are relative.

There were two tests since the participants completed to different prosocial tasks, one involving generosity and the other involving charity.

For the generosity task:

• This time there were a main effects of both religiosity and compassion on prosocial behavior. The more religious, the more prosocial. Those who watched the compassion-inducing video were the more prosocial on average than those who watched the neutral video.
• The interaction appeared again in the manner as in Study 1.

For the charity task:

• There were main effects of both religiosity and compassion on prosocial behavior.
• There was no interaction.

This is where they screw up, in my opinion.

The pattern of the moderation was in the predicted direction but failed to reach statistical significance.

This is not an acceptable statement unless the findings are marginal. This was not. The p-value was .408. This is not even close to meaningful. Still, they went ahead with the analysis of the interaction and reported an effect of compassion on charity for the less religious participants and no effect for the more religious.  The problem is that post-hoc analysis like this assumes that a significant interaction was observed. Their tests inflated alpha (the probability of a Type I Error) and can only mislead. They stated that they had found “partial support” for their hypothesis, but they did not in this case.

The relationships in the generosity task are very clear when we look at a Figure 2. The interaction is the interesting finding. Compassion had little effect on the more religious, but a very large effect on the less religious, who gave practically nothing when compassion was not induced. There is no analysis to tell us if the less religious surpassed the religious by a statistically significant amount when compassion was induced, but they were clearly out done by the more religious when not made to feel compassion.

The charity task showed no such interaction and the authors did not include a graph of this effect that I could recreate, nor did they provide the information to make one.

Study 3

For this study, the sample of 120 completed a state compassion inventory (a measure of their feelings of general compassion at the moment) and a series of “economic tasks designed to measure their generosity, trust, trustworthiness, and motivation to reward others’ generosity.” What differed in this study, however, was that the ‘points’ they earned in these tasks could be exchanged for cash at the end of the study. Participants did not know how much cash, but they knew that the more points they earned, the more cash they would receive.

Results

Hypothesis Tests (See Figure 3)

The findings of this study were very different from the other two.

• State compassion was not related to religiosity.
• Religiosity was not related to prosocial behavior.
• There was an interaction of religiosity and compassion on prosocial behavior. The amount of compassion felt had more of an effect on the behavior the less religious than it did on the more religious.

The graph of these findings, a reproduction of their graph since they did not provide information to create one that would make more sense (to me anyway), is a bit misleading. The values are z-scores, so they are relative to one another and not actual values. What is interesting, though is how little the prosocial score varied in the more religious group and how that line barely dips below the mean value (represented by 0).

There is also a problem with the press release in that it makes the claim that the high state compassion/less religious group out-performed the others. There is no statistical analysis comparing the groups in that way, so this is a misstatement. We do not know if less religious individuals are more generous than more religious when motivated to act prosocially. We just know that they are more generous when motivated by compassion than when compassion is low.

The Study Overall

As I noted, my opinion of the studies as a whole is relatively high, but I do have some major criticisms. Some of the language makes me cringe (e.g., results are the product of statistical tests, so “We tested our results”…), but I have seen more and more of this as scientific reports in general have grown sloppier.  Study design and method is much more important, as is the quality of the reporting beyond language.

The authors also throw around the term “robust”, claiming in the first study that the relationship between compassion and prosocial behavior is “particularly robust” for less religious individuals. That term refers to findings which are “sturdy” and will stand up when some supports are removed – effects which appear to hold up in different situations. Since this was one analysis of one data set, that term just doesn’t work. It does not fit in any of their uses of it.

In fact, they err in Study 2 by saying that the effect was “attenuated” for the more religious. That term is relative; attenuated compared to what? The effect was not “robust” in one condition and “attenuated” in another; they can only be compared to each other. The effect was greater in the less religious than the more religious.

Missing Information

There are a number of bits of information which are considered to be, at minimum, required for a good research report. A general rule of thumb for methods and results sections is to include enough (without being redundant) information to allow other researchers to replicate (in a strict sense) the study and to confirm that the statistical findings are properly interpreted.

I am not sure that this article meets that criterion. The methods are pretty well fleshed out and the paper is full of statistics, but some descriptive statistics are missing that I would have liked to have seen (e.g., means reported overall for measures, but not by group) and there was not enough of the right information to recreate them.

Grouping the Data and Errors of Generalization

One overall criticism which warrants discussion is in the grouping of data. There are some problems with this and they are related. The sensitivity of the religiosity measure is one problem that, by itself, is not a big target for criticism. Combined with the second problem of grouping participants, though, it becomes more serious.

The practice of comparing groups of people based on a variable which is distributed on a spectrum is a common one. The question the researcher wants to answer is important in deciding whether to group and, in this case, I do not disagree with that choice, but I question how they grouped and how it was communicated. If the data are clustered (the distribution is multi-modal), grouping is simplified, but if the data are distributed more loosely, it can be tricky and dangerous.

First, the researcher loses information, therefore they lose sensitivity and usually lose power. The sensitivity problem is relevant in the first study, but mostly because it makes the findings difficult to interpret.

Second, if the way that the grouping is communicated is not consistent and clear, it is likely to be misinterpreted, compounding any existing problems with the method. I discussed this problem in my last post. Most of the reports referred to the groups compared as “highly religious” verses “atheists and agnostics” or something like that. However, where are all of the people in the middle (i.e., most likely the bulk of the sample)?  Within each group there was variation in religiosity and comparisons are made using averages. Generalizing only works when the samples are representative of the population of interest and this applies in either direction of the generalization (i.e., specific to mixed or mixed to specific).

Third, researchers must decide where to draw the lines between high and low (and anything in between). Since the majority of variables in psychology are normal distributed (therefore symmetrical), the lines are usually drawn using rankings of sample values and the most common way to split a sample in half is to put all values above the median into “higher” and those below into “lower” (called a “median split”). However, ease is not a good reason to use this technique.   Here is an interesting demonstration of the dangers of dichotomizing normally-distributed variables.

But… religiosity is not usually distributed normally; it’s usually skewed. Skew means that it’s not symmetrical, so a median-split would make even less sense.

In this case, it seems that the authors tried to have the best of both worlds by treating religiosity as a continuous variable, but doing post-hoc analysis on it, discussing it, and graphing it as if it were dichotomous, choosing values which were one standard deviation from the mean in both directions as the central tendencies of each group. The biggest problem with this is the assumption of normality. If the variable is not normally-distributed (and I suspect that it is not), this grouping is a bit tough to swallow.

When this problem is mixed with a limited range as it is in the first study (the religiosity scale only had four points), it’s a problem. The four possible values were 1 = no religion, 2 = not very strong (religious identity), 3 = somewhat strong, and 4 = strong. Since the mean was 2.99, the bulk of the sample were fairly religious. one standard deviation (1.03) below the mean is not exactly in non-believerland and one above is off the scale (literally). It is just very difficult to see where “higher” leaves off and “lower” takes over.

Although the range is adequate in the other two studies, the problem of discussing groups which do not actually exist and have fuzzy definitions remains. In my opinion that is one of the reasons it was so misreported.

But, overall, the research is of a relatively high quality and interesting. I would like to see more variation in the prosocial tasks, given that the outcome of the charity task was so different from the tasks of generosity.

It seems that the less religious are at least as generous as the more religious, but their reasons for acting prosocially differ. I would like to see the day when, as a group, we are generous and prosocial consistently, without the need to be provoked and without needing to feel an emotional connection to the receiver.

Saslow, L., Willer, R., Feinberg, M., Piff, P., Clark, K., Keltner, D., & Saturn, S. (2012). My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals Social Psychological and Personality Science DOI: 10.1177/1948550612444137

It seems that the best motivation for me (to write) is frustration.

A press release by UC Berkeley about a study that was recently published on the relationships among religiosity, compassion, and prosocial behavior has been making the rounds over the last couple of days, waved by proud atheists as evidence of superiority and bashed by the more skeptical as bad science (even though most haven’t appeared to have read the study). The latter has been exacerbated by the fact that the majority of reports include very big mistakes.

I’m going to analyze the study, but I will post that separately since it is likely to be long and I also want to discuss the incredible mess of BS this has become. In that post I will also discuss the reasons some of the errors that seem minor here are actually quite serious.

I blame the press release. Unfortunately, as the culture of higher education becomes more consumer-oriented, strapped-for-cash universities must market themselves strategically, and scientists are forced to compete for funding, the integrity of science is diminished. I don’t know if the study’s authors were involved, but my guess is that the press release’s author is mostly responsible for spinning the findings, omitting important parts of the findings, and cherry-picking statements by the scientists to make it appear that their speculative explanations for those findings are solid conclusions. I think that spin, along with the fact that the findings involve an interaction (a notoriously difficult concept to grasp due to its non-linear nature), confused science writers, many of whom are not in the practice of reading studies and many of whom added their own spin to the mix.

The study, published in the Journal of Social Psychological and Personality Science, is titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.  The first error of the press release was made in its title: “Highly religious people are less motivated by compassion than are non-believers” by incorrectly describing the subjects of the study. It also errs in a few minor ways (e.g., describing the study as “three experiments” when one only one was experimental, one was analysis of existing data, and one was quasi experimental), but the biggest problems are practically criminal in the science world. The author correctly (with the exception of the description of the subjects) reported that the relationship between compassion and prosocial behavior was stronger among less religious participants than more religious participants, but omitted the findings which clearly showed that the more religious participants were, in general, more compassionate and generous overall.

When sloppy reporting took over, the result was an utter mess.

On the one hand, the authors claim in their introduction to be interested in what motivates less religious people to act prosocially, so perhaps the spin was the plan all along. However, if they were not interested in religiosity as a variable – if they were not interested in comparing the more religious to the less religious, then they should not have limited their study to the population of interest.

Something I found interesting is that the third paragraph of the press release makes a statement which should have sent red flags up because there was no follow-up that made sense. It started with:

The results challenge a widespread assumption that acts of generosity and charity are largely driven by feelings of empathy and compassion, researchers said.

And yet the rest of the piece focused on the fact that empathy and compassion DID drive generosity and charity in less religious participants, even if it repeatedly incorrectly referred to those participants as “non-believers”. If the author had included the findings that more religious participants were more generous, this would have made sense.

The author errs again with this circular definition of “compassion”, conflating the dictionary definitions of “compassion” and “prosocial behavior” with an effect:

Compassion is defined in the study as an emotion felt when people see the suffering of others which then motivates them to help, often at a personal risk or cost.

Compassion cannot be defined in a study as an emotion felt by someone. That’s a variable we can’t measure directly. In a study, we use operational definitions. In this case, compassion was defined differently in each of the three studies within the article.

The author continues the spin with statements such as (bold mine):

When they looked into how much compassion motivated participants to be charitable in such ways as giving money or food to a homeless person, non-believers and those who rated low in religiosity came out ahead: “These findings indicate that although compassion is associated with pro-sociality among both less religious and more religious individuals, this relationship is particularly robust for less religious individuals,” the study found.

What the bold suggests is that participants who fell into the “low religiosity” category were more prosocial and/or more compassionate. That’s not what the finding means. What was greater was simply the relationship between compassion and prosocial behavior. Because the “high religiosity” participants were more compassionate, they were also more prosocial overall (marginal significance – see discussion below).

Finally, this statement is grossly misleading:

Those who scored low on the religiosity scale, and high on momentary compassion, were more inclined to share their winnings with strangers than other participants in the study.

What participants do in a study is really not interesting. What their behavior tells us about how people behave in the world is. Because there was no statistical analysis comparing the “low-religiosity/high-momentary compassion” group to the other groups, this finding does not allow us to infer anything about the population of interest. You and I could complete a game of Scrabble with a final score of 102 to 103 (respectively), but I would not brag to my friends that I am the better Scrabble player.

Most reports of the study either posted the press release as-is or quoted large chunks of it. Headlines ranged from the simple and correct, if misleading, “Compassion may motivate faithful less” to the still incorrect, but closer “Confirmed: Atheists more motivated by compassion in charitable giving than believers are” to the blatantly incorrect statement suggested by, “Are Religious People Less Compassionate?”

Almost none of the reports describe the samples correctly. The samples were broken into two groups based on measured religiosity, a common practice in social psychology. Thus, there was a range of values and those who scored in the top half were considered “more religious” and the bottom half “less religious”. The “less religious” group cannot be described as “athiests”, “agnostics”, “non-believers”, or even “people low in religiosity”. The “more religious” group cannot be described as “highly religious”, either.

One site that I usually find more accurate contained a much more serious error than an incorrect sample description. It reported a finding not found in the press release that is actually the opposite of what was reported in the study:

Two other experiments also confirmed that more religious participants seemed to be less generous.

They should have stuck to quotes of the press release.

The worst by far, though was on a site called “PsychCentral“. The first paragraph reads:

A provocative new study from the University of California, Berkeley suggests highly religious individuals are less likely to help a stranger than less religious people.

Um. No. That’s not what the study suggests. At all.

and this a few paragraphs down:

Experts say the results challenge a widespread assumption that acts of generosity and charity are largely driven by feelings of empathy and compassion.

Why they added “experts say” to this is a mystery because it’s basically a lie, but this statement is particularly baffling when you consider that the progression of statements: 1) nonreligious are more generous 2) nonreligious are more motivated by compassion to be generous 3) results challenge the assumption that generosity is driven by compassion. On what planet does that make sense?

I’d lament that people don’t actually read, but the worst part is that someone actually wrote this. Sure, it’s mostly quotes and they made stuff up, but they changed some words and moved things around, so they had to pay attention to something.

A friend shared one decent report; it did not seem that the author read the journal article, but at least he read one report (on livescience.com) that is almost a duplication of the press release and put some thought into it. He noted that the findings reported an interaction – that generosity was more related to compassion among the less religious than it was among the more religious – and questioned who was more generous overall. If the press released had not omitted those findings, he would have had an answer, or at least a theoretically-likely hypothesis. Still, he did a little bit of research on his own and noted some well-known findings that the religious tend to give more to secular charities than atheists.

Finally, the press release included a few statements by the authors which were highly speculative. It is standard for authors to discuss possible explanations for their findings, but they are often presented to the public as conclusions the authors reached. In this case as in many others, reports of the study often imply that these things are findings when the study did not examine them at all. For example, the last paragraph of the press release is:

“Overall, this research suggests that although less religious people tend to be less trusted in the U.S., when feeling compassionate, they may actually be more inclined to help their fellow citizens than more religious people,” Willer said.

The rawstory.com report ended the same way, except they changed the last word to “concluded”. This statement serves as a hypothesis for another study, but it is not a finding of the study and cannot provide a conclusion.

I have heard a lot of explanations of what motivates more religious people, but most have been based on personal beliefs or experiences – duty to God, moral obligation, sense of community, etc.  This question may have been answered and looking to other studies might yield something, but this study does not address it – it did not set out to address that question.

A full analysis of the journal article will follow in a separate post later today.

UPDATE: Analysis can be found here.
ANOTHER UPDATE: The award for the most botched report now goes to MSN for this doosey.

Workshop: Skepticism in the Classroom

Matt engages the audience. Photo by Dean Baird (minor retouching and cropping by me)

Matt recapped the most important concepts from his piece last year and presented more of his fun and interesting demonstrations. I used to think that cognitive psychologists had all of the fun because we study the interesting ways that our brains and minds fool us and can blow those minds by showing them. However, after some thought I realized that the physics teachers I know have the coolest, scariest, ickiest, and most surprising demonstrations. They deal with the physical world and there are almost as many bizarre things in the physical world as there are in the mind.

No, I do not remember what I was saying when I made this face. Photo by Dean Baird (minor retouching and cropping by me)

I was a bit nervous about this workshop because some of the material I presented is very different from my usual “Oew” and “Ah” and “aHA!” stuff. In addition, its connection to promoting skepticism is distant, at least on the surface. The title of my presentation was Deep Thoughts: Facilitating Critical Thinking at All Ages. In teaching critical thinking, the age of the student is extremely important in determining methods and focus. For adults, the biggest roadblock to critical thinking is overconfidence. This is just a nice way of saying “arrogance” or “closed-mindedness”. The irony is that we humans are so overconfident that we think the term applies to other people and not ourselves.

For young children, there are few roadblocks. What we should focus on is guiding cognitive development in a way that minimizes overconfidence. In my opinion, the best way to do this is to encourage the practice of consideration and deep thinking. This, I suggest, is accomplished through discussion of philosophical questions.

I have yet to read a review of the workshop. However, the immediate feedback I received was very positive and I heard my words flowing from the mouths of others all weekend, including on the stage. It is entirely possible that others have been thinking about the same issues, but I choose to take it as evidence that my ideas were discussed and found worthy of some consideration.

A Short TAM9 Review

Unfortunately, I was still tweaking my workshop presentation and was unable to attend the other workshops. I caught only some of the activism workshop – the one I needed the most – but luckily there is a wonderful manual available which was produced by Desiree Schell and Maria Walters. Last year’s reception, which kicks off the official meeting on Thursday night, featured music and live interviews. This seemed to defeat the ‘meet and greet’ purpose of most attendees. The reception this year returned to the usual format of conversation, but there were so many people that it was difficult to find anyone. Friday morning JREF president D.J. Grothe announced the final headcount. Attendees, organizers, and presenters at TAM9 From Outer Space totaled 1652, approximately 300 more people than last year, which was 200 more than the year before.

In general, the long list of speakers booked for this year included the most inspiring scientists and science communicators in the skeptical community. The original keynote speaker, Astrophysicist Neil deGrasse Tyson, is a personal hero of mine. His talk repeated much of what he covered in his very long and satisfying TAM6 talk, but I never tire of the material or his presentation style. Unfortunately, the other keynote speech, delivered by Richard Dawkins, was as boring (to me, anyway) as Tyson’s was entertaining. I have never found Dawkins to be a dynamic speaker, but this was particularly snore-worthy. He chose to spend much of his time describing his soon-to-be-released children’s book rather than discussing anything of note. Likewise, I find PZ Myers’ style a little bit dull, but I usually enjoy his talks simply because he chooses to talk about some of the most interesting topics. This year is no exception. His was one of the few talks that I missed, but I am looking forward to his discussion of alien anatomy when the JREF posts video of his talk (they committed to making all of the content available online).

Every other talk (not including the Sunday Paper Session, which varied in quality) was fantastic.

Some of the highlights for me:

• Carol Tavris delivered a speech about reducing cognitive dissonance by first considering the target’s vantage point (i.e., empathy). This was probably the best speech I have ever heard, and I have heard a LOT of speeches and talks.


Dylan Keenberg, Sabrina Gibson, and me Photo by Daniel Loxton (minor touch-ups and adjustments by me)

• Dylan Keenberg, a former student and collaborator of mine, delivered a wonderful Sunday talk describing one method for talking to others (Rogerian argumentation) which is highly likely to reduce both cognitive dissonance and misunderstandings. The most important aspect of this method is, once again, empathy. In order to more than simply fake empathy, though, one must be open to the possibility that one’s current understanding is wrong. My informal polling of TAM9 speakers and other community leaders tells me that I am justified in feeling extremely proud.
  
  

  Dylan Keenberg and Daniel Loxton Photo by Sabrina Gibson (minor touch-ups and cropping by me)

  Daniel Loxton’s discussion of these two talks (Tavris’s and Keenberg’s) as well as the two which specifically addressed activism (one by JREF Communications Sadie Crabtree and the other by union organizer and radio host Desiree Schell) is much more thoughtful, thorough, and interesting than what I could write at the moment.

• Daniel also wrote about the panel to discuss diversity. In a nutshell, the discussion was quite a mess for the first half, but the more they discussed the more each clarified, and in some cases, changed their views until they settled on a middle ground that I think all could embrace. Essentially, they agreed that applying skepticism to a more diverse set of problems/questions/domains would result in a more diverse community without compromising the integrity of skepticism as a movement. Political, moral, and social ideology are ‘outside the scope’ of skepticism because they remove objectivity. In addition, untestable claims (e.g., “Does God exist?”) are off-limits because they cannot be addressed scientifically.

  I am always thrilled to hear D.J. speak about such things from a stage because he tends to be clear, firm, and directly on-message. Last year, for example, he made a point of asking nearly every speaker to clearly define the scope of their organization and each answered with some form of “scientific skepticism”. This year, he elaborated on this by noting that he strives for a diversity of religious views.

  However, I did not leave TAM9 with the optimism that Daniel Loxton left with. One reason for this was that D.J. made those statements while discussing “Diversity in Skepticism” with Debbie Goddard, Greta Christina, Jamila Bey, and Hemant Mehta. Debbie Goddard is the campus outreach director for CFI, a secular organization with a branch devoted to skepticism (CSI). The panel’s moderator, Desiree Schell, is firmly rooted in the skeptical community as the host of Skeptically Speaking and an occasional blogger on Skeptic North. The other three panelists are closely identified with atheism and, in my opinion, have contributed little, if anything, to skepticism itself. I kept wondering who this “we” was in the discussion (e.g., “We could offer…”).

  The conflation of atheism and skepticism is a very serious problem with dire consequences. The most important of these is the degradation of the integrity of skepticism itself. The scientific method only works when scientists are open to interpreting any result objectively – to consider all evidence with an open mind and to hold all conclusions tentatively. The conclusion that there is no God cannot be arrived at empirically, so it cannot be “the result of properly-applied skepticism” as some claim. I am very worried about this trend to conflate these two for several reasons, including the manner in which the majority of atheists talk to and about the faithful.

• Bill Nye’s talk was condensed from the longer talk he gave at the Skeptic Society’s Science Symposium last month. In his position as the executive director of The Planetary Society he is concerned with science education and the consequences of failing in this area. For this reason, he is another hero to me.
• The panel discussion of the future of space exploration was almost as lively as the diversity panel would be two days later. Most notably, Neil deGrasse Tyson’s verbal sparring with Lawrence Krauss left Bill Nye and moderator Phil Plait with little room to get a word in. However, Pamela Gay managed to do so by literally shushing Tyson – three times! For that, if not for the plea during her solo talk for all in the audience to be activists for education, made her another hero. Phil’s talk last year still rings in my years, so the odd man out on that panel – Lawrence Krauss – was the only one on the stage that I would not walk a few miles, breaking a path in the snow, to hear speak.
• Speaking of heroes, there were two announcements at TAM9 which deserve to be noted. One was that The Richard Dawkins Foundation has committed to fund child care at meetings and conferences like TAM. The other involves everyone’s hero, Genie Scott. At the end of a talk in which she described the parallels between evolution denial and AGW denial (described and discussed in a great post by Donald Prothero), she announced that the NCSE is beginning an initiative to fight climate change denial in public education.
• Finally, two Jennifers, Jennifer Michael Hecht and Jennifer Ouellette, merged poetry and popular culture with skepticism and science, respectively, in the most uplifting and inspiring ways. Hecht condensed a normally hour-long history of doubt into half an hour by speaking quickly, but this only enhanced the talk. Jennifer held a cultural mirror up to science and space exploration, showing clips and images from A Trip to the Moon to Doctor Who and beyond. These were as, if not more, intellectually fulfilling as the talks given by the psychologists (okay, I’m biased) and neurologists (Elizabeth Loftus, Richard Wiseman, Susana Martinez-Conde, and Stephen Macknik all spoke). Wiseman even introduced me to a new favorite ‘suggested lyrics’ video, so I think that I will leave you with that.