I really, really hate infographics.

Really.

They’re worse than memes. Well, I guess technically they are memes. They are notoriously inaccurate. They are usually agenda-driven and often spin facts to the point of wrongness. But mostly I just see them as click bait.

I have less disdain for listicles, but they sometimes bug me, too.

Yes, babies sleep a lot. Bet you didn’t know that!
/sarcasm

The only source is a link. The link goes to–surprise, surprise–an infographic.

Ugh.

Actually, it goes to an infographic that was re-posted from another site where I assume it originated (embedded below).

The graphic itself cites sources at the bottom, so I dug in. But before I talk about the source material, let’s take a look at the list.

1. We can only dream about faces we have already seen, whether we actively remember them or not.

Um, what? As opposed to faces that we haven’t seen?

No, we can’t “see” the faces of real people we’ve never seen, but we are perfectly capable of making up faces in our dreams. People we make up are certainly people we’ve never seen.

2. When dolphins sleep, only half their brain shuts down. The other half stays awake to help with breathing cycles.

True. Dolphins and whales sleep one hemisphere at a time.

3. Men have dreams about other men 70% of the time. But women dream about women and men equally.

I haven’t seen this in any of my textbooks, nor have I read any studies that confirm this. I’m skeptical, but it’s one of those factoids that, if true, I’d think “so what?”

4. While you are sleeping your body recharges, your cells repair themselves, and your body releases important hormones.

True.

5. Parasomnia is a type of sleep disorder that makes you do unnatural movements despite being asleep. Crimes committed on parasomnia include: sleep driving, writing bad cheques, murder, child molesting and sexual assault.

True (although they’re talking about a specific type of parasomnia) and FASCINATING. When I taught introductory psychology, we spent the most class time on parasomnias because it’s so fascinating. Share your sleepwalking stories in the comments because I never tire of them.

6. 12% of people dream only in black and white. This number used to be higher but since the advent of color television, more people dream in color than before.

Another bit I’ve never heard. I wouldn’t be surprised, but I also don’t know why it matters.

7. Dreaming is normal. People who do not dream generally have personality disorders.

This is bullshit. First, dreaming isn’t just normal, but it’s something that everyone does. People who say that they don’t dream simply don’t remember their dreams.

And no, failing to remember your dreams absolutely does not mean that you have a personality disorder.

No. Nuh uh. Not remotely.

8. Sleep positions may determine your personality.

Um, no. No “may” about it. This is about as true as saying your favorite color determines your personality.

9. 1 in 4 married couples sleep in separate beds.

Again, I don’t recall ever hearing about or reading a statistic on this, but 1 in 4 seems awfully high to me. I wouldn’t be terribly surprised. There are lots of reasons for it. But 1 in 4? I’m skeptical.

10. British soldiers were the first to develop a method in staying up 36 hours without sleep. When fatigued, they put on special visors that emulated the brightness of a sunrise and it woke them up.

I can’t confirm this, but it’s plausible.

11. Longest sleeping mammals are koalas (22 hours) and the shortest sleeping are giraffes (1.9 hours taken in 5-10 minute sessions).

May be true. But seriously, who cares?

12. You need different amounts of sleep depending on your age. Babies need the most (16 hours) and people over 65 need the least (6 hours).

Although it is true that age matters when it comes to sleep requirements, there is a lot of variability among individuals and the recommendations have changed in recent years. In general, the older you are, the less sleep you probably need, but most Americans are at least a little sleep deprived.

13. You’ll die from sleep deprivation before food deprivation. It takes 2 weeks to starve, but 10 days without sleep can kill you.

I have to wonder where this one came from.

First, the term “sleep deprivation” includes too little sleep, not just no sleep.

Sleep deprivation can have very serious health effects. However, one does not usually die from lack of sleep. It’s sort of like dying from holding your breath. You can’t. You’ll eventually pass out and your body will resume breathing without your will to do so.

One can certainly die from the health problems that sleep deprivation contributes to or may even cause, but in 10 days? Not likely. There aren’t a whole lot of documented cases of total deprivation, so I can’t even guess where this number came from.

At most I think that we can say that sleep is necessary for good health. Might even say that you don’t want to go without it, but if you do, you just mind find yourself in a situation in which you have no longer have a choice.

14. Blind people can still see images in dreams. Those born blind experience dreams involving emotion, sound, smell, and touch instead of sight.

Well, of course. Our brain activity when dreaming is very similar to waking states. The biggest difference is that the information isn’t coming in through our senses. It makes sense that the experience would mimic the life our brain has developed to experience.

15. Within 5 minutes of waking up, 50% of your dream is forgotten. Within 10 minutes, 90% is gone.

This is just made up.

16. 1 out of 50 teenagers still wet their beds.

I looked at about a dozen sources and all put the number at 1-2 in 100, so I think 1 in 50 is the highest in the range of estimates. But the statement is a bit misleading in that it suggests that it’s a chronic problem for that many teens. The estimate includes those who experience a bed wetting incident once in their teen years, which is probably the majority of cases.

So that’s it. A few fun facts, some boring and unverified factoids, and some outright bits of bullshit. That’s what happens when you regurgitate something that was regurgitated by someone else using something that was copied from something else that was created by someone who basically made stuff up.

Now let’s take a look at the ‘citations’.

The source list for the infographic is printed on the bottom of the graphic itself, so I dug in. There are 16 citations and 16 list items, but they don’t appear to be related to one another in a one-on-one fashion at least.

The first source is a site called “MEDPAGE TODAY”. It’s not a bad piece, but I can’t find an item on the infographic that corresponds to what’s discussed on the page.

The next three sources resulted in error pages. The second of these defunct links is to–another surprise–a LISTICLE. Someone was kind enough to post this one in a forum, so we can see the original content. Unfortunately, no citations were included.

The next item, sleepapnea.org, documented only the main page of the website. I cannot tell which “facts” were gleaned from this source.

Item number 6 is a Slate piece about sleeping with your dog. There is nothing in the infographic or the listicle about sleeping with your dog.

The next three are news articles about parasomnia. Two are dead-ends. So three links to support one item on the list and two are outdated (giving the author the benefit of the doubt).

The next link is incomplete, so there’s no way to access it, and the one that follows that one is the same as link number 2.

The next link might be the jackpot. It’s a site cataloging “interesting facts about dreams”. It lists 99 items from seven sources. One of those sources is specific and the rest are for-the-masses books like “The Big Book of Dreams”.

Then there are a few more dead ends and a final page that doesn’t appear to correspond to anything on the list.

Okay, so now we have some idea of where this stuff came from (don’t say it), but not much.

From Visually.

Let me cut to the chase: No.

Full disclosure: I own a Fitbit Flex™. I haven’t worn it in months, but I do like the thing. It’s a long, boring, irrelevant story why I’m not wearing it, so I won’t get into that. Just know that I am sincere when I say that I like my Fitbit.

I don’t think my Fitbit is telling me exactly how far I’ve walked or how many calories I’ve burned. I do think that it gives me a ballpark figure that can be used to compare one day to other days. I don’t think that it’s telling me exactly when I am sleeping, either, but the information about how restless I am some nights compared to others is interesting and even sometimes useful.

I also like that it buzzes on my wrist to wake me up in the morning.

Recording some basic information about my daily activity, giving me a reason to record what I eat, waking me up in the morning, and reminding me to get off my ass. These are not big goals and perhaps I don’t need a $100 gadget to accomplish them, but is the device bullshit?

Mother Jones sure seems to think so. Last week the site posted a piece titled “Science Says Your Fitbit is a Joke“. When I searched for more reports on the topic I found a Jezebel piece titled “Your Fitbit is Bullshit, Says Science“, and one on RYOT titled “Science Says Your Fit Bit’s Full of It“.

Since I don’t particularly trust any of these sites as a source of accurately-interpreted science news, especially those which appear to be trying a little too hard to find clever, edgy headlines, I was skeptical.

The Jezebel piece is easily dismissed as simply lazy blogging. The author links to Mother Jones, paraphrasing one of the main points of the piece (bold mine):

Mother Jones points to a new study showing that your iPhone or device of choice does just a good a job[sic], if not a better one, at doing things like tracking calories and measuring activity.

Except that’s not what the Mother Jones piece said. It reads:

But, according to a new study published Tuesday in the Journal of the American Medical Association (JAMA), the apps on your smartphone do the job just as well, or even better—at least in terms of measuring your steps and your calories.

So Jezebel incorrectly paraphrased Mother Jones. And they aren’t the only ones. RYOT wrote this:

A new study came out showing that iPhone apps and other (much cheaper) devices do a better job than the bands at tracking your activity, steps, and even calories.

Even assuming Mother Jones is correct, these paraphrases are not. Walking is, of course, an activity, but it’s only one type of activity. Mother Jones was correct in noting that smartphones can do the work of pedometers, but Jezebel and RYOT are overstepping (pun intended). Wearables such as the Fitbit Flex™ and Jawbone Up™ can track your activity while you are swimming, bowling, or doing jumping jacks. Can your smart phone do that? Well, I suppose if you’re holding it in your hand, it can, but who does that? Also, if you don’t sleep with your smart phone strapped to your body, it cannot track your activity during sleep.

But is Mother Jones right? Might smartphones do a better job? Well, let’s find out.

Although Jezebel linked to the original source, a press release by University of Pennsylvania School of Medicine, it does not appear that the author read the press release, nor did the author of the RYOT piece, who linked only to the press release (failing to even give Mother Jones credit for the thoughts expressed). There is no reference to smart phones doing a better job at anything, much less a list of activities. The only study which is discussed is the one described in the Mother Jones piece, which examined the accuracy of several devices at counting steps and found all to be similar.

But let’s get to the bottom line. Is it a fair assessment that science says these devices are bullshit? A Joke? Full of it?

First, the study discussed involved counting steps. That’s it. There is absolutely no comparison of different devices in regard to tracking calories and no examination of other activity or reports such as distance walked. The study was fairly well-designed–something that does not come across in the reporting. Although there were only 14 participants, each wore every device at the same time and each walked both 500- and 1500-step trials twice.

Here are the results from the 500-step trials:

The results of the 1500-step trials are nearly identical. Now, it appears that the Fitbit One™ and the Fitbit Zip™ are spot-on, and the fact that the error bars are practically non-existent is pretty impressive. Unfortunately, the authors failed to do any sort of statistical analysis at all. Instead, they stated these results:

We found that many smartphone applications and wearable devices were accurate for tracking step counts. Data from smartphones were only slightly different than observed step counts, but could be higher or lower.Wearable devices differed more and 1 device reported step counts more than 20% lower than observed.

Given the error bars for the rest of the devices, including the Fitbit Flex™, I’d believe that most did not differ from one another significantly, with the possible exception of the Nike Fuelband™, which apparently sucks. However, there are two very clear winners, with the Digi-Walker™ close behind. A simple statistical analysis would have confirmed this.

Regardless, it is very clear that smartphones do not outperform most of the wearables. What is really in question is whether people use the devices and use them to their advantage. This may be questionable, but it has hardly been decided by science and it has certainly not been decided in favor of tossing one’s Fitbit™. The JAMA article concludes:

Increased physical activity facilitated by these devices could lead to clinical benefits not realized by low adoption of pedometers. Our findings may help reinforce individuals’ trust in using smartphone applications and wearable devices to track health behaviors, which could have important implications for strategies to improve population health.

So there’s hope that devices, whether they are wearables or just smart phones, will translate to improved health for at least some users.

To be fair to Mother Jones, their piece is much more fleshed out than the other two, covering more than this one study. In my opinion, the Jezebel and RYOT pieces simply plagiarized Mother Jones. However, I saw nothing in the rest of the piece to warrant such harsh criticism of wearables.

Now after writing most of this I came across this wonderful piece on Mashable that was posted yesterday. I could have simply linked to it, saying “read this” because it’s pretty much what I have said, right down to a comparison to the game of “telephone” that I just edited out of this post.

In the end, I have to agree with its author, Chris Taylor, who opens with:

There are poorly designed scientific studies, and then there’s poor reporting on scientific studies by journalists who should know better.

So this Good Morning America piece showed up in my Facebook feed the other day touting the sensational headline “Odd-Defying Babies Born 10:11 12/13/14″.

Now, I think it would be adorable to have a baby born on 10:11 12/13/14 (in America, of course. In Europe, that would be 10:11 13/12/14, which just doesn’t hold the same cuteness). Human beings love the symbolism that comes from conventions such as labeling and ordering. My good friend and fellow Skeptic, Ani Aharonian (Insight blogger and guest blogger here) was married on 5/8/13 for a reason (can you guess?). But “odds-defying”? No. These babies defied no odds.

Odds are a property of something that is yet to be. They are really only valuable as a means of predicting something and have no value after the event has occurred. This is a bit like post-hoc (after-the-fact) thinking about lottery outcomes. Did your next-door neighbor defy the odds when they won the lottery? Well, it depends on your perspective, especially in time.

A given individual has an extremely slim chance of winning the jackpot of a lottery. For a Powerball lottery, a single ticket has a one in 175 million chance of winning. However, if 525 million tickets are sold, there’s a pretty good chance that at three, much less one, will be winners. Is it “odds-defying” that your next-door neighbor won? Well, perhaps from your perspective, but not to a total stranger in another state.

Similarly, the question of predicting the odds that any given child is born at that time is quite different from the post-hoc consideration of the odds. The prediction actually changes whether you are predicting the birth from before or after conception. Clearly, if you are carrying a child whose due date is around 12/13/14, the chance are much, much higher than if you have yet to conceive (especially if it’s already April 2013). But nobody even thinks much about these things until after the fact, probably because they are more concern about having a healthy child and mother at the end of it all. Post-hoc, we are really not talking about odds anymore, so unless someone predicted a very different outcome for one of these births, there was no defying of odds.

Well, except this:

Approximately, on average, eight babies are born in the U.S. every minute, so there should be around eight babies in the U.S. right now with the “lucky” birth time of 10:11 12/13/14. There are, according to the article, only two. It’s entirely possible that the author missed some, but let’s say she didn’t. That’s an unusual event. Probably not *very* unusual, as I’m sure the number of children born each minute varies a great deal, but still quite different from what we would predict given what had happened in the past. It’s also a bit interesting that both were born in the mid-west–one in the city of Cleveland, Ohio and the other in the relatively small city of Billings, Montana. Cleveland is not a small town, but why are the biggest metropolitan areas such as Los Angeles and New York not represented? For that matter, they are both girls, yet averages tell us that one should be a boy. Probably because the variability in these events is very, very high and while averages are the best predictors we have, “best” isn’t always very good.

But the reason I wanted to talk about the piece wasn’t just the sensational headline. The contents are pretty eye-rolling, too.

“We knew she was going to be born today [Saturday], we just didn’t know it would be at 10:11 a.m.,” Campbell said in a statement. “Everyone is telling us we should play the lottery. We feel this is a lucky day and are excited to get family photos with Santa.”

It’s a number. It’s cute and all, but really, it’s just a number. It won’t help you win the lottery. It won’t help her succeed in life.

These conventions don’t care who you are. They don’t care where you live or what you ate for breakfast. So don’t complain about the lack of diversity in this sample, either. The universe is colorblind. Sort of.

A piece by Joseph Brean that I read recently displayed the headline New hope for narcissists: New Canadian study suggests there may be a cure for self-centred[sic] grandiosity after all. My first thought was “I doubt it.”

The press release for this study is pretty accurate and, although I have some criticisms of the study, the journal article doesn’t make the dubious claims found in this piece. However, I will say that it’s a little hard to tell if Brean is fully responsible because it is unclear whether he interviewed the scientists who wrote the article himself. Even if he did, it’s hard to know what that interview looked like. I’ve been on the other side of such interviews and usually the end product is a gross misrepresentation of what I’ve said.

This piece also brought to mind the issue that free and easy access to original sources has some serious trade-offs. Laypersons often misinterpret studies (hell, scientists often do) and in communities such as skepticism, some of those laypersons speak those misinterpretations on stages with an air of authority. But more importantly, some calling themselves science writers misunderstand, cherry-pick, overgeneralize, over-extrapolate, and otherwise misrepresent the implications of findings, especially those in the social sciences. With the vast majority of the public (including a lot of scientists short on time and resources) getting most of their science news from non-scientists and trusting those sources, I find this to be a bit of a problem.

So let’s take a look at the statements made in this piece and what I think is wrong with them, given the original study. The National Post piece introduces the study this way:

…narcissist is usually described as the product of a long and complicated psychological development. Like hypochondriacs, narcissists are made, not born.

New research out of Wilfrid Laurier University, however, suggests narcissism might be simpler than that. More than just a moral failing or psychiatric symptom, narcissism might reflect a basic mechanical failure of the brain’s natural tendency to mimic.

Um, no. No, the research does not suggest that, nor do I believe that descriptions of narcissism imply that it’s all about parenting. Most disorders, including personality disorders, appear to be the result of a complex interplay of environment and genetics.

But where the author really goes off track is in the next sentence:

Intriguingly, it also suggests that narcissism’s opposite, empathy, might even improve with practice.

Um, what?

When I read the press release, I saw nothing in it that even hints at practice effects, so either Brean (the author) actually read (and misinterpreted) the article or he gleaned this bad information from the interview to which he refers. Or perhaps he just made it up because he thought he understood the research. Who knows?

Then there’s a quote of the lead author. Obhi, which reads, “Narcissists don’t imitate automatically.”

If that’s a quote, I hope it was taken out of context, because it reads as a statement of fact, a given, when it is merely one possible explanation for the findings, and one that does not appear in the journal article. In fact, it contradicts the findings and even the title of the journal article: Automatic Imitation is Reduced in Narcissists. This is a poorly-chosen title for at least two reasons that I will discuss below.

So what can we reasonably take from this study? Well, let me first summarize the study.

Obhi, Hogeveen, Giocomin, and Jordon conducted a rather simple study with a final sample of 24 subjects. The subjects performed a task which involved responding to a cue by lifting one of two fingers off of the keyboard. The cues were embedded in images of hands in which one of the fingers is raised. For some trials, the cue matched the position of the hand in the picture (e.g., the picture showed the index finger raised and the cue instructed the subject to raise their index finger) and in some trials the cue was incongruent with the picture (e.g.,  the picture showed the index finger raised and the cue instructed the subject to raise their middle finger).

This task is a paradigm that many readers will be familiar with, even if you don’t recognize it immediately. It is similar to the Stroop Task, the Simon Task, and the task used in Implicit Attribution/Association Tests.

Essentially, the task involves suppressing an automatic response–going against one’s initial, automatic response to a stimulus. For example, the classic Stroop Task involves identifying the color of ink in which a word is printed. The trick is that the words the participant is looking at name various colors. Nearly everyone will take longer to name or respond to the ink color when it is different from the word than when the word is the color of the ink.

The Stroop Test. It takes longer to identify the ink color when a color word does not match the ink color (bottom) than when it does (top).

This well-established effect demonstrates how automatic reading can be. Automatic behaviors are those which require few cognitive resources and are sometimes performed without much awareness. Automatic responses are often very difficult to “shut off”. When we attempt to perform a similar task which competes, the automatic response must be actively suppressed. This is what happens in the Stroop Task; we must suppress the response of reading the word in order to identify its ink color. The differences in reaction time between color-congruent and color-incongruent trials are a good measure of how much the automatic task interferes with the primary task.

The Simon Task and Implicit Attribution tests take advantage of something we call mirror responses. Research has confirmed that we see someone do something (e.g., as simple as raising a hand), our brains automatically respond with activity which is very similar to what we might see if we were performing the task ourselves. This response has many advantages, including facilitating joint attention. Joint attention is demonstrated when you move your eyes to see what someone else is looking at.

Many automatic tasks such as reading are acquired through lots and lots of practice, but not all. Some, like joint attention, are probably innate reflexes. Mirror responses are probably a mixture of both.

Obhi and colleagues also asked participants to complete a short version of the NPI [Narcissistic Personality Inventory] and divided them into two groups (“high” and “low”) based on their NPI scores. They found greater differences in reaction times (RTs) between congruent and incongruent trials among those in the low narcissism group than among those in the high narcissism group. In other words, participants who scored higher in narcissism demonstrated less interference than those who scored low in narcissism. Even more pronounced, however, were the differences in accuracy. Those high in narcissism erred in about 5.3% of the trials while the error rate for those low in narcissism was more than 12%.

As an aside, remember that quote about narcissists not imitating automatically? Well, the findings suggest otherwise. There certainly was an interference effect for both groups. Those high in narcissism did indeed respond more quickly and with fewer errors when the cues matched the images. Furthermore, the title of the article was Automatic Imitation is Reduced in Narcissists, yet they did not study narcissists. Subjects were not selected for their NPI scores and no diagnoses were recorded. The participants in the “high” group were simply those whose NPI scores were in the top half of the subjects tested.

The authors discussed possible explanations for these findings thoroughly. Essentially, the best explanation they give is that those high in narcissism more easily suppress mirror responses. This could be due to greater self-regulation or it could be that mirror responses are not as automatic. I cannot account for the decision to title the paper with one of those explanations, except that they dismissed the self-regulation hypothesis with a non sequitur.

What I can say for certain is that the findings do not suggest that narcissism is a product of poor mirror responses. They also do not suggest that practicing such responses, which are largely learned implicitly, would make them more automatic or make the individual more empathetic and less narcissistic. Neither of these hypotheses is impossible, but neither is likely, either.

The authors also acknowledged many of the study’s limitations, including the relatively small sample size, but my reaction to this study remains mixed. On the one hand, I applaud these researchers because although it seems that this area should have been well-studied, it isn’t. On the other hand, this is a very simple study to execute and the paper has four authors, yet they completely missed the opportunity to ensure strong findings by putting in just a little bit more work.

The study would be improved leaps and bounds by running a larger sample and testing whether self-regulation or motivation were factors. The latter can be easily accomplished by including a condition with non-social context (e.g., arrows instead of fingers). Furthermore, they could have pre-screened the subject pool and recruited only those who scored exceptionally high or exceptionally low in narcissism, thus raising their power tremendously.

That said, these findings are consistent with what we know about narcissism. A key feature of narcissism is reduced empathy and it stands to reason that reactions to the actions of others would be more automatic in those with greater empathy.

In the discussion section of every study is a laundry list of possible explanations and implications for the study’s findings, but most of these are speculation and usually labeled quite clearly as such by the authors. This does not seem to stop laypersons and “science writers” from accepting the speculations they find most interesting or desirable.

In this case, the implications of the study for a behavioral test of empathy are much more plausible than those for treatment of narcissism.

h/t Tim Farley of WhatsTheHarm.net and Skeptools.

Obhi SS, Hogeveen J, Giacomin M, & Jordan CH (2013). Automatic Imitation Is Reduced in Narcissists. Journal of experimental psychology. Human perception and performance PMID: 23957308

I hope I grabbed your attention with that title, but do not expect to find the answer to that question here. What I am going to discuss today is a study that many people seem to think answers that question, but it doesn’t.

As I noted in my last post, the study I’ll be discussing was grossly misreported, starting with its press release. Since the study itself appears to be behind a pay wall for most people, I’ll describe as much detail as I can in a blog post as I discuss the study’s validity and findings of the study, published in the Journal of Social Psychological and Personality Science and titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.

But for those who are not at all interested in the research methods or a breakdown of why I rate the quality of the study the way I do, I will give you the the bottom line so you can skip the rest or only read the sections that interest you (I’ve used headings to make it easier).

Summary

I think that the findings will hold up to replication, despite some issues I have with the way they did a few things. Overall, the research quality is quite high.

The groups they compared did not include atheists, agnostics, believers, non-believers, highly religious, or any other label that you can throw at it. In the studies they used raw religiosity scores and made some comparisons of “higher” and “lower” using values from the distribution. In a sense, the compared those who scored in the lower half of the sample to those who scored in the upper half.

They found:

• Differences in prosocial behavior cannot be dismissed as due to political affiliation, socio-economic status, or other factors often held up as responsible.
• Religiosity is correlated with trait compassion; the more religious, the more compassionate.
• Trait compassion is related to prosocial behavior in general. This relationship is stronger in the less religious than in the more religious. This does not mean that the less religious are more compassionate (see number 1) or that the less religious are more prosocial. It just means that compassion is a bigger factor in prosocial behavior in the less religious.
• The findings of the first study can be interpreted one way that isn’t discussed in the paper: when the relationship between compassion and religiosity is accounted for, the more religious are not more prosocial than the less religious.
• The findings in the second study, which involved inducing feelings of compassion, were similar for generosity, except that the more religious were more prosocial even after accounting for compassion.
• The findings of the second study also included a different pattern when the prosocial behavior was giving to charity. Compassion induced more giving, but the effect was weak and did not differ much across religiosity. Religiosity had a significant affect on charity. This can be explained by the guidelines provided by many churches for how much of one’s salary one should give.
• In the third study, in which state compassion (how compassionate the individual felt at that time) was measured and the prosocial behavior measure involved real-world cash, religiosity was not related to either compassion or prosocial behavior.
• In the third study, state compassion was positively correlated with prosocial behavior, but the effect was greater in the less religious than in the more religious.

What the findings as a whole say to me, and what I believe the press report tried, but failed, to express, at least with convincing support: We do not need religion to be prosocial. We need compassion.

This is great news for secularists.

However, it doesn’t say anything negative about religion or the religious, nor does it provide anything that should make atheists feel superior. It just shows that one can be good without God; that motivations can come from other sources.

Now on to the details…

NOTE: to keep this as short as possible, I’ve included a lot of links to terms and demonstrations. Where I describe problems in more detail I still water-down quite a bit. I will do my best to make it understandable without rambling on and on, but keep in mind that it takes many years to learn enough about research design and statistics to understand why some of these are problematic. Furthermore, not all researchers will agree on the consequences of some of these problems. I am still learning this stuff myself (probably always will be learning).

The Study (description)

The article reports three studies, each related to the relationship between compassion and prosocial behavior in less-religious individuals. I have created graphs using the information in the paper, but in some cases I did not have exact numbers, so while the relationships are visually accurate, there are only values where I could use exact numbers.

Theoretical Foundation

The introduction discusses research which documents that religiosity is associated with prosocial behavior. Specifically, religious people give more and volunteer more than nonreligious people, over and above what they give to and do for religious organizations. The researchers note that the nonreligious do give; when we compare groups, we do so using averages. However, it may be that the motivations for prosocial behavior vary in a way that interacts with religiosity. In other words, the more religious among us may be motivated to prosocial behavior by one set of factors and the less motivated by another.

The researchers hypothesized that compassion is a more influential factor in prosocial behavior for the less religious than for the more religious among us.

Study 1

The first study examined the relationships among religiosity and traits of compassion and prosocial tendencies. What this basically means is that situational factors were not involved; traits are a matter of personality or attitude. For example, “trait anxiety” refers to how anxious a person is in general, while “state anxiety” refers to how anxious that same individual feels in a given situation.

This study involved analyzing data from a 2004 “survey”. I put that term in quotes because it usually refers to a set of questions that do not measure more than what is apparent at face value. Established measures of latent variables (variables which cannot be measured directly such as feelings and attitudes) are usually called an “inventory” or “scale” and we refer to them loosely as “measures”. In this case, the survey involved such measures and I want to make that clear.

The sample was comprised of 1337 participants and covariates (variables other than those of interest which could explain differences among the groups) of gender, political orientation, and education were included in the analysis. The variables of interest were religiosity, compassion, and prosocial behavior. Religious identity (identification with a specific religion or no religion) was also considered.

Results

Correlations

• Covariates had little impact on the results.
• Trait compassion was positively correlated with religiosity* and prosocial behavior. On average, the more compassionate the individual, the more religious they were and the more the more prosocial they were.
• The relationship between religiosity and prosocial behavior was marginally significant (statistically).

Hypothesis Test (See Figure 1)

Figure 1: Study 1 results. "Higher" and "lower" are defined here as 1 SD from the mean.

• A regression analysis revealed an interaction of religiosity and compassion on prosocial behavior. What this means: The effect of compassion on prosocial behavior differed among levels of religiosity.
• More specifically, the level of trait compassion affected prosocial behavior less as religiosity increased.
• There was also a main effect of compassion, but that was apparent in the correlational analysis.
• There was no main effect of religiosity on prosocial behavior. This is interesting, because they found a marginally significant correlation, but it does not mean the there are no difference in prosocial behavior. I would interpret these findings, when put together, as suggestive of little or no difference between the more religious and the less religious in prosocial behavior over and above the differences accounted for by compassion.

The authors discuss the findings a little differently, though, focusing on the differences in the way that compassion affected prosocial behavior (the interaction in the first hypothesis test result) and ignoring the way that the effect of religiosity disappeared when compassion was entered into the equation. It seems more interesting to me to treat compassion as the moderator. It also makes more sense in the end.

Study 2

This study was experimental in that the researchers manipulated state compassion. In other words, they induced feelings of compassion in half of the participants and compared the amount of prosocial behavior those participants engaged in to the amount of such behavior in a control condition.

The sample included 101 participants and the study was conducted online, so the age range was exceptional (from 18 to 68 years). Participants were randomly assigned to one of the two conditions and each watched a short video under the guise that there would be a test of memory afterward.  The videos were established manipulations of feelings of compassion and neutral emotion (i.e., other researchers tested their effectiveness). Following the video, participants completed two tasks which are well-established measures of prosocial behavior commonly used in such research.

Results

• Again, covariates had little impact on the results.

Hypothesis Tests (See Figure 2)

Figure 3: Results of Study 3. Values are relative.

There were two tests since the participants completed to different prosocial tasks, one involving generosity and the other involving charity.

For the generosity task:

• This time there were a main effects of both religiosity and compassion on prosocial behavior. The more religious, the more prosocial. Those who watched the compassion-inducing video were the more prosocial on average than those who watched the neutral video.
• The interaction appeared again in the manner as in Study 1.

For the charity task:

• There were main effects of both religiosity and compassion on prosocial behavior.
• There was no interaction.

This is where they screw up, in my opinion.

The pattern of the moderation was in the predicted direction but failed to reach statistical significance.

This is not an acceptable statement unless the findings are marginal. This was not. The p-value was .408. This is not even close to meaningful. Still, they went ahead with the analysis of the interaction and reported an effect of compassion on charity for the less religious participants and no effect for the more religious.  The problem is that post-hoc analysis like this assumes that a significant interaction was observed. Their tests inflated alpha (the probability of a Type I Error) and can only mislead. They stated that they had found “partial support” for their hypothesis, but they did not in this case.

The relationships in the generosity task are very clear when we look at a Figure 2. The interaction is the interesting finding. Compassion had little effect on the more religious, but a very large effect on the less religious, who gave practically nothing when compassion was not induced. There is no analysis to tell us if the less religious surpassed the religious by a statistically significant amount when compassion was induced, but they were clearly out done by the more religious when not made to feel compassion.

The charity task showed no such interaction and the authors did not include a graph of this effect that I could recreate, nor did they provide the information to make one.

Study 3

For this study, the sample of 120 completed a state compassion inventory (a measure of their feelings of general compassion at the moment) and a series of “economic tasks designed to measure their generosity, trust, trustworthiness, and motivation to reward others’ generosity.” What differed in this study, however, was that the ‘points’ they earned in these tasks could be exchanged for cash at the end of the study. Participants did not know how much cash, but they knew that the more points they earned, the more cash they would receive.

Results

Hypothesis Tests (See Figure 3)

The findings of this study were very different from the other two.

• State compassion was not related to religiosity.
• Religiosity was not related to prosocial behavior.
• There was an interaction of religiosity and compassion on prosocial behavior. The amount of compassion felt had more of an effect on the behavior the less religious than it did on the more religious.

The graph of these findings, a reproduction of their graph since they did not provide information to create one that would make more sense (to me anyway), is a bit misleading. The values are z-scores, so they are relative to one another and not actual values. What is interesting, though is how little the prosocial score varied in the more religious group and how that line barely dips below the mean value (represented by 0).

There is also a problem with the press release in that it makes the claim that the high state compassion/less religious group out-performed the others. There is no statistical analysis comparing the groups in that way, so this is a misstatement. We do not know if less religious individuals are more generous than more religious when motivated to act prosocially. We just know that they are more generous when motivated by compassion than when compassion is low.

The Study Overall

As I noted, my opinion of the studies as a whole is relatively high, but I do have some major criticisms. Some of the language makes me cringe (e.g., results are the product of statistical tests, so “We tested our results”…), but I have seen more and more of this as scientific reports in general have grown sloppier.  Study design and method is much more important, as is the quality of the reporting beyond language.

The authors also throw around the term “robust”, claiming in the first study that the relationship between compassion and prosocial behavior is “particularly robust” for less religious individuals. That term refers to findings which are “sturdy” and will stand up when some supports are removed – effects which appear to hold up in different situations. Since this was one analysis of one data set, that term just doesn’t work. It does not fit in any of their uses of it.

In fact, they err in Study 2 by saying that the effect was “attenuated” for the more religious. That term is relative; attenuated compared to what? The effect was not “robust” in one condition and “attenuated” in another; they can only be compared to each other. The effect was greater in the less religious than the more religious.

Missing Information

There are a number of bits of information which are considered to be, at minimum, required for a good research report. A general rule of thumb for methods and results sections is to include enough (without being redundant) information to allow other researchers to replicate (in a strict sense) the study and to confirm that the statistical findings are properly interpreted.

I am not sure that this article meets that criterion. The methods are pretty well fleshed out and the paper is full of statistics, but some descriptive statistics are missing that I would have liked to have seen (e.g., means reported overall for measures, but not by group) and there was not enough of the right information to recreate them.

Grouping the Data and Errors of Generalization

One overall criticism which warrants discussion is in the grouping of data. There are some problems with this and they are related. The sensitivity of the religiosity measure is one problem that, by itself, is not a big target for criticism. Combined with the second problem of grouping participants, though, it becomes more serious.

The practice of comparing groups of people based on a variable which is distributed on a spectrum is a common one. The question the researcher wants to answer is important in deciding whether to group and, in this case, I do not disagree with that choice, but I question how they grouped and how it was communicated. If the data are clustered (the distribution is multi-modal), grouping is simplified, but if the data are distributed more loosely, it can be tricky and dangerous.

First, the researcher loses information, therefore they lose sensitivity and usually lose power. The sensitivity problem is relevant in the first study, but mostly because it makes the findings difficult to interpret.

Second, if the way that the grouping is communicated is not consistent and clear, it is likely to be misinterpreted, compounding any existing problems with the method. I discussed this problem in my last post. Most of the reports referred to the groups compared as “highly religious” verses “atheists and agnostics” or something like that. However, where are all of the people in the middle (i.e., most likely the bulk of the sample)?  Within each group there was variation in religiosity and comparisons are made using averages. Generalizing only works when the samples are representative of the population of interest and this applies in either direction of the generalization (i.e., specific to mixed or mixed to specific).

Third, researchers must decide where to draw the lines between high and low (and anything in between). Since the majority of variables in psychology are normal distributed (therefore symmetrical), the lines are usually drawn using rankings of sample values and the most common way to split a sample in half is to put all values above the median into “higher” and those below into “lower” (called a “median split”). However, ease is not a good reason to use this technique.   Here is an interesting demonstration of the dangers of dichotomizing normally-distributed variables.

But… religiosity is not usually distributed normally; it’s usually skewed. Skew means that it’s not symmetrical, so a median-split would make even less sense.

In this case, it seems that the authors tried to have the best of both worlds by treating religiosity as a continuous variable, but doing post-hoc analysis on it, discussing it, and graphing it as if it were dichotomous, choosing values which were one standard deviation from the mean in both directions as the central tendencies of each group. The biggest problem with this is the assumption of normality. If the variable is not normally-distributed (and I suspect that it is not), this grouping is a bit tough to swallow.

When this problem is mixed with a limited range as it is in the first study (the religiosity scale only had four points), it’s a problem. The four possible values were 1 = no religion, 2 = not very strong (religious identity), 3 = somewhat strong, and 4 = strong. Since the mean was 2.99, the bulk of the sample were fairly religious. one standard deviation (1.03) below the mean is not exactly in non-believerland and one above is off the scale (literally). It is just very difficult to see where “higher” leaves off and “lower” takes over.

Although the range is adequate in the other two studies, the problem of discussing groups which do not actually exist and have fuzzy definitions remains. In my opinion that is one of the reasons it was so misreported.

But, overall, the research is of a relatively high quality and interesting. I would like to see more variation in the prosocial tasks, given that the outcome of the charity task was so different from the tasks of generosity.

It seems that the less religious are at least as generous as the more religious, but their reasons for acting prosocially differ. I would like to see the day when, as a group, we are generous and prosocial consistently, without the need to be provoked and without needing to feel an emotional connection to the receiver.

Saslow, L., Willer, R., Feinberg, M., Piff, P., Clark, K., Keltner, D., & Saturn, S. (2012). My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals Social Psychological and Personality Science DOI: 10.1177/1948550612444137

It seems that the best motivation for me (to write) is frustration.

A press release by UC Berkeley about a study that was recently published on the relationships among religiosity, compassion, and prosocial behavior has been making the rounds over the last couple of days, waved by proud atheists as evidence of superiority and bashed by the more skeptical as bad science (even though most haven’t appeared to have read the study). The latter has been exacerbated by the fact that the majority of reports include very big mistakes.

I’m going to analyze the study, but I will post that separately since it is likely to be long and I also want to discuss the incredible mess of BS this has become. In that post I will also discuss the reasons some of the errors that seem minor here are actually quite serious.

I blame the press release. Unfortunately, as the culture of higher education becomes more consumer-oriented, strapped-for-cash universities must market themselves strategically, and scientists are forced to compete for funding, the integrity of science is diminished. I don’t know if the study’s authors were involved, but my guess is that the press release’s author is mostly responsible for spinning the findings, omitting important parts of the findings, and cherry-picking statements by the scientists to make it appear that their speculative explanations for those findings are solid conclusions. I think that spin, along with the fact that the findings involve an interaction (a notoriously difficult concept to grasp due to its non-linear nature), confused science writers, many of whom are not in the practice of reading studies and many of whom added their own spin to the mix.

The study, published in the Journal of Social Psychological and Personality Science, is titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.  The first error of the press release was made in its title: “Highly religious people are less motivated by compassion than are non-believers” by incorrectly describing the subjects of the study. It also errs in a few minor ways (e.g., describing the study as “three experiments” when one only one was experimental, one was analysis of existing data, and one was quasi experimental), but the biggest problems are practically criminal in the science world. The author correctly (with the exception of the description of the subjects) reported that the relationship between compassion and prosocial behavior was stronger among less religious participants than more religious participants, but omitted the findings which clearly showed that the more religious participants were, in general, more compassionate and generous overall.

When sloppy reporting took over, the result was an utter mess.

On the one hand, the authors claim in their introduction to be interested in what motivates less religious people to act prosocially, so perhaps the spin was the plan all along. However, if they were not interested in religiosity as a variable – if they were not interested in comparing the more religious to the less religious, then they should not have limited their study to the population of interest.

Something I found interesting is that the third paragraph of the press release makes a statement which should have sent red flags up because there was no follow-up that made sense. It started with:

The results challenge a widespread assumption that acts of generosity and charity are largely driven by feelings of empathy and compassion, researchers said.

And yet the rest of the piece focused on the fact that empathy and compassion DID drive generosity and charity in less religious participants, even if it repeatedly incorrectly referred to those participants as “non-believers”. If the author had included the findings that more religious participants were more generous, this would have made sense.

The author errs again with this circular definition of “compassion”, conflating the dictionary definitions of “compassion” and “prosocial behavior” with an effect:

Compassion is defined in the study as an emotion felt when people see the suffering of others which then motivates them to help, often at a personal risk or cost.

Compassion cannot be defined in a study as an emotion felt by someone. That’s a variable we can’t measure directly. In a study, we use operational definitions. In this case, compassion was defined differently in each of the three studies within the article.

The author continues the spin with statements such as (bold mine):

When they looked into how much compassion motivated participants to be charitable in such ways as giving money or food to a homeless person, non-believers and those who rated low in religiosity came out ahead: “These findings indicate that although compassion is associated with pro-sociality among both less religious and more religious individuals, this relationship is particularly robust for less religious individuals,” the study found.

What the bold suggests is that participants who fell into the “low religiosity” category were more prosocial and/or more compassionate. That’s not what the finding means. What was greater was simply the relationship between compassion and prosocial behavior. Because the “high religiosity” participants were more compassionate, they were also more prosocial overall (marginal significance – see discussion below).

Finally, this statement is grossly misleading:

Those who scored low on the religiosity scale, and high on momentary compassion, were more inclined to share their winnings with strangers than other participants in the study.

What participants do in a study is really not interesting. What their behavior tells us about how people behave in the world is. Because there was no statistical analysis comparing the “low-religiosity/high-momentary compassion” group to the other groups, this finding does not allow us to infer anything about the population of interest. You and I could complete a game of Scrabble with a final score of 102 to 103 (respectively), but I would not brag to my friends that I am the better Scrabble player.

Most reports of the study either posted the press release as-is or quoted large chunks of it. Headlines ranged from the simple and correct, if misleading, “Compassion may motivate faithful less” to the still incorrect, but closer “Confirmed: Atheists more motivated by compassion in charitable giving than believers are” to the blatantly incorrect statement suggested by, “Are Religious People Less Compassionate?”

Almost none of the reports describe the samples correctly. The samples were broken into two groups based on measured religiosity, a common practice in social psychology. Thus, there was a range of values and those who scored in the top half were considered “more religious” and the bottom half “less religious”. The “less religious” group cannot be described as “athiests”, “agnostics”, “non-believers”, or even “people low in religiosity”. The “more religious” group cannot be described as “highly religious”, either.

One site that I usually find more accurate contained a much more serious error than an incorrect sample description. It reported a finding not found in the press release that is actually the opposite of what was reported in the study:

Two other experiments also confirmed that more religious participants seemed to be less generous.

They should have stuck to quotes of the press release.

The worst by far, though was on a site called “PsychCentral“. The first paragraph reads:

A provocative new study from the University of California, Berkeley suggests highly religious individuals are less likely to help a stranger than less religious people.

Um. No. That’s not what the study suggests. At all.

and this a few paragraphs down:

Experts say the results challenge a widespread assumption that acts of generosity and charity are largely driven by feelings of empathy and compassion.

Why they added “experts say” to this is a mystery because it’s basically a lie, but this statement is particularly baffling when you consider that the progression of statements: 1) nonreligious are more generous 2) nonreligious are more motivated by compassion to be generous 3) results challenge the assumption that generosity is driven by compassion. On what planet does that make sense?

I’d lament that people don’t actually read, but the worst part is that someone actually wrote this. Sure, it’s mostly quotes and they made stuff up, but they changed some words and moved things around, so they had to pay attention to something.

A friend shared one decent report; it did not seem that the author read the journal article, but at least he read one report (on livescience.com) that is almost a duplication of the press release and put some thought into it. He noted that the findings reported an interaction – that generosity was more related to compassion among the less religious than it was among the more religious – and questioned who was more generous overall. If the press released had not omitted those findings, he would have had an answer, or at least a theoretically-likely hypothesis. Still, he did a little bit of research on his own and noted some well-known findings that the religious tend to give more to secular charities than atheists.

Finally, the press release included a few statements by the authors which were highly speculative. It is standard for authors to discuss possible explanations for their findings, but they are often presented to the public as conclusions the authors reached. In this case as in many others, reports of the study often imply that these things are findings when the study did not examine them at all. For example, the last paragraph of the press release is:

“Overall, this research suggests that although less religious people tend to be less trusted in the U.S., when feeling compassionate, they may actually be more inclined to help their fellow citizens than more religious people,” Willer said.

The rawstory.com report ended the same way, except they changed the last word to “concluded”. This statement serves as a hypothesis for another study, but it is not a finding of the study and cannot provide a conclusion.

I have heard a lot of explanations of what motivates more religious people, but most have been based on personal beliefs or experiences – duty to God, moral obligation, sense of community, etc.  This question may have been answered and looking to other studies might yield something, but this study does not address it – it did not set out to address that question.

A full analysis of the journal article will follow in a separate post later today.

UPDATE: Analysis can be found here.
ANOTHER UPDATE: The award for the most botched report now goes to MSN for this doosey.

Oirignally Posted February 4, 2009

I just caught a report on our local Fox News station about a couple that appeared on “Good Morning New York”, complaining about Elmo’s potty mouth:

Notice that the segment opens with a statement of FACT — the toy has a dirty message — rather than a question about whether this might be true.

This is a perfect example of how human perception is influenced by knowledge. Perception driven by expectation and belief is called “top-down” processing, whereas perception that starts with information from the senses is considered “bottom-up”. Most of our daily perception is top-down in nature.

In many of my classes, I demonstrate this by playing music clips backwards. Some claim that these clips have “hidden messages”. Everyone from the Beatles to Queen to Eminem to Brittany Spears has been accused of it. An entire website is devoted to the study of what David John Oates calls the greatest discovery of all time: Reverse Speech. He even sells training courses and other products (surprise, surprise).

Oates claims that our unconscious is revealing itself through our speech and that these messages can be heard if we listen to recordings of this speech backwards. He spends countless hours listening to audio recordings of politicians, celebrities, and music — listening for anything that sounds like English and documenting it. Sometimes the audio must be slowed down before one can perceive the message.

His website is the source of many of the clips that I use. Fun classroom demonstrations are the only things of value there. My personal favorite is Neil Armstrong saying, “Man will space walk” as he is walking on the moon!

Click here to listen to Neil Armstrong

Oates says, “This is a reflection of his logical thoughts at the time. Man will continue to walk into space.” However, some have claimed that this is evidence that the moon landing was a hoax; that man will walk in space someday, but he is not at the moment.

If you listen to these clips, you will probably hear it, too. So, what is going on?

The “backmasking” and “subliminal messages” paranoia that began during the early days of the cold war has been the object of many studies, all of which point to one conclusion: we do not perceive these messages unless we are told to listen for them. One of the earliest (and best) of these, by Vokey & Read, is discussed in their article in American Psychologist in 1985. The authors conducted several experiments to fully address the question, but the most interesting is the last they discuss.

They started with a recording of the 23rd Psalm because the backmasking cryers of the time claimed that Satan was the source of the messages and religious material was free of them. In addition, they chose a passage from Jabberwocky because it is meaningless when heard forward. After listening carefully they found pieces that could be interpreted as something meaningful, albeit silly. For example, part of Jabberwocky played backwards sounds something like, “Saw a girl with a weasel in her mouth.” When participants in the study were told the message to listen for, they agreed 85% of the time. However, they did not perceive that message when given a different expectation.

This effect is fairly easy to demonstrate. Try listening to the following clips for anything intelligible:

1. Queen Singing “Another One Bites the Dust”

2. The Beverly Hillbillies Theme Song

3. “I Don’t Know How to Love Him” from Jesus Christ Superstar

It’s no fun if you cheat, so listen several times and write down anything that sounds like it might be a message.

Did you get much?

THEN, click here to read the messages. Finally, listen again. It may take a time or two, but eventually you’ll hear at least some of them. Afterward, you will never perceive the jibberish you heard the first time. Instead, you will only hear the message.

In the Elmo clip, notice that one of the women on the street said, “It sounds a lot clearer this time around.” The newscaster in the studio also said that he did not hear the message, but when he played it again he did. This behavior is exactly what I would expect — and what I do expect when I play these clips for students. They never disappoint me because they are human.

Humans do not like ambiguity. We like to make sense of the world and will do so whenever we can. When told what to expect, we often form perceptions that match that expectation. Our perceptions often persist, even when we know that they were formed in this manner.

And there is the fear factor. And the outrage. And the frustration when people do not agree with you, although you believe it is obvious.

There have been many claims of evil in toyland. You might remember a few years ago, when the Teletubbies were under fire because a talking Po doll seemed to say, “Faggot, Faggot”. So, Po is a bigot even though, according to Reverend Jerry Falwell, his buddy Tinky Winky is gay.

Last year Wal-Mart pulled a doll from their shelves because of a complaint that the Doll said “Islam is the light”. Watch this ominous video about a woman who, after taking the doll away from one child, found the same message in a video game she bought for another child. Notice that the report, a broadcast from Indiana, calls it a “discovery” and does not address the question of intention or reality AT ALL.

Yes, it’s eerie that the doll and game sound the same, however, the source of the baby-talk audio may be the same. Both the doll and the game were designed to mimic a baby speaking little more than jibberish.

So, you can blame Fisher Price. You can call Elmo a pervert and conjure up a conspiracy theory. But the evidence says otherwise.

UPDATE

In December I blogged about this case of Hannah Montana’s secret plan to turn young girls into… well… you decide.