When I saw the Washington Post headline “Internet searches are convincing us we’re smarter than we really are” in my Facebook feed yesterday, I was only a little bit skeptical. Most readers are probably aware that I have been studying self-esteem and narcissism for some time, particularly the aspect of overconfidence. Over confidence prevents learning and interferes with rationality, so it is important to understand its sources.

It seems to be a rare moment these days when I can point to a mainstream media piece reporting a finding from the field of psychology without mistakes ranging from a minor distortion of the implications to facts so incorrect that they’ve reported the opposite of what was found. I’m thankful to say that this time the Washington Post’s piece is quite good, although the headline doesn’t quite fit.

The research is not flawless, but the authors address most of the limitations by running a series of experiments. The overall large sample size and a number of controls and checks compensate for the fact that it was conducted online. It’s not perfect, but few studies in this field are. It is just one series of experiments, so any conclusions drawn should be tentative.

All of that said, the findings are interesting. What they found: people who were asked to use the internet to find or confirm their answers to a series of questions gave, on average, higher ratings of their ability to answer questions in a different domain than those who were asked not to consult the internet. This finding held across experiments and the researchers were able to ferret out some details, including:

• This was an increase in confidence among internet users, not a decrease in confidence among those not allowed to consult the internet.
• The increase was not due to access to information or even the use of the internet to get the information. It was the act of searching for that information that caused the increase in confidence.
• Participants were not considering their access to knowledge in their responses about ability, but their knowledge. They appeared to be conflating shared knowledge (the internet) with personal knowledge (what’s in their heads).

In one experiment, half of the participants were asked to search for specific web pages (e.g., a scientificamerican.com page about dimples on golf balls) while the other half received the information on those pages. This shows that the difference in confidence cannot be attributed to the knowledge itself, but the act of using the internet to search for it. In another, participants did not rate their ability to answer the subsequent questions, but instead predicted their brain activity while doing so. We cannot attribute the difference in confidence levels to assumptions that access to information would be available. People really believe they possess the knowledge.

Overconfidence is one characteristic in a list of those associated with narcissism, so it is useful to look at these findings in relation to narcissism in general, especially considering the historical context. The world wide web’s emergence is relatively recent. In The Narcissism Epidemic, Twenge and Campbell document a sharp increase in narcissism over the past 30-40 years. Although they attribute that increase to changes in culture, tracing it back to a best-selling self-help book called I’m Okay, You’re Okay that was popular in the early 1970s, cultural trends of entitlement can be seen decades earlier in “you deserve it” advertising approaches. Certainly the self-esteem movement of the 80s and 90s moved it along. So far, the internet’s blame has been limited to the vanity aspect of narcissism. There has been quite a bit of research suggesting that social media users are, on average, more narcissistic than others, but such correlations are confounded by factors of age, gender, and others difficult to tease out. It would also be extremely difficult to determine the direction of cause.

The experiments are limited to internet search and the specific characteristic of overconfidence, but they do not suffer from the same problems of confounding as those related to social media use. These findings suggest that searching the internet actually causes an increase in one’s self-assessed knowledge. Perhaps people think of the internet as an extension of the self.

The bottom line, I think, is that overconfidence clearly has many sources. Given this research, it appears that we can count the existence of Google™ among them.

Fisher M, Goddu MK, & Keil FC (2015). Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge. Journal of experimental psychology. General PMID: 25822461

Let me cut to the chase: No.

Full disclosure: I own a Fitbit Flex™. I haven’t worn it in months, but I do like the thing. It’s a long, boring, irrelevant story why I’m not wearing it, so I won’t get into that. Just know that I am sincere when I say that I like my Fitbit.

I don’t think my Fitbit is telling me exactly how far I’ve walked or how many calories I’ve burned. I do think that it gives me a ballpark figure that can be used to compare one day to other days. I don’t think that it’s telling me exactly when I am sleeping, either, but the information about how restless I am some nights compared to others is interesting and even sometimes useful.

I also like that it buzzes on my wrist to wake me up in the morning.

Recording some basic information about my daily activity, giving me a reason to record what I eat, waking me up in the morning, and reminding me to get off my ass. These are not big goals and perhaps I don’t need a $100 gadget to accomplish them, but is the device bullshit?

Mother Jones sure seems to think so. Last week the site posted a piece titled “Science Says Your Fitbit is a Joke“. When I searched for more reports on the topic I found a Jezebel piece titled “Your Fitbit is Bullshit, Says Science“, and one on RYOT titled “Science Says Your Fit Bit’s Full of It“.

Since I don’t particularly trust any of these sites as a source of accurately-interpreted science news, especially those which appear to be trying a little too hard to find clever, edgy headlines, I was skeptical.

The Jezebel piece is easily dismissed as simply lazy blogging. The author links to Mother Jones, paraphrasing one of the main points of the piece (bold mine):

Mother Jones points to a new study showing that your iPhone or device of choice does just a good a job[sic], if not a better one, at doing things like tracking calories and measuring activity.

Except that’s not what the Mother Jones piece said. It reads:

But, according to a new study published Tuesday in the Journal of the American Medical Association (JAMA), the apps on your smartphone do the job just as well, or even better—at least in terms of measuring your steps and your calories.

So Jezebel incorrectly paraphrased Mother Jones. And they aren’t the only ones. RYOT wrote this:

A new study came out showing that iPhone apps and other (much cheaper) devices do a better job than the bands at tracking your activity, steps, and even calories.

Even assuming Mother Jones is correct, these paraphrases are not. Walking is, of course, an activity, but it’s only one type of activity. Mother Jones was correct in noting that smartphones can do the work of pedometers, but Jezebel and RYOT are overstepping (pun intended). Wearables such as the Fitbit Flex™ and Jawbone Up™ can track your activity while you are swimming, bowling, or doing jumping jacks. Can your smart phone do that? Well, I suppose if you’re holding it in your hand, it can, but who does that? Also, if you don’t sleep with your smart phone strapped to your body, it cannot track your activity during sleep.

But is Mother Jones right? Might smartphones do a better job? Well, let’s find out.

Although Jezebel linked to the original source, a press release by University of Pennsylvania School of Medicine, it does not appear that the author read the press release, nor did the author of the RYOT piece, who linked only to the press release (failing to even give Mother Jones credit for the thoughts expressed). There is no reference to smart phones doing a better job at anything, much less a list of activities. The only study which is discussed is the one described in the Mother Jones piece, which examined the accuracy of several devices at counting steps and found all to be similar.

But let’s get to the bottom line. Is it a fair assessment that science says these devices are bullshit? A Joke? Full of it?

First, the study discussed involved counting steps. That’s it. There is absolutely no comparison of different devices in regard to tracking calories and no examination of other activity or reports such as distance walked. The study was fairly well-designed–something that does not come across in the reporting. Although there were only 14 participants, each wore every device at the same time and each walked both 500- and 1500-step trials twice.

Here are the results from the 500-step trials:

The results of the 1500-step trials are nearly identical. Now, it appears that the Fitbit One™ and the Fitbit Zip™ are spot-on, and the fact that the error bars are practically non-existent is pretty impressive. Unfortunately, the authors failed to do any sort of statistical analysis at all. Instead, they stated these results:

We found that many smartphone applications and wearable devices were accurate for tracking step counts. Data from smartphones were only slightly different than observed step counts, but could be higher or lower.Wearable devices differed more and 1 device reported step counts more than 20% lower than observed.

Given the error bars for the rest of the devices, including the Fitbit Flex™, I’d believe that most did not differ from one another significantly, with the possible exception of the Nike Fuelband™, which apparently sucks. However, there are two very clear winners, with the Digi-Walker™ close behind. A simple statistical analysis would have confirmed this.

Regardless, it is very clear that smartphones do not outperform most of the wearables. What is really in question is whether people use the devices and use them to their advantage. This may be questionable, but it has hardly been decided by science and it has certainly not been decided in favor of tossing one’s Fitbit™. The JAMA article concludes:

Increased physical activity facilitated by these devices could lead to clinical benefits not realized by low adoption of pedometers. Our findings may help reinforce individuals’ trust in using smartphone applications and wearable devices to track health behaviors, which could have important implications for strategies to improve population health.

So there’s hope that devices, whether they are wearables or just smart phones, will translate to improved health for at least some users.

To be fair to Mother Jones, their piece is much more fleshed out than the other two, covering more than this one study. In my opinion, the Jezebel and RYOT pieces simply plagiarized Mother Jones. However, I saw nothing in the rest of the piece to warrant such harsh criticism of wearables.

Now after writing most of this I came across this wonderful piece on Mashable that was posted yesterday. I could have simply linked to it, saying “read this” because it’s pretty much what I have said, right down to a comparison to the game of “telephone” that I just edited out of this post.

In the end, I have to agree with its author, Chris Taylor, who opens with:

There are poorly designed scientific studies, and then there’s poor reporting on scientific studies by journalists who should know better.

I felt compelled to write about eyewitness memory by a recent blog entry which I feel paints a misleading picture of the nature of memory and the (un)reliability of eyewitness memory.

Other skeptics have written about the subject and coverage has ranged from the pretty good to not so great. This blog post is somewhere in between. I have a great deal of respect for Vandy Beth Glenn and her knowledge on a wide range of topics, but I feel the need to address some issues in this piece that I think are important.

My first problem is that the inaccuracy of memory is overstated.

We don’t remember very well events that really happened to us.

Vandy Beth asks:

[S]hould eyewitness testimony be declared inadmissible evidence in court?… Eyewitness testimony has … brought many infamous individuals to account for their crimes. However, we shouldn’t doubt that at least as many innocents and patsies have been wrongly made to pay as well… We shouldn’t accept a faulty system just because it works more than 50 percent of the time.

How does one get the impression that eyewitness memory is so bad and so hopelessly unreliable that a reasonable solution is to toss it out all together? The literature on the subject of eyewitness memory (let alone memory generally) is vast and complex. Watching a presentation on false memory by Loftus and visiting the Innocence Project website, does not give one the requisite knowledge to make informed policy recommendations. Though I’m sure this is not the extent of Glenn’s knowledge on the subject, these are the only sources cited.

Daniel Loxton has argued that when skeptics write about things outside of their domain of expertise, they must exercise due diligence.

Skeptics solicit … trust. We make the implicit (and sometimes explicit) promise that we are able to provide the nuanced, objective, evidence-based facts. That combination of stated commitment to science, limited qualifications, and weighty ethical responsibilities … place a very high due diligence burden upon skeptics.

When Ed Clint, co-founder of the Skeptic Ink Blog network, and blogger at Incredulous, posted this article, I voiced concern regarding the exaggerated negative portrayal of memory, however both he and Vandy Beth seemed to believe this view was justified.

Ed, essentially began a lengthy explanation about eyewitness memory errors, not quite addressing my criticism and then side-stepping it altogether, saying that it doesn’t matter how many have been wrongfully convicted …

We can’t know if eyewitness testimony is the leading cause of wrongful convictions because most such convictions will never be found out, and the pattern of which get found out could be biased in one or other direction. The evidence suggests that it is, and that’s as far as we can go.

But is that really important here? If anything beats this cause, it is prosecutor misconduct, and reform is needed there, too. But prosecutor misconduct is, at least, a known problem that juries and judges are mindful of (or should be). Are they equally mindful of the problem of unreliability of eyewitnesses? Or do they assume it is correct, if the witness is firm and sure on the stand? I’d guess the latter is the case, and in the most dire need of being addressed. This is true whether eyewitness misidentification is the #1 cause or #10 cause of wrongful convictions. I, frankly, don’t give a fuck what number it is. Steve Titus should not be dead.

Vandy Beth appealed to Elizabeth Loftus’s authority and reputation.

I admit I didn’t do my own research…I relied on my sources, like Elizabeth Loftus and the Innocence Project, to be themselves reliable. If you think they’re wrong, and can back it up, I’d be glad to learn about it.

I am confident that both Ed and Vandy Beth will recognize that I mean no disrespect in the criticism that follows. I consider Ed a friend and, having just met Vandy Beth, hope to one day call her a friend as well.

Yes, Steve Titus (and the many other exonerated individuals whose stories are detailed on the Innocence Project webpage) should not have been convicted. It very rightfully violates our sense of justice that these wrongful convictions have occurred and we feel outrage. This may motivate us to want to offer solutions to the problem that would prevent miscarriages of justice like this from occurring again and give us the sense that we have a good grasp of the nature of memory. But, it is not that simple.

We cannot know the truth in each criminal case, thus preventing us from being able to estimate the true incidence of wrongful convictions as a result of eyewitness memory. However, there is no data of which I am aware that suggests that the legal system gets it wrong in half of all cases as suggested by the text quoted above. A database of 317 exonerations since 1989 tells us that errors occur and that they are not exactly uncommon. But perspective is important, lest we erroneously conclude that the justice system is as broken as we mistakenly perceive our memories to be… According to published statistics from the Department of Justice, in 2010 alone, cases were filed against 91,047 defendants. Ninety-three percent, 81,934, of defendants were convicted. And of those convictions, 97 percent, or 79,260, pled guilty.

Agreement from more than one witness is not much assurance of accuracy, either. Approximately 36% of the first 250 Innocence Project exoneration cases involved the testimony of more than one witness (Garrett, 2011). If all witnesses are subjected to the same biased lineup procedure, it’s not inconceivable that they may all choose the innocent suspect. And proposing to eliminate eyewitness testimony altogether is an absurd “toss the baby out with the bathwater” sort of solution.

It is tempting to assume that we might get rid of problematic eyewitness memory and instead rely on physical evidence, such as DNA evidence, because we perceive it to be more reliable. One problem with this approach is that in 90 to 95% of cases DNA testing is not an available option (Innocence Project, 2014). Furthermore, physical evidence is not immune to error or bias; it can be contaminated or it can be interpreted incorrectly (Murphy & Thompson, 2010; Thompson, 2006). Adherence to established standards and procedures for the collection, storage, testing, and interpretation of that evidence helps minimize the incidence of errors. Eyewitness memory can be thought of as analogous to trace evidence. It is evidence that law enforcement must collect from the mind of the witness while exercising the utmost care not to contaminate it.

Memory researchers would have long ago been done with the context of the legal system when it was first established that eyewitness memory can be inaccurate and unreliable. But the literature does not support such a negative view of memory. If our memories were so poor we would have serious problems functioning in day to day to life and it would not be possible for the layperson to live blissfully unaware of the potential frailty of memory. Elizabeth Loftus’s work on false memory shows us that false memories are possible (e.g. a quarter of participants reported a false event in the famous “Lost in a Mall” study, Loftus & Pickrell, 1995) but not that false memories are so pervasive that we should never trust our memories. “We make no claims about the percentage of people who might be able to be misled in this way, only that these cases provide existence proof for the phenomenon of false memory formation” (Loftus, Coan, & Pickrell, 1996; p.207)

Long before Loftus even began her work on the misinformation effect and later false memories, Hugo Munsterberg (1908) had noted the potential for inaccuracy in eyewitness memory. We could have written off eyewitnesses as hopeless then and moved on to other problems. Instead, Munsterberg and eyewitness researchers since have spent decades trying to identify what factors influence accuracy in an effort to offer more practical solutions to minimize inaccuracies in eyewitness testimony and identifications while also contributing to a better understanding of the nature of memory as a whole.

While confidence, detail, and vividness do not predict accuracy, eyewitness researchers have been exploring the influence of countless other variables as well. We have identified some factors related to accuracy which we cannot control (termed estimator variables, see Wells, 1978) but are nonetheless helpful to be aware of because they signal which instances of eyewitness memory are more error-prone, such as, poor lighting, short exposure to culprit, stress, witness intoxication, the presence of a weapon, mismatch between witness and culprit race, etc. Other factors which can influence accuracy and are under the control of the legal system (termed system variables) have also been explored: whether witnesses are shown books of mug shots, making composite sketches, the specific lineup instructions given, the composition of the lineup such as the number of persons included and how the fillers were selected, the format in which the lineup is presented (e.g. simultaneously or sequentially), blind administration, etc. All of these variables are at play and this is what forms the complexity that requires greater familiarity with the primary research.

Regarding eyewitness identification research specifically Wells & Loftus (2003) say “The primary lesson of the eyewitness identification work is that mistaken identification rates can be very high under certain conditions and many of these conditions could actually be avoided by the use of scientific procedures for lineups”, p.150. This research has helped to inform important guidelines and recommendations for law enforcement agencies; a committee of experts assembled by The Justice Department published a set of guidelines for law enforcement (Technical Working Group for Eyewitness Evidence, 1999). Increasingly more jurisdictions (state and local) are adopting reforms aimed at improving the accuracy and reliability of eyewitness identifications and testimony; for example, approximately 32% of law enforcement agencies have switched to a sequential lineup procedure (Police Executive Research Forum, 2013).

Memory is certainly malleable and it is reconstructive, but this does not mean that eyewitness testimony can never be trusted or be valuable in court. It can … IF certain conditions are met.

REFERENCES

Garrett, B. (2011). Convicting the Innocent. Harvard University Press.

The Innocence Project (2014). Unreliable or Improper Forensic Science, retrieved from: http://www.innocenceproject.org/understand/Unreliable-Limited-Science.php

Loftus, E.F., Coan, J.A. & Pickrell, J.E. (1996) Manufacturing false memories using bits of reality. In L. M. Reder (Ed.) Implicit memory and metacognition. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 195-220.

Loftus, E., & Pickrell, J. (1995). The Formation of False Memories Psychiatric Annals, 25 (12), 720-725 DOI: 10.3928/0048-5713-19951201-07

Munsterberg, H. (1908). On the Witness Stand.

Murphy, E., & Thompson, W. C. (2010). Understanding potential errors and fallacies in forensic DNA statistics: An amicus brief in McDaniel v. Brown Criminal Law Bulletin, 46 (4), 709-757.

Police Executive Research Forum (PERF), & United States of America. (2013). National Survey of Eyewitness Identification Procedures in Law Enforcement Agencies.

Technical Working Group for Eyewitness Evidence. (1999). Eyewitness Evidence: A guide for Law Enforcement. Washington, DC: National
Institute of Justice.

Thompson, W.C. (2006). Tarnish on the “gold standard”: Understanding recent problems in forensic DNA testing The Champion, 30 (1), 10-16

Wells, G. (1978). Applied eyewitness-testimony research: System variables and estimator variables. Journal of Personality and Social Psychology, 36 (12), 1546-1557 DOI: 10.1037//0022-3514.36.12.1546

Wells, G. L. & Loftus, E.F. (2003). Eyewitness memory for people and events. A. M. Goldstein (Ed.) Handbook of Psychology. Vol 11 Forensic Psychology (I.B. Weiner, Editor-in-Chief). New York: John Wiley & Sons, pp 149-160

A piece by Joseph Brean that I read recently displayed the headline New hope for narcissists: New Canadian study suggests there may be a cure for self-centred[sic] grandiosity after all. My first thought was “I doubt it.”

The press release for this study is pretty accurate and, although I have some criticisms of the study, the journal article doesn’t make the dubious claims found in this piece. However, I will say that it’s a little hard to tell if Brean is fully responsible because it is unclear whether he interviewed the scientists who wrote the article himself. Even if he did, it’s hard to know what that interview looked like. I’ve been on the other side of such interviews and usually the end product is a gross misrepresentation of what I’ve said.

This piece also brought to mind the issue that free and easy access to original sources has some serious trade-offs. Laypersons often misinterpret studies (hell, scientists often do) and in communities such as skepticism, some of those laypersons speak those misinterpretations on stages with an air of authority. But more importantly, some calling themselves science writers misunderstand, cherry-pick, overgeneralize, over-extrapolate, and otherwise misrepresent the implications of findings, especially those in the social sciences. With the vast majority of the public (including a lot of scientists short on time and resources) getting most of their science news from non-scientists and trusting those sources, I find this to be a bit of a problem.

So let’s take a look at the statements made in this piece and what I think is wrong with them, given the original study. The National Post piece introduces the study this way:

…narcissist is usually described as the product of a long and complicated psychological development. Like hypochondriacs, narcissists are made, not born.

New research out of Wilfrid Laurier University, however, suggests narcissism might be simpler than that. More than just a moral failing or psychiatric symptom, narcissism might reflect a basic mechanical failure of the brain’s natural tendency to mimic.

Um, no. No, the research does not suggest that, nor do I believe that descriptions of narcissism imply that it’s all about parenting. Most disorders, including personality disorders, appear to be the result of a complex interplay of environment and genetics.

But where the author really goes off track is in the next sentence:

Intriguingly, it also suggests that narcissism’s opposite, empathy, might even improve with practice.

Um, what?

When I read the press release, I saw nothing in it that even hints at practice effects, so either Brean (the author) actually read (and misinterpreted) the article or he gleaned this bad information from the interview to which he refers. Or perhaps he just made it up because he thought he understood the research. Who knows?

Then there’s a quote of the lead author. Obhi, which reads, “Narcissists don’t imitate automatically.”

If that’s a quote, I hope it was taken out of context, because it reads as a statement of fact, a given, when it is merely one possible explanation for the findings, and one that does not appear in the journal article. In fact, it contradicts the findings and even the title of the journal article: Automatic Imitation is Reduced in Narcissists. This is a poorly-chosen title for at least two reasons that I will discuss below.

So what can we reasonably take from this study? Well, let me first summarize the study.

Obhi, Hogeveen, Giocomin, and Jordon conducted a rather simple study with a final sample of 24 subjects. The subjects performed a task which involved responding to a cue by lifting one of two fingers off of the keyboard. The cues were embedded in images of hands in which one of the fingers is raised. For some trials, the cue matched the position of the hand in the picture (e.g., the picture showed the index finger raised and the cue instructed the subject to raise their index finger) and in some trials the cue was incongruent with the picture (e.g.,  the picture showed the index finger raised and the cue instructed the subject to raise their middle finger).

This task is a paradigm that many readers will be familiar with, even if you don’t recognize it immediately. It is similar to the Stroop Task, the Simon Task, and the task used in Implicit Attribution/Association Tests.

Essentially, the task involves suppressing an automatic response–going against one’s initial, automatic response to a stimulus. For example, the classic Stroop Task involves identifying the color of ink in which a word is printed. The trick is that the words the participant is looking at name various colors. Nearly everyone will take longer to name or respond to the ink color when it is different from the word than when the word is the color of the ink.

The Stroop Test. It takes longer to identify the ink color when a color word does not match the ink color (bottom) than when it does (top).

This well-established effect demonstrates how automatic reading can be. Automatic behaviors are those which require few cognitive resources and are sometimes performed without much awareness. Automatic responses are often very difficult to “shut off”. When we attempt to perform a similar task which competes, the automatic response must be actively suppressed. This is what happens in the Stroop Task; we must suppress the response of reading the word in order to identify its ink color. The differences in reaction time between color-congruent and color-incongruent trials are a good measure of how much the automatic task interferes with the primary task.

The Simon Task and Implicit Attribution tests take advantage of something we call mirror responses. Research has confirmed that we see someone do something (e.g., as simple as raising a hand), our brains automatically respond with activity which is very similar to what we might see if we were performing the task ourselves. This response has many advantages, including facilitating joint attention. Joint attention is demonstrated when you move your eyes to see what someone else is looking at.

Many automatic tasks such as reading are acquired through lots and lots of practice, but not all. Some, like joint attention, are probably innate reflexes. Mirror responses are probably a mixture of both.

Obhi and colleagues also asked participants to complete a short version of the NPI [Narcissistic Personality Inventory] and divided them into two groups (“high” and “low”) based on their NPI scores. They found greater differences in reaction times (RTs) between congruent and incongruent trials among those in the low narcissism group than among those in the high narcissism group. In other words, participants who scored higher in narcissism demonstrated less interference than those who scored low in narcissism. Even more pronounced, however, were the differences in accuracy. Those high in narcissism erred in about 5.3% of the trials while the error rate for those low in narcissism was more than 12%.

As an aside, remember that quote about narcissists not imitating automatically? Well, the findings suggest otherwise. There certainly was an interference effect for both groups. Those high in narcissism did indeed respond more quickly and with fewer errors when the cues matched the images. Furthermore, the title of the article was Automatic Imitation is Reduced in Narcissists, yet they did not study narcissists. Subjects were not selected for their NPI scores and no diagnoses were recorded. The participants in the “high” group were simply those whose NPI scores were in the top half of the subjects tested.

The authors discussed possible explanations for these findings thoroughly. Essentially, the best explanation they give is that those high in narcissism more easily suppress mirror responses. This could be due to greater self-regulation or it could be that mirror responses are not as automatic. I cannot account for the decision to title the paper with one of those explanations, except that they dismissed the self-regulation hypothesis with a non sequitur.

What I can say for certain is that the findings do not suggest that narcissism is a product of poor mirror responses. They also do not suggest that practicing such responses, which are largely learned implicitly, would make them more automatic or make the individual more empathetic and less narcissistic. Neither of these hypotheses is impossible, but neither is likely, either.

The authors also acknowledged many of the study’s limitations, including the relatively small sample size, but my reaction to this study remains mixed. On the one hand, I applaud these researchers because although it seems that this area should have been well-studied, it isn’t. On the other hand, this is a very simple study to execute and the paper has four authors, yet they completely missed the opportunity to ensure strong findings by putting in just a little bit more work.

The study would be improved leaps and bounds by running a larger sample and testing whether self-regulation or motivation were factors. The latter can be easily accomplished by including a condition with non-social context (e.g., arrows instead of fingers). Furthermore, they could have pre-screened the subject pool and recruited only those who scored exceptionally high or exceptionally low in narcissism, thus raising their power tremendously.

That said, these findings are consistent with what we know about narcissism. A key feature of narcissism is reduced empathy and it stands to reason that reactions to the actions of others would be more automatic in those with greater empathy.

In the discussion section of every study is a laundry list of possible explanations and implications for the study’s findings, but most of these are speculation and usually labeled quite clearly as such by the authors. This does not seem to stop laypersons and “science writers” from accepting the speculations they find most interesting or desirable.

In this case, the implications of the study for a behavioral test of empathy are much more plausible than those for treatment of narcissism.

h/t Tim Farley of WhatsTheHarm.net and Skeptools.

Obhi SS, Hogeveen J, Giacomin M, & Jordan CH (2013). Automatic Imitation Is Reduced in Narcissists. Journal of experimental psychology. Human perception and performance PMID: 23957308

While I was on vacation, I missed a post by Sharon Hill on Skeptical Inquirer online. She recently re-shared the piece on Facebook, so I had an opportunity to give it a good read. Sharon’s pieces are usually filled with thoughtful reminders to reign in arrogance and do more than just tolerate other view points, embrace them and learn from them. I highly recommend following her regular columns there or at her blog, Doubtful News.

Is science hazardous?

This recent piece seems to be in response to the current discussion about the limits (and lack thereof) of science, such as this piece by Steven Pinker. However, it lacks the nuance I’ve seen in criticisms of Pinker’s piece.

Hill’s piece seems to define scientism, science, and several other terms somewhat vaguely, oversimplifying the issue and overcomplicating it at the same time. She begins the argument by claiming, if I may use an analogy, that there are many different ways to skin a cat, but then goes on to support that claim by pointing out that there are questions about whether the cat should be skinned, how much the process will cost, and whether the cat has a name. Answering these questions and skinning the cat are different tasks with different goals.

But it is this claim that I take the most issue with:

People who advocate fanatical reliance on science—where all competing methods of gaining knowledge are illegitimate—are practicing scientism.

This definition may very well put me in the category of “practicing scientism”, but it depends on what she means by “illegitimate”. While I recognize that personal knowledge can come from any number of methods and sources, respecting personal knowledge is not a reasonable stance when it comes to enacting policies and making choices which involve other people. To make the best choices, we need to rely on shared knowledge.

And I certainly do believe that empirical methods are necessary to gain genuine, reliable information about the world. In fact, that’s a basic assumption of science (more on that later).

The “just apply science” plan is an overly simplistic solution that not everyone will automatically buy into. There are other, also valid ways of evaluating problems. All the world’s problems cannot be solved by throwing science at it. At least not now (probably never).

This is a confusing statement with twists and turns.

First, whether or not “everyone will automatically buy into” a solution is no measure of the solution’s value.

Second, the statement about evaluating problems conflates the different tasks and goals associated with solving problems. Science, philosophy, and the humanities are different animals. To complicate matters, science incorporates philosophy and the humanities incorporate some scientific thinking. None of these things can tell us what to value, either.

For example, philosophy studies problems; it doesn’t solve them. Philosophy can only provide a way of thinking, not the information that one is to think about. Science, for that matter, doesn’t solve problems, either. It seeks and provides information and explanation. Technology solves problems, but it doesn’t do so just by thinking about them. Technology uses the products of science and scientific thinking (which includes products of philosophy) to solve problems.

So, this seems like a lot of apples and oranges and bananas to me.

The piece also contains more than a few straw men. For example:

For a start, scientism has utility problems. If we need to justify everything with empirical evidence, and then justify that evidence with evidence, and so on, not only do we get bogged down in minutiae, we end up in a scientistic loop which we can’t resolve. There must be a point where we accept a premise as a given – that reality is real, that we aren’t being fooled by a devious creator.

This is not only a straw man, it’s a misleading. Science does accept several premises as givens. In most college-level introductory science textbooks you can find these listed as “canons” or “assumptions”. For example, science assumes that the universe is deterministic, that all events have natural causes. Without this assumption, science can tell us nothing about the world with confidence because anything we observe might be explained by the supernatural.

So in a sense, the argument supports “scientism”.

Hill goes on to admonish over-enthusiasm for science because it “can mask the attention that should be paid to human social issues that are too complex…”, yet her examples are not issues too complex for science, but questions of policy which involve more than just information (e.g., one example involves the ethical question of whether to carry a fetus to full term knowing that it will be born with a debilitating condition). Science informs values, it doesn’t dictate them. However, values can’t answer those questions by themselves any more than science can.

Look at our laws. Many are informed by science (cigarette restrictions, driving after alcohol consumption, environmental regulations) but are tempered by other human interests such as personal pleasures, social norms and economic considerations.

Again, this seems a bit of a straw man. While there are those who claim that science can dictate values (which are embedded in each of those “human interests”), that is not a typical view and does not seem to be the view that Hill is railing against.

Science cannot tell us what we value or what we should value, but without scientifically-derived information and thought processes, we will fail to make choices and policies which promote those values.

Here is an example from my recent talks at TAM2013 and Dragon*Con, as covered in What Intelligence Tests Miss by Keith Stanovich:

In a study by Ubel, participants were asked to allocate 100 livers to 200 children who needed transplants. The children were presented in two groups: A and B. As you can imagine, most participants divided the livers equally, giving half to one group and half to another.

However, when the participants were told that the children in group A had an estimated 80% average chance of surviving the surgery, while the children in group B had only an estimated 20% average chance, the allocations varied much more. About one quarter of the participants gave all of the livers to group A, one quarter gave half to A and half to B, and half of the participants distributed the livers in a manner in between these two choices (i.e., one quarter gave 75 of the livers to group A and 25 to group B).

When asked why they gave livers to group B, participants justified their actions by saying things like “needy people deserve transplants, whatever their chance of survival.” This, of course, ignores the real question, which is how to allocate a limited number of livers to save the most lives. It tells us nothing about why the individual chose one child over another.

Participants in another study were given the same task except that the recipients were not grouped. Instead, they listed the recipients individually, ranked by the individual chance of survival. If the justifications were true, we would expect at least 25% of the participants to allocate the livers to every other child, or somewhat randomly down the list. Instead, participants had no problem allocating all of the livers to the top 100 children on the list.

The difference between the answers when the children are grouped and the answers when they are listed individually is called a “Framing Effect”. The way the problem is framed determines how a majority of the participants respond to it.

Now, science can’t tell us what’s “right” in this situation, but it can sure tell us how to meet our goals once we have decided what those goals are.

Let’s assume that our goal is to maximize the number of children who will be saved. Rational thought tells us that, given that goal and the choice of the two groups, we should give all of the livers to group A (science tells us that those are the children with the best chance for survival). The difference between that choice and the equal distribution is an expected 30 dead children.

It should be obvious from this example that considering our values and goals is not enough to make the best choices. We need good information and good thought processes to make the kinds of decisions that allow us to meet our goals.

One more statement that got under my skin:

When we overly indulge our science bias in informing decisions, such as in the realm of policy, the risk of making an unpopular guidance or rule increases.

Wait a minute. Is our goal to put the most popular policies in place or the best policies? For my part, I want policies that are best for society and the individuals within it. I don’t care if they are popular or not.

Science is not perfect or infallible, even when implemented correctly. Our knowledge is incomplete, which means that we will make a lot of mistakes when we take actions based on that limited knowledge. However, it will always beat human judgments in the long run, allowing us to make the best decisions and take the best actions toward our goals.

Works cited:

Stanovich, Keith E. (2009). What intelligence tests miss: The psychology of rational thought. New Haven, CT: Yale University Press.

Ubel, P.A. (2000). Pricing life: Why it’s time for health care rationing. Cambridge, MA: MIT Press.

I hope I grabbed your attention with that title, but do not expect to find the answer to that question here. What I am going to discuss today is a study that many people seem to think answers that question, but it doesn’t.

As I noted in my last post, the study I’ll be discussing was grossly misreported, starting with its press release. Since the study itself appears to be behind a pay wall for most people, I’ll describe as much detail as I can in a blog post as I discuss the study’s validity and findings of the study, published in the Journal of Social Psychological and Personality Science and titled “My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals”.

But for those who are not at all interested in the research methods or a breakdown of why I rate the quality of the study the way I do, I will give you the the bottom line so you can skip the rest or only read the sections that interest you (I’ve used headings to make it easier).

Summary

I think that the findings will hold up to replication, despite some issues I have with the way they did a few things. Overall, the research quality is quite high.

The groups they compared did not include atheists, agnostics, believers, non-believers, highly religious, or any other label that you can throw at it. In the studies they used raw religiosity scores and made some comparisons of “higher” and “lower” using values from the distribution. In a sense, the compared those who scored in the lower half of the sample to those who scored in the upper half.

They found:

• Differences in prosocial behavior cannot be dismissed as due to political affiliation, socio-economic status, or other factors often held up as responsible.
• Religiosity is correlated with trait compassion; the more religious, the more compassionate.
• Trait compassion is related to prosocial behavior in general. This relationship is stronger in the less religious than in the more religious. This does not mean that the less religious are more compassionate (see number 1) or that the less religious are more prosocial. It just means that compassion is a bigger factor in prosocial behavior in the less religious.
• The findings of the first study can be interpreted one way that isn’t discussed in the paper: when the relationship between compassion and religiosity is accounted for, the more religious are not more prosocial than the less religious.
• The findings in the second study, which involved inducing feelings of compassion, were similar for generosity, except that the more religious were more prosocial even after accounting for compassion.
• The findings of the second study also included a different pattern when the prosocial behavior was giving to charity. Compassion induced more giving, but the effect was weak and did not differ much across religiosity. Religiosity had a significant affect on charity. This can be explained by the guidelines provided by many churches for how much of one’s salary one should give.
• In the third study, in which state compassion (how compassionate the individual felt at that time) was measured and the prosocial behavior measure involved real-world cash, religiosity was not related to either compassion or prosocial behavior.
• In the third study, state compassion was positively correlated with prosocial behavior, but the effect was greater in the less religious than in the more religious.

What the findings as a whole say to me, and what I believe the press report tried, but failed, to express, at least with convincing support: We do not need religion to be prosocial. We need compassion.

This is great news for secularists.

However, it doesn’t say anything negative about religion or the religious, nor does it provide anything that should make atheists feel superior. It just shows that one can be good without God; that motivations can come from other sources.

Now on to the details…

NOTE: to keep this as short as possible, I’ve included a lot of links to terms and demonstrations. Where I describe problems in more detail I still water-down quite a bit. I will do my best to make it understandable without rambling on and on, but keep in mind that it takes many years to learn enough about research design and statistics to understand why some of these are problematic. Furthermore, not all researchers will agree on the consequences of some of these problems. I am still learning this stuff myself (probably always will be learning).

The Study (description)

The article reports three studies, each related to the relationship between compassion and prosocial behavior in less-religious individuals. I have created graphs using the information in the paper, but in some cases I did not have exact numbers, so while the relationships are visually accurate, there are only values where I could use exact numbers.

Theoretical Foundation

The introduction discusses research which documents that religiosity is associated with prosocial behavior. Specifically, religious people give more and volunteer more than nonreligious people, over and above what they give to and do for religious organizations. The researchers note that the nonreligious do give; when we compare groups, we do so using averages. However, it may be that the motivations for prosocial behavior vary in a way that interacts with religiosity. In other words, the more religious among us may be motivated to prosocial behavior by one set of factors and the less motivated by another.

The researchers hypothesized that compassion is a more influential factor in prosocial behavior for the less religious than for the more religious among us.

Study 1

The first study examined the relationships among religiosity and traits of compassion and prosocial tendencies. What this basically means is that situational factors were not involved; traits are a matter of personality or attitude. For example, “trait anxiety” refers to how anxious a person is in general, while “state anxiety” refers to how anxious that same individual feels in a given situation.

This study involved analyzing data from a 2004 “survey”. I put that term in quotes because it usually refers to a set of questions that do not measure more than what is apparent at face value. Established measures of latent variables (variables which cannot be measured directly such as feelings and attitudes) are usually called an “inventory” or “scale” and we refer to them loosely as “measures”. In this case, the survey involved such measures and I want to make that clear.

The sample was comprised of 1337 participants and covariates (variables other than those of interest which could explain differences among the groups) of gender, political orientation, and education were included in the analysis. The variables of interest were religiosity, compassion, and prosocial behavior. Religious identity (identification with a specific religion or no religion) was also considered.

Results

Correlations

• Covariates had little impact on the results.
• Trait compassion was positively correlated with religiosity* and prosocial behavior. On average, the more compassionate the individual, the more religious they were and the more the more prosocial they were.
• The relationship between religiosity and prosocial behavior was marginally significant (statistically).

Hypothesis Test (See Figure 1)

Figure 1: Study 1 results. "Higher" and "lower" are defined here as 1 SD from the mean.

• A regression analysis revealed an interaction of religiosity and compassion on prosocial behavior. What this means: The effect of compassion on prosocial behavior differed among levels of religiosity.
• More specifically, the level of trait compassion affected prosocial behavior less as religiosity increased.
• There was also a main effect of compassion, but that was apparent in the correlational analysis.
• There was no main effect of religiosity on prosocial behavior. This is interesting, because they found a marginally significant correlation, but it does not mean the there are no difference in prosocial behavior. I would interpret these findings, when put together, as suggestive of little or no difference between the more religious and the less religious in prosocial behavior over and above the differences accounted for by compassion.

The authors discuss the findings a little differently, though, focusing on the differences in the way that compassion affected prosocial behavior (the interaction in the first hypothesis test result) and ignoring the way that the effect of religiosity disappeared when compassion was entered into the equation. It seems more interesting to me to treat compassion as the moderator. It also makes more sense in the end.

Study 2

This study was experimental in that the researchers manipulated state compassion. In other words, they induced feelings of compassion in half of the participants and compared the amount of prosocial behavior those participants engaged in to the amount of such behavior in a control condition.

The sample included 101 participants and the study was conducted online, so the age range was exceptional (from 18 to 68 years). Participants were randomly assigned to one of the two conditions and each watched a short video under the guise that there would be a test of memory afterward.  The videos were established manipulations of feelings of compassion and neutral emotion (i.e., other researchers tested their effectiveness). Following the video, participants completed two tasks which are well-established measures of prosocial behavior commonly used in such research.

Results

• Again, covariates had little impact on the results.

Hypothesis Tests (See Figure 2)

Figure 3: Results of Study 3. Values are relative.

There were two tests since the participants completed to different prosocial tasks, one involving generosity and the other involving charity.

For the generosity task:

• This time there were a main effects of both religiosity and compassion on prosocial behavior. The more religious, the more prosocial. Those who watched the compassion-inducing video were the more prosocial on average than those who watched the neutral video.
• The interaction appeared again in the manner as in Study 1.

For the charity task:

• There were main effects of both religiosity and compassion on prosocial behavior.
• There was no interaction.

This is where they screw up, in my opinion.

The pattern of the moderation was in the predicted direction but failed to reach statistical significance.

This is not an acceptable statement unless the findings are marginal. This was not. The p-value was .408. This is not even close to meaningful. Still, they went ahead with the analysis of the interaction and reported an effect of compassion on charity for the less religious participants and no effect for the more religious.  The problem is that post-hoc analysis like this assumes that a significant interaction was observed. Their tests inflated alpha (the probability of a Type I Error) and can only mislead. They stated that they had found “partial support” for their hypothesis, but they did not in this case.

The relationships in the generosity task are very clear when we look at a Figure 2. The interaction is the interesting finding. Compassion had little effect on the more religious, but a very large effect on the less religious, who gave practically nothing when compassion was not induced. There is no analysis to tell us if the less religious surpassed the religious by a statistically significant amount when compassion was induced, but they were clearly out done by the more religious when not made to feel compassion.

The charity task showed no such interaction and the authors did not include a graph of this effect that I could recreate, nor did they provide the information to make one.

Study 3

For this study, the sample of 120 completed a state compassion inventory (a measure of their feelings of general compassion at the moment) and a series of “economic tasks designed to measure their generosity, trust, trustworthiness, and motivation to reward others’ generosity.” What differed in this study, however, was that the ‘points’ they earned in these tasks could be exchanged for cash at the end of the study. Participants did not know how much cash, but they knew that the more points they earned, the more cash they would receive.

Results

Hypothesis Tests (See Figure 3)

The findings of this study were very different from the other two.

• State compassion was not related to religiosity.
• Religiosity was not related to prosocial behavior.
• There was an interaction of religiosity and compassion on prosocial behavior. The amount of compassion felt had more of an effect on the behavior the less religious than it did on the more religious.

The graph of these findings, a reproduction of their graph since they did not provide information to create one that would make more sense (to me anyway), is a bit misleading. The values are z-scores, so they are relative to one another and not actual values. What is interesting, though is how little the prosocial score varied in the more religious group and how that line barely dips below the mean value (represented by 0).

There is also a problem with the press release in that it makes the claim that the high state compassion/less religious group out-performed the others. There is no statistical analysis comparing the groups in that way, so this is a misstatement. We do not know if less religious individuals are more generous than more religious when motivated to act prosocially. We just know that they are more generous when motivated by compassion than when compassion is low.

The Study Overall

As I noted, my opinion of the studies as a whole is relatively high, but I do have some major criticisms. Some of the language makes me cringe (e.g., results are the product of statistical tests, so “We tested our results”…), but I have seen more and more of this as scientific reports in general have grown sloppier.  Study design and method is much more important, as is the quality of the reporting beyond language.

The authors also throw around the term “robust”, claiming in the first study that the relationship between compassion and prosocial behavior is “particularly robust” for less religious individuals. That term refers to findings which are “sturdy” and will stand up when some supports are removed – effects which appear to hold up in different situations. Since this was one analysis of one data set, that term just doesn’t work. It does not fit in any of their uses of it.

In fact, they err in Study 2 by saying that the effect was “attenuated” for the more religious. That term is relative; attenuated compared to what? The effect was not “robust” in one condition and “attenuated” in another; they can only be compared to each other. The effect was greater in the less religious than the more religious.

Missing Information

There are a number of bits of information which are considered to be, at minimum, required for a good research report. A general rule of thumb for methods and results sections is to include enough (without being redundant) information to allow other researchers to replicate (in a strict sense) the study and to confirm that the statistical findings are properly interpreted.

I am not sure that this article meets that criterion. The methods are pretty well fleshed out and the paper is full of statistics, but some descriptive statistics are missing that I would have liked to have seen (e.g., means reported overall for measures, but not by group) and there was not enough of the right information to recreate them.

Grouping the Data and Errors of Generalization

One overall criticism which warrants discussion is in the grouping of data. There are some problems with this and they are related. The sensitivity of the religiosity measure is one problem that, by itself, is not a big target for criticism. Combined with the second problem of grouping participants, though, it becomes more serious.

The practice of comparing groups of people based on a variable which is distributed on a spectrum is a common one. The question the researcher wants to answer is important in deciding whether to group and, in this case, I do not disagree with that choice, but I question how they grouped and how it was communicated. If the data are clustered (the distribution is multi-modal), grouping is simplified, but if the data are distributed more loosely, it can be tricky and dangerous.

First, the researcher loses information, therefore they lose sensitivity and usually lose power. The sensitivity problem is relevant in the first study, but mostly because it makes the findings difficult to interpret.

Second, if the way that the grouping is communicated is not consistent and clear, it is likely to be misinterpreted, compounding any existing problems with the method. I discussed this problem in my last post. Most of the reports referred to the groups compared as “highly religious” verses “atheists and agnostics” or something like that. However, where are all of the people in the middle (i.e., most likely the bulk of the sample)?  Within each group there was variation in religiosity and comparisons are made using averages. Generalizing only works when the samples are representative of the population of interest and this applies in either direction of the generalization (i.e., specific to mixed or mixed to specific).

Third, researchers must decide where to draw the lines between high and low (and anything in between). Since the majority of variables in psychology are normal distributed (therefore symmetrical), the lines are usually drawn using rankings of sample values and the most common way to split a sample in half is to put all values above the median into “higher” and those below into “lower” (called a “median split”). However, ease is not a good reason to use this technique.   Here is an interesting demonstration of the dangers of dichotomizing normally-distributed variables.

But… religiosity is not usually distributed normally; it’s usually skewed. Skew means that it’s not symmetrical, so a median-split would make even less sense.

In this case, it seems that the authors tried to have the best of both worlds by treating religiosity as a continuous variable, but doing post-hoc analysis on it, discussing it, and graphing it as if it were dichotomous, choosing values which were one standard deviation from the mean in both directions as the central tendencies of each group. The biggest problem with this is the assumption of normality. If the variable is not normally-distributed (and I suspect that it is not), this grouping is a bit tough to swallow.

When this problem is mixed with a limited range as it is in the first study (the religiosity scale only had four points), it’s a problem. The four possible values were 1 = no religion, 2 = not very strong (religious identity), 3 = somewhat strong, and 4 = strong. Since the mean was 2.99, the bulk of the sample were fairly religious. one standard deviation (1.03) below the mean is not exactly in non-believerland and one above is off the scale (literally). It is just very difficult to see where “higher” leaves off and “lower” takes over.

Although the range is adequate in the other two studies, the problem of discussing groups which do not actually exist and have fuzzy definitions remains. In my opinion that is one of the reasons it was so misreported.

But, overall, the research is of a relatively high quality and interesting. I would like to see more variation in the prosocial tasks, given that the outcome of the charity task was so different from the tasks of generosity.

It seems that the less religious are at least as generous as the more religious, but their reasons for acting prosocially differ. I would like to see the day when, as a group, we are generous and prosocial consistently, without the need to be provoked and without needing to feel an emotional connection to the receiver.

Saslow, L., Willer, R., Feinberg, M., Piff, P., Clark, K., Keltner, D., & Saturn, S. (2012). My Brother’s Keeper? Compassion Predicts Generosity More Among Less Religious Individuals Social Psychological and Personality Science DOI: 10.1177/1948550612444137

Many Skeptics, Daniel Loxton and Massimo Pigliucci (So you think you’re a skeptic, don’t you?) come to mind, have discussed the need to restrict one’s public discussions in the name of Skepticism to topics within one’s area of expertise. In the absence of such expertise, we should only convey to the public a scientific consensus, if one exists. So how is a non-scientist or someone working in a different field supposed to know whether a scientific consensus exists and/or what that consensus is?

Well, that is what I had initially intended to write about today. I am afraid this post goes a little off-track, but it still covers important ground.

This post began as a set of corrections to some of the misleading statements in a recent post by Rebecca Watson in which she points out what she believes is wrong with Ben Radford’s use of an article to support his argument that images of thin women in media do not “encourage” eating disorders such as anorexia nervosa. A friend alerted me to this post, knowing that I participated in a discussion on Facebook on the matter.

In her post, Rebecca accuses Ben of cherry-picking and quoting out of context. In a response to Rebecca , Ben defends himself quite well regarding the accusations that he misrepresented the authors (there were two), but I think that there is a lot missing from the discussion that is important and, in fact, this is shaping up to be a great example of why organized skepticism is needed and why more working scientists should get involved.

First, I share Rebecca’s concern that readers will assume that Botta’s findings support Ben’s claim. Ben writes,

Rebecca is assuming that the quotes were selected as representing the conclusions of those particular studies from which they were cited. I made no such claim.

This is true, but Rebecca assumed what most laypersons are likely to assume. It is far too easy to mislead when discussing complex topics such as this one. Evidence to support one’s assertions is difficult, too, when the assertion is that a claim is untrue. I think that Ben’s argument would have been better served by a more detailed discussion of the complexity of the issue and, perhaps, a quote from a recent review of the literature. Those are difficult to come by, but they provide the “big picture” view to which he referred.

Quoting from the introductions of research reports as Ben did is never a good idea, in my opinion, because it is hearsay. However, it is also problematic when people assume that a researcher’s conclusion can support an argument; it can’t. What counts as evidence is a finding.

For example, what prompted me to participate in the Facebook conversation was Rebecca’s comment:

Study after study shows that straight women and gay men develop eating disorders because our society tells them that they must be thin and pretty in order to attract a man (eg: [link to study by Siever]), and that attracting a man should be their #1 goal in life.

The study to which she linked found correlations among disordered eating, body dissatisfaction, the importance placed on physical attractiveness, and sexual orientation by gender. How that was translated into a causal chain involving societal influences and personal goals is quite alarming, but the paper itself provides some clues. The discussion is longer than any other section of the paper, less than parsimonious, and somewhat speculative. This is one of the reasons that researchers cite and discuss findings, not the conclusions other scientists draw.

As I explained to Rebecca,

Correlation does not equal cause.

Causal conclusions are extremely difficult to draw when you cannot randomly assign subjects to conditions and you cannot randomly assign people to be a straight women or gay men. It takes massive amounts of converging evidence from a variety of studies which eliminate rival hypotheses.

So to say that “straight women and gay men develop eating disorders because our society tells them [anything]” is a very bold statement that is not supported by the evidence. It is certainly not supported by the study to which you linked.

But the confirmation bias leads people to follow their current beliefs and demand to be proven wrong. For example, Rebecca ended her post with,

In the Facebook thread, I genuinely wanted to see what evidence actually supported the idea that the link between media and body image is a myth, but I got nothing from Radford or anyone observing the thread. So, I’m forced to continue to side with what appears to be the consensus opinion: the media’s portrayal of the thin ideal most likely negatively impacts the body image of those who process those images poorly.

First, the “who process those images poorly” part is a new assertion and one that reveals her understanding (or lack of understanding) of the findings. What does it mean to “process images poorly” anyway? She initially wrote, “processing them the wrong way”. Is there are “right way” and a “wrong way”? Botta’s variable of “image processing” involves whether the individual questions the weight of characters as realistic, whether they believe they are realistic, and whether they compare themselves to the characters. There’s no right or wrong here.

What’s more important is her comment that she wanted evidence that the claim is untrue. As I explained to Rebecca on Facebook, that’s not how skepticism works. The burden of evidence is on those making the claim, not those refuting it.

This post is becoming much longer than I’d intended, so I will skip criticism of Posavac’s article and Rebecca’s misrepresentation of the findings. Instead I’d like to discuss the Botta article as it seems to be at the center of this battle. My interpretation of it is quite different from Rebecca’s.

Some background that I found interesting: Botta is not a psychologist, but a professor of communications. Now, expertise can come from many places. Formal education in an area is not a guarantee that one is an expert (although, given no other information, it is much more likely that someone with a PhD in a given area knows that area well than someone without a PhD in that area). Likewise, one can gain expertise through years informal study and practice. My point, though, is that we should not assume that Botta is an expert on eating disorders.

What is more telling about this particular article is the journal in which it was published: the Journal of Communication. It is not published in the psychological literature where psychologists are likely to see and criticize it. A communications journal is not an inappropriate venue for this article, however, because it is not about eating disorders.

This 11 year old study is about body image.

In fact, Botta used the EDI (Eating Disorder Inventory), a well-established, valid, reliable inventory of disordered eating. However, she chose to use only those sub-scales related to body image disturbance, thus making generalization to disordered eating impossible. She did include a measure of “bulimic behaviors”. It’s really very puzzling. I cannot think of an explanation for this, but one would not be relevant. What is relevant: she did not measure disordered eating with the exception of “bulimic behaviors” and references to other eating disorders as a conclusion to this study would be a GROSS overgeneralization.

There are many, many studies regarding both immediate and long-term effects of media exposure on body image and the findings are not consistent, particularly those examining television viewing. Of course, the quality, methodology, and generalizability of those studies are not consistent, either. As always, null findings are likely to be underreported.

There is no scientific consensus that “ideal thin” media is a direct causal factor in body dissatisfaction. Skepticism of any claim to this effect is certainly warranted.

This area of research is fraught with methodological problems similar to the study of acupuncture and psychotherapy. Body image measures rely on self-report and studies involving media imagery make hypotheses obvious to the participants. What’s more, most studies involve short experiments from which generalizations about real-world effects are extremely difficult.

What the massive body of literature on body image suggests: person variables (characteristics of individuals) such as self-esteem (Jarry & Kossert, 2007), attachment orientation (Greenwood & Pietromonaco, 2004), neuroticism (Daily & Buunk, 2009), and even other components of personality (Roberts & Good, 2010) have direct effects on body image and/or mediate/moderate the relationship between media and body image. In other words, any relationship which exists between media exposure and body image is complex and, at the very least, indirect.

But that is not even what Ben was talking about in his original piece, nor is it what Rebecca started to discuss (given the title of her post). The topic was the question of whether the “popular view” that exposure to images of thin women promotes eating disorders is myth.

Even a surface treatment of that question has me scratching my head a bit given the nearly equally accepted “popular view” that there is an epidemic of obesity in this country.

While body image is highly correlated with eating disorders – it is even one of the diagnostic criteria – there is no evidence that poor body image, or even the desire to be thin, causes eating disorders. Stating that eating disorders are a direct result of wanting to be thin is a lot like saying that obsessive-compulsive disorder is the result of wanting to be organized or clean. Assuming a causal link exists, the direction of cause could easily be the opposite of what people think.

But let’s assume that what we’re really talking about is body image and not eating disorders. Do Botta’s findings really refute Ben’s assertions? Given that he did not discuss the study in detail, I am not sure if Ben knows, but let’s take a look.

Botta’s study is a complex analysis of a number of self-report measures which involves a large number of statistical tests. As such, some significant relationships are likely to occur by chance and it is important to consider the specific hypotheses when interpreting them. I’ve noted where Botta reported significant relationships that I believe should be considered with caution. That said, here is a list of what she found:

Factors in whether participants endorsed a thin ideal

What did NOT predict endorsement of a thin ideal:

• total television exposure
• exposure to “thin dramas” [shows like Melrose Place and Beverly Hills, 90210 – did I mention this study was more than decade old?]
• the extent to which participants reported questioning characters’ bodies when watching television

What DID predict endorsement of a thin ideal:

• the extent to which participants compared themselves to the characters
• the extent to which participants viewed the characters as realistic

Factors in body dissatisfaction

What did NOT predict body dissatisfaction:

• total television exposure
• exposure to “thin dramas”
• the extent to which participants reported questioning characters’ bodies when watching television
• the extent to which participants viewed the characters as realistic

What DID predict body dissatisfaction:

• Body Mass Index
• endorsement of the thin ideal
• the extent to which participants compared themselves to the characters

Botta also reported a significant interaction of endorsement of a thin ideal and total television exposure, but with a p-value of .03. Given the large number of tests produced in this kind of analysis, it pays to be a little more conservative and consider only those less than .01 as significant. Likewise, she reported that ethnicity predicted body dissatisfaction, however, she only reports the p-value as less than .05.

Factors in drive for thinness

What did NOT predict drive for thinness:

• total television exposure
• exposure to “thin dramas”
• the extent to which participants reported questioning characters’ bodies when watching television

What DID predict drive for thinness:

• Body Mass Index
• endorsement of the thin ideal
• the extent to which participants compared themselves to the characters

Again, Botta also reported that viewing the characters as realistic was related to drive for thinness, but with a p-value of less than .05.

Factors in bulimic “action tendencies”

What did NOT predict bulimic tendencies:

• total television exposure

What DID predict bulimic tendencies:

• Body Mass Index
• endorsement of the thin ideal
• the extent to which participants compared themselves to the characters

Again, Botta also reported that exposure to “thin dramas” predicted bulimic tendencies, but not in the direction you might think. The more they reported being exposed to “thin dramas”, the LESS they reported engaging in bulimic behaviors. Regardless, with a p-value of less than .05, I don’t believe it needs to be explained as anything more than an odd finding. In addition, there was an interaction of thin ideal endorsement with the questioning of characters that was difficult to fully interpret or explain.

So, do Botta’s findings support the claim that media images cause eating disorders?

Not. At. All.

Do they suggest that media images cause eating disorders?

Nope.

Do they suggest that media images cause body dissatisfaction?

Again, nope.

Do they suggest that media images teach women that thin is best?

No.

What do her findings suggest?

The amount of media exposure, even specifically to thin characters, is not directly related to whether women think that thin is best, body dissatisfaction, drive for thinness, or bulimic tendencies.

If her findings are similar to those of other researchers (and they are), they actually kinda support Ben’s assertion [we’re still in the “what if we were talking about body image” state].

What Botta’s findings also suggest: Women who compare themselves to characters on TV are more likely to think that “thin is best” than women who do not compare themselves to those characters. Also, thin ideal endorsement and current BMI are related to body dissatisfaction (not a surprise), drive for thinness, and bulimic behaviors.

So, it seems to me that what these findings tell us, beyond “media is not the problem”, is that women who are unhappy with their bodies engage in behaviors which are likely to make them even more unhappy. They have warped views of what is ideal and compare themselves to people who, in their judgment, fit that view. How this equates to “every word of it disagrees with Radford’s assertion that media images have no relationship to body image” I don’t know.

I think it is plain, though, that the issue is complex and so is the literature about it. It is fairly easy for the average human to view this kind of literature as supporting their current view of the world. It is also human to defend that view, even when it is not supported, and to ignore explanations of why they should be skeptical. That’s one of the reasons we need Skeptics (like Ben Radford).

Some References

Botta, R. (1999). Television images and adolescent girls’ body image disturbance Journal of Communication, 49 (2), 22-41 DOI: 10.1111/j.1460-2466.1999.tb02791.x

Dailey, S.E., & Buunk, A.P. (2009). Female body dissatisfaction after exposure to overweight and thin media images: The role of body mass index and neuroticism Personality and Individual Differences, 47 (1), 47-51 : 10.1016/j.paid.2009.01.044

Greenwood, D.N., & Pietromonaco, P.R. (2004). The interplay among attachment orientation, idealized media images of women, and body dissatisfaction: A social psychological analysis in The psychology of entertainment media: Blurring the lines between entertainment and persuasion. Shrum, L. J. (Ed.), 291-308 Other: 2003-88226-016

Jarry JL, & Kossert AL (2007). Self-esteem threat combined with exposure to thin media images leads to body image compensatory self-enhancement. Body image, 4 (1), 39-50 PMID: 18089250

Roberts, A., & Good, E. (2010). Media images and female body dissatisfaction: The moderating effects of the Five-Factor traits Eating Behaviors, 11 (4), 211-216 DOI: 10.1016/j.eatbeh.2010.04.002

Siever, M. (1994). Sexual orientation and gender as factors in socioculturally acquired vulnerability to body dissatisfaction and eating disorders. Journal of Consulting and Clinical Psychology, 62 (2), 252-260 DOI: 10.1037//0022-006X.62.2.252

SPOILER!!

Read this post first, please.

SPOILER STARTS HERE

Here’s the story…

I caught this tweet this morning:
@BadAstronomer: Beam me up some breakfast. Warp factor three egg omelette. http://twitpic.com/27mfxu

I will pretty much click on anything that starts with “beam me up”, so I did and, as you know, I saw this:

Well, I really saw this:

I even heard theme music in my head. I imagined “NCC-1701″ written on the hull in salsa. Because the original series is the cornerstone and James T. Kirk is the greatest captain of all time, space, film, and literature.

But… would I have perceived it as such if:

• the tweet did not include references to Star Trek?
• the tweet was not from Phil Plait, whom I know to be an avid fellow scifi fanatic?
• the tweet was not from Phil Plait, whom I know to be at Comic-con right now for several reasons?
• I was not lamenting not being at Comic-con myself?
• had not just rearranged a shelf on which our Original Series collection sat, taunting me?

Would Phil Plait had perceived the icon of the god that is Captain James T. Kirk if he were not at Comic-con? If he were not a sci-fi fanatic? You get the picture.

The experience of seeing the DVDs or being at (or thinking about) Comic-con is called a “prime” in the psychological literature, because it activates related information, lowering your perceptual threshold for it. The prime prepares you, in a way, to receive related input.

The tweet itself is a suggestion; it is information about to experience. You then use, in part, top-down processes to interpret the image given that information.

Some good examples of how this works using another sense (hearing) can be found in one of my first blog entries about a very naughty Elmo doll.

Priming effects sometimes confound pareidolia (the tendency to perceive familiar patterns in otherwise meaningless input). However, in laboratory tests, such as that conducted by Vokey & Read and some work a student of mine presented at the Western Psychological Association convention this year, little is perceived from very ambiguous sensory information without priming or suggestion. The “Elmo” post includes several illustrations of this.

So, the question remains: Did we see the Enterprise because it really does look like the Enterprise, or we just amazingly geeky?

Either way, I’m going to add it to my collection. If my readers are even HALF as geeky as me…

Vokey JR, & Read JD (1985). Subliminal messages. Between the devil and the media. The American psychologist, 40 (11), 1231-9 PMID: 4083611

Last month I attended the Annual Convention of the Western Psychological Association (WPA), at which two of my students were scheduled to present research. I will spare you the five-page (single-spaced) description of my peril-fraught journey to Cancun and the disappointment of losing the posters along the way and just tell you that I am very proud of how my students handled it.

Totally unrelated lizard image. If you close your eyes, you might be able to imagine a beautiful poster with a couple of people standing in front of it.

Now for the science.

In my experience, most teachers, particularly college instructors, believe that entitlement attitudes, student expectations, study strategies, work habits, and aptitude have changed dramatically in recent years. As students spend more time working in addition to school, they miss more class and devote less time to studying. In addition, because modern technology makes it possible to use (and provide a copy to students) slide show presentations and distribute study guides. As the proportion of courses taught by adjunct faculty, whose teaching load is greater than tenure-track faculty, increases, so does the proportion of exams given in multiple choice format. This, along with outcomes-based learning which shaped students’ habits in elementary and secondary school, promotes rehearsal study strategies.

We hypothesize that the result is a cycle of incompetence as an increasing proportion of college students who believe that memorization of material is an effective way to study and that they are entitled to be given the material and assessments which maximize the benefits of this strategy. In addition, these students do not understand what they memorize and are unprepared for coursework which builds on the material they should have learned. Because they then attribute their failures to outside forces, they do not change their habits and a vicious cycle continues. The literature on academic entitlement is thin, however, some recent findings suggest that academic entitlement attitudes are positively correlated with narcissism, external attribution patterns, feelings of superiority, and exploitative attitudes (Greenberger, et al., 2008; Achacoso, 2002).

To examine these variables, we asked students (N = 95) in upper-division psychology courses to complete a number of measures. Our specific predictions were:

• Entitlement attitudes are positively correlated with external attribution style, narcissism, and feelings of superiority.
• Metacognitive skills are negatively associated with rehearsal learning strategies and positively associated with entitlement beliefs.

As with previous discussions, I will minimize the amount of statistics and technical information I discuss I use to describe the study and its findings, but if you would like more specific information, please feel free to email me.

Our measures:

• The Superiority Scale (Robbins & Patton, 1985).
• Narcissistic Personality Inventory (Raskin & Hall, 1981)
• The Multidimensional-Multiattributional Causality Scale: Achievement Subscale (Lefcourt, Baeyer, Ware, & Cox, 1979). This scale measures the degree to which the participant attributes academic achievement to ability, effort, context (such as the difficulty of the course), and luck. The former two are internal attributes and the latter two are external.
• Learning Strategies Survey we developed to measure the study habits that student think work best. Scores determined relative amounts of passive, rehearsal, and active learning strategies.
1. Passive = attending lectures without taking notes, attending review sessions to study for exams, and using templates or examples to write papers.
2. Rehearsal = using instructor-provided lecture notes, memorizing terms and concepts (e.g., flash cards), using study guides, and studying from sample questions or past exams.
3. Active = taking notes in class, active reading from learning objectives, and drafting & revising papers incorporating feedback.
• An Academic Entitlement Survey we developed which encompassed expectations about the source of grades, what students believe should be expected of them, what they believe instructors should provide, etc.
• A Metacognitive Measure: Participants evaluated the validity of ten syllogisms (all invalid), then estimated the percentage they answered correctly. Performance in this task is fairly difficult for most people to judge.
• In addition, we asked participants to indicate the number of hours studying each week outside of class they believed was reasonable to do well in the course and how many class meetings per semester it was reasonable to miss.

The number of variables and the complex relationships we hypothesized make the findings a bit confusing, but it can be simplified to a series of strong correlations.

Not at all surprising was that the more class meetings students thought it was acceptable to miss, the less time studying they felt should be needed to to do well. What is surprising is that the more missed class meetings they thought were acceptable, the more they felt that academic achievement is determined by luck. Attribution to luck was also positively correlated with rehearsal learning strategies.

The more students attributed achievement to context, the more entitleed they felt.

In addition:

• Entitlement attitudes were positively correlated with narcissism and superiority, a finding which is consistent with most studies on entitlement.
• The greater the entitlement attitude, the more likely students were to use rehearsal learning strategies and the less likely they were to use active strategies.
• The more entitled students felt, the more they attributed academic achievement to external causes (context and luck) and the less they attributed it to effort (attribution to ability was not correlated with any variable).
• Superiority attitudes were positively correlated with attributions to context.
• The most telling finding and the strongest correlations: Overestimation of performance was positively correlated with estimated performance, but negatively with actual performance. In other words, the better students thought that they had done on the argument judgments, the worse they actually performed and more they overestimated their performance.

The worse students performed, the better they thought they'd performed.

These findings are consistent with those of Kruger and Dunning (1999), who found that incompetence is perpetuated by ignorance of incompetence. (Dubbed “The Dunning-Kruger Effect“)

Rehearsal learning strategies were correlated with entitlement and external attributions, suggesting that students who believe that rehearsal strategies work best are more likely to feel entitled to use them and less likely to attribute their failures to those strategies or their own efforts. Instead, they will attribute them to external forces such as luck, instructors, and other situational factors. As a result, they continue to use the same failed strategies.

Student use rehearsal strategies which are highly ineffective, but since they attribute failures to external factors such as context and luck, they do not recognize that they do not understand the material. Thus they are stuck in a cycle of metacognitive ignorance and rehearsal strategies ensuring that they continue with poor strategies and poor outcomes, remaining ignorant of the need for change.

Before we publish these findings, we intend to test the validity and reliability of our original measures and use structural equation modeling to map the complex relationships among the variables. This should be completed in the fall with new participants and I fully expect these findings to be replicated.

References:

Anchacoso, M.V. (2006). “What do you mean my grade is not an A?”: An investigation of academic entitlement, causal attributions, and self-regulation in college students. Dissertation Abstracts International, 67 (6-A) : 2006-99023-155

Greenberger, E., Lessard, J., Chen, C., & Farruggia, S.P. (2008). Self-entitled college students: Contributions of personality, parenting, and motivational factors Journal of Youth and Adolescence, 37, 1193-1204 : 10.1007/s10964-008-9284-9

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments Journal of Personality and Social Psychology, 77 (6), 1121-1134 : 10.1037/0022-3514.77.6.1121

Lefcourt, H., von Baeyer, C., Ware, E., & Cox, D. (1979). The multidimensional-multiattributional causality scale: The development of a goal specific locus of control scale. Canadian Journal of Behavioural Science/Revue canadienne des sciences du comportement, 11 (4), 286-304 DOI: 10.1037/h0081598

Raskin, R., & Hall, C. (1981). The Narcissistic Personality Inventory: Alternative Form Reliability and Further Evidence of Construct Validity Journal of Personality Assessment, 45 (2), 159-162 DOI: 10.1207/s15327752jpa4502_10

Robbins, S.B., & Patton, M.J. (1985). Self-psychology and career development: Construction of the Superiority and Goal Instability Scales Journal of Counseling Psychology, 32 (2), 221-231 : 10.1037/0022-0167.32.2.221