Know Not Only What You Know, But Why and How You Know It
Many Skeptics, Daniel Loxton and Massimo Pigliucci (So you think you’re a skeptic, don’t you?) come to mind, have discussed the need to restrict one’s public discussions in the name of Skepticism to topics within one’s area of expertise. In the absence of such expertise, we should only convey to the public a scientific consensus, if one exists. So how is a non-scientist or someone working in a different field supposed to know whether a scientific consensus exists and/or what that consensus is?
Well, that is what I had initially intended to write about today. I am afraid this post goes a little off-track, but it still covers important ground.
This post began as a set of corrections to some of the misleading statements in a recent post by Rebecca Watson in which she points out what she believes is wrong with Ben Radford’s use of an article to support his argument that images of thin women in media do not “encourage” eating disorders such as anorexia nervosa. A friend alerted me to this post, knowing that I participated in a discussion on Facebook on the matter.
In her post, Rebecca accuses Ben of cherry-picking and quoting out of context. In a response to Rebecca , Ben defends himself quite well regarding the accusations that he misrepresented the authors (there were two), but I think that there is a lot missing from the discussion that is important and, in fact, this is shaping up to be a great example of why organized skepticism is needed and why more working scientists should get involved.
First, I share Rebecca’s concern that readers will assume that Botta’s findings support Ben’s claim. Ben writes,
Rebecca is assuming that the quotes were selected as representing the conclusions of those particular studies from which they were cited. I made no such claim.
This is true, but Rebecca assumed what most laypersons are likely to assume. It is far too easy to mislead when discussing complex topics such as this one. Evidence to support one’s assertions is difficult, too, when the assertion is that a claim is untrue. I think that Ben’s argument would have been better served by a more detailed discussion of the complexity of the issue and, perhaps, a quote from a recent review of the literature. Those are difficult to come by, but they provide the “big picture” view to which he referred.
Quoting from the introductions of research reports as Ben did is never a good idea, in my opinion, because it is hearsay. However, it is also problematic when people assume that a researcher’s conclusion can support an argument; it can’t. What counts as evidence is a finding.
For example, what prompted me to participate in the Facebook conversation was Rebecca’s comment:
Study after study shows that straight women and gay men develop eating disorders because our society tells them that they must be thin and pretty in order to attract a man (eg: [link to study by Siever]), and that attracting a man should be their #1 goal in life.
The study to which she linked found correlations among disordered eating, body dissatisfaction, the importance placed on physical attractiveness, and sexual orientation by gender. How that was translated into a causal chain involving societal influences and personal goals is quite alarming, but the paper itself provides some clues. The discussion is longer than any other section of the paper, less than parsimonious, and somewhat speculative. This is one of the reasons that researchers cite and discuss findings, not the conclusions other scientists draw.
As I explained to Rebecca,
Correlation does not equal cause.
Causal conclusions are extremely difficult to draw when you cannot randomly assign subjects to conditions and you cannot randomly assign people to be a straight women or gay men. It takes massive amounts of converging evidence from a variety of studies which eliminate rival hypotheses.
So to say that “straight women and gay men develop eating disorders because our society tells them [anything]” is a very bold statement that is not supported by the evidence. It is certainly not supported by the study to which you linked.
But the confirmation bias leads people to follow their current beliefs and demand to be proven wrong. For example, Rebecca ended her post with,
In the Facebook thread, I genuinely wanted to see what evidence actually supported the idea that the link between media and body image is a myth, but I got nothing from Radford or anyone observing the thread. So, I’m forced to continue to side with what appears to be the consensus opinion: the media’s portrayal of the thin ideal most likely negatively impacts the body image of those who process those images poorly.
First, the “who process those images poorly” part is a new assertion and one that reveals her understanding (or lack of understanding) of the findings. What does it mean to “process images poorly” anyway? She initially wrote, “processing them the wrong way”. Is there are “right way” and a “wrong way”? Botta’s variable of “image processing” involves whether the individual questions the weight of characters as realistic, whether they believe they are realistic, and whether they compare themselves to the characters. There’s no right or wrong here.
What’s more important is her comment that she wanted evidence that the claim is untrue. As I explained to Rebecca on Facebook, that’s not how skepticism works. The burden of evidence is on those making the claim, not those refuting it.
This post is becoming much longer than I’d intended, so I will skip criticism of Posavac’s article and Rebecca’s misrepresentation of the findings. Instead I’d like to discuss the Botta article as it seems to be at the center of this battle. My interpretation of it is quite different from Rebecca’s.
Some background that I found interesting: Botta is not a psychologist, but a professor of communications. Now, expertise can come from many places. Formal education in an area is not a guarantee that one is an expert (although, given no other information, it is much more likely that someone with a PhD in a given area knows that area well than someone without a PhD in that area). Likewise, one can gain expertise through years informal study and practice. My point, though, is that we should not assume that Botta is an expert on eating disorders.
What is more telling about this particular article is the journal in which it was published: the Journal of Communication. It is not published in the psychological literature where psychologists are likely to see and criticize it. A communications journal is not an inappropriate venue for this article, however, because it is not about eating disorders.
This 11 year old study is about body image.
In fact, Botta used the EDI (Eating Disorder Inventory), a well-established, valid, reliable inventory of disordered eating. However, she chose to use only those sub-scales related to body image disturbance, thus making generalization to disordered eating impossible. She did include a measure of “bulimic behaviors”. It’s really very puzzling. I cannot think of an explanation for this, but one would not be relevant. What is relevant: she did not measure disordered eating with the exception of “bulimic behaviors” and references to other eating disorders as a conclusion to this study would be a GROSS overgeneralization.
There are many, many studies regarding both immediate and long-term effects of media exposure on body image and the findings are not consistent, particularly those examining television viewing. Of course, the quality, methodology, and generalizability of those studies are not consistent, either. As always, null findings are likely to be underreported.
There is no scientific consensus that “ideal thin” media is a direct causal factor in body dissatisfaction. Skepticism of any claim to this effect is certainly warranted.
This area of research is fraught with methodological problems similar to the study of acupuncture and psychotherapy. Body image measures rely on self-report and studies involving media imagery make hypotheses obvious to the participants. What’s more, most studies involve short experiments from which generalizations about real-world effects are extremely difficult.
What the massive body of literature on body image suggests: person variables (characteristics of individuals) such as self-esteem (Jarry & Kossert, 2007), attachment orientation (Greenwood & Pietromonaco, 2004), neuroticism (Daily & Buunk, 2009), and even other components of personality (Roberts & Good, 2010) have direct effects on body image and/or mediate/moderate the relationship between media and body image. In other words, any relationship which exists between media exposure and body image is complex and, at the very least, indirect.
But that is not even what Ben was talking about in his original piece, nor is it what Rebecca started to discuss (given the title of her post). The topic was the question of whether the “popular view” that exposure to images of thin women promotes eating disorders is myth.
Even a surface treatment of that question has me scratching my head a bit given the nearly equally accepted “popular view” that there is an epidemic of obesity in this country.
While body image is highly correlated with eating disorders – it is even one of the diagnostic criteria – there is no evidence that poor body image, or even the desire to be thin, causes eating disorders. Stating that eating disorders are a direct result of wanting to be thin is a lot like saying that obsessive-compulsive disorder is the result of wanting to be organized or clean. Assuming a causal link exists, the direction of cause could easily be the opposite of what people think.
But let’s assume that what we’re really talking about is body image and not eating disorders. Do Botta’s findings really refute Ben’s assertions? Given that he did not discuss the study in detail, I am not sure if Ben knows, but let’s take a look.
Botta’s study is a complex analysis of a number of self-report measures which involves a large number of statistical tests. As such, some significant relationships are likely to occur by chance and it is important to consider the specific hypotheses when interpreting them. I’ve noted where Botta reported significant relationships that I believe should be considered with caution. That said, here is a list of what she found:
Factors in whether participants endorsed a thin ideal
What did NOT predict endorsement of a thin ideal:
- total television exposure
- exposure to “thin dramas” [shows like Melrose Place and Beverly Hills, 90210 – did I mention this study was more than decade old?]
- the extent to which participants reported questioning characters’ bodies when watching television
What DID predict endorsement of a thin ideal:
- the extent to which participants compared themselves to the characters
- the extent to which participants viewed the characters as realistic
Factors in body dissatisfaction
What did NOT predict body dissatisfaction:
- total television exposure
- exposure to “thin dramas”
- the extent to which participants reported questioning characters’ bodies when watching television
- the extent to which participants viewed the characters as realistic
What DID predict body dissatisfaction:
- Body Mass Index
- endorsement of the thin ideal
- the extent to which participants compared themselves to the characters
Botta also reported a significant interaction of endorsement of a thin ideal and total television exposure, but with a p-value of .03. Given the large number of tests produced in this kind of analysis, it pays to be a little more conservative and consider only those less than .01 as significant. Likewise, she reported that ethnicity predicted body dissatisfaction, however, she only reports the p-value as less than .05.
Factors in drive for thinness
What did NOT predict drive for thinness:
- total television exposure
- exposure to “thin dramas”
- the extent to which participants reported questioning characters’ bodies when watching television
What DID predict drive for thinness:
- Body Mass Index
- endorsement of the thin ideal
- the extent to which participants compared themselves to the characters
Again, Botta also reported that viewing the characters as realistic was related to drive for thinness, but with a p-value of less than .05.
Factors in bulimic “action tendencies”
What did NOT predict bulimic tendencies:
- total television exposure
What DID predict bulimic tendencies:
- Body Mass Index
- endorsement of the thin ideal
- the extent to which participants compared themselves to the characters
Again, Botta also reported that exposure to “thin dramas” predicted bulimic tendencies, but not in the direction you might think. The more they reported being exposed to “thin dramas”, the LESS they reported engaging in bulimic behaviors. Regardless, with a p-value of less than .05, I don’t believe it needs to be explained as anything more than an odd finding. In addition, there was an interaction of thin ideal endorsement with the questioning of characters that was difficult to fully interpret or explain.
So, do Botta’s findings support the claim that media images cause eating disorders?
Not. At. All.
Do they suggest that media images cause eating disorders?
Nope.
Do they suggest that media images cause body dissatisfaction?
Again, nope.
Do they suggest that media images teach women that thin is best?
No.
What do her findings suggest?
The amount of media exposure, even specifically to thin characters, is not directly related to whether women think that thin is best, body dissatisfaction, drive for thinness, or bulimic tendencies.
If her findings are similar to those of other researchers (and they are), they actually kinda support Ben’s assertion [we’re still in the “what if we were talking about body image” state].
What Botta’s findings also suggest: Women who compare themselves to characters on TV are more likely to think that “thin is best” than women who do not compare themselves to those characters. Also, thin ideal endorsement and current BMI are related to body dissatisfaction (not a surprise), drive for thinness, and bulimic behaviors.
So, it seems to me that what these findings tell us, beyond “media is not the problem”, is that women who are unhappy with their bodies engage in behaviors which are likely to make them even more unhappy. They have warped views of what is ideal and compare themselves to people who, in their judgment, fit that view. How this equates to “every word of it disagrees with Radford’s assertion that media images have no relationship to body image” I don’t know.
I think it is plain, though, that the issue is complex and so is the literature about it. It is fairly easy for the average human to view this kind of literature as supporting their current view of the world. It is also human to defend that view, even when it is not supported, and to ignore explanations of why they should be skeptical. That’s one of the reasons we need Skeptics (like Ben Radford).
Some References
Botta, R. (1999). Television images and adolescent girls’ body image disturbance Journal of Communication, 49 (2), 22-41 DOI: 10.1111/j.1460-2466.1999.tb02791.x
Dailey, S.E., & Buunk, A.P. (2009). Female body dissatisfaction after exposure to overweight and thin media images: The role of body mass index and neuroticism Personality and Individual Differences, 47 (1), 47-51 : 10.1016/j.paid.2009.01.044
Greenwood, D.N., & Pietromonaco, P.R. (2004). The interplay among attachment orientation, idealized media images of women, and body dissatisfaction: A social psychological analysis in The psychology of entertainment media: Blurring the lines between entertainment and persuasion. Shrum, L. J. (Ed.), 291-308 Other: 2003-88226-016
Jarry JL, & Kossert AL (2007). Self-esteem threat combined with exposure to thin media images leads to body image compensatory self-enhancement. Body image, 4 (1), 39-50 PMID: 18089250
Roberts, A., & Good, E. (2010). Media images and female body dissatisfaction: The moderating effects of the Five-Factor traits Eating Behaviors, 11 (4), 211-216 DOI: 10.1016/j.eatbeh.2010.04.002
Siever, M. (1994). Sexual orientation and gender as factors in socioculturally acquired vulnerability to body dissatisfaction and eating disorders. Journal of Consulting and Clinical Psychology, 62 (2), 252-260 DOI: 10.1037//0022-006X.62.2.252
[…] This post was mentioned on Twitter by Jack Scanlan, JM Abrassart. JM Abrassart said: Know Not Only What You Know, But Why and How You Know It http://fb.me/O0W9sjQU […]
Thank you, Barb. This is very helpful.
I think this illustrates why I often preface statements with, “I’m an art historian, not a scientist.”
But, Jenna, this is just critical analysis and you certainly have a brain wired for that! It’s nice that I am able to draw some conclusions because I have some specialized knowledge, but I think you’ll agree that without that knowledge the default position should be one of doubt.
Yes, my default position is one of doubt, but I don’t understand enough about p-values and n-values and all of that. I can read a scientific paper and understand what the conclusions are, but I’m not able to deconstruct the methodology and separate the good studies from the bad and determine the validity and significance of the results. I am a strong adherent to the Loxtonian mandate of leaving expertise to the experts. I also look for meta-analyses if possible.
If, however, anyone has questions about art history or visual culture, I’m your gal. Heck, I often straddle the line betwixt empiricism and post-modernism.
Oh, and cupcakes.
[…] I didn’t realize atheists needed credentials, nor is it relevant. Yes, I have criticized her openly (on more than one occasion, actually) in the past for speaking outside her knowledge base. I do not […]