I felt compelled to write about eyewitness memory by a recent blog entry which I feel paints a misleading picture of the nature of memory and the (un)reliability of eyewitness memory.

Other skeptics have written about the subject and coverage has ranged from the pretty good to not so great. This blog post is somewhere in between. I have a great deal of respect for Vandy Beth Glenn and her knowledge on a wide range of topics, but I feel the need to address some issues in this piece that I think are important.

My first problem is that the inaccuracy of memory is overstated.

We don’t remember very well events that really happened to us.

Vandy Beth asks:

[S]hould eyewitness testimony be declared inadmissible evidence in court?… Eyewitness testimony has … brought many infamous individuals to account for their crimes. However, we shouldn’t doubt that at least as many innocents and patsies have been wrongly made to pay as well… We shouldn’t accept a faulty system just because it works more than 50 percent of the time.

How does one get the impression that eyewitness memory is so bad and so hopelessly unreliable that a reasonable solution is to toss it out all together? The literature on the subject of eyewitness memory (let alone memory generally) is vast and complex. Watching a presentation on false memory by Loftus and visiting the Innocence Project website, does not give one the requisite knowledge to make informed policy recommendations. Though I’m sure this is not the extent of Glenn’s knowledge on the subject, these are the only sources cited.

Daniel Loxton has argued that when skeptics write about things outside of their domain of expertise, they must exercise due diligence.

Skeptics solicit … trust. We make the implicit (and sometimes explicit) promise that we are able to provide the nuanced, objective, evidence-based facts. That combination of stated commitment to science, limited qualifications, and weighty ethical responsibilities … place a very high due diligence burden upon skeptics.

When Ed Clint, co-founder of the Skeptic Ink Blog network, and blogger at Incredulous, posted this article, I voiced concern regarding the exaggerated negative portrayal of memory, however both he and Vandy Beth seemed to believe this view was justified.

Ed, essentially began a lengthy explanation about eyewitness memory errors, not quite addressing my criticism and then side-stepping it altogether, saying that it doesn’t matter how many have been wrongfully convicted …

We can’t know if eyewitness testimony is the leading cause of wrongful convictions because most such convictions will never be found out, and the pattern of which get found out could be biased in one or other direction. The evidence suggests that it is, and that’s as far as we can go.

But is that really important here? If anything beats this cause, it is prosecutor misconduct, and reform is needed there, too. But prosecutor misconduct is, at least, a known problem that juries and judges are mindful of (or should be). Are they equally mindful of the problem of unreliability of eyewitnesses? Or do they assume it is correct, if the witness is firm and sure on the stand? I’d guess the latter is the case, and in the most dire need of being addressed. This is true whether eyewitness misidentification is the #1 cause or #10 cause of wrongful convictions. I, frankly, don’t give a fuck what number it is. Steve Titus should not be dead.

Vandy Beth appealed to Elizabeth Loftus’s authority and reputation.

I admit I didn’t do my own research…I relied on my sources, like Elizabeth Loftus and the Innocence Project, to be themselves reliable. If you think they’re wrong, and can back it up, I’d be glad to learn about it.

I am confident that both Ed and Vandy Beth will recognize that I mean no disrespect in the criticism that follows. I consider Ed a friend and, having just met Vandy Beth, hope to one day call her a friend as well.

Yes, Steve Titus (and the many other exonerated individuals whose stories are detailed on the Innocence Project webpage) should not have been convicted. It very rightfully violates our sense of justice that these wrongful convictions have occurred and we feel outrage. This may motivate us to want to offer solutions to the problem that would prevent miscarriages of justice like this from occurring again and give us the sense that we have a good grasp of the nature of memory. But, it is not that simple.

We cannot know the truth in each criminal case, thus preventing us from being able to estimate the true incidence of wrongful convictions as a result of eyewitness memory. However, there is no data of which I am aware that suggests that the legal system gets it wrong in half of all cases as suggested by the text quoted above. A database of 317 exonerations since 1989 tells us that errors occur and that they are not exactly uncommon. But perspective is important, lest we erroneously conclude that the justice system is as broken as we mistakenly perceive our memories to be… According to published statistics from the Department of Justice, in 2010 alone, cases were filed against 91,047 defendants. Ninety-three percent, 81,934, of defendants were convicted. And of those convictions, 97 percent, or 79,260, pled guilty.

Agreement from more than one witness is not much assurance of accuracy, either. Approximately 36% of the first 250 Innocence Project exoneration cases involved the testimony of more than one witness (Garrett, 2011). If all witnesses are subjected to the same biased lineup procedure, it’s not inconceivable that they may all choose the innocent suspect. And proposing to eliminate eyewitness testimony altogether is an absurd “toss the baby out with the bathwater” sort of solution.

It is tempting to assume that we might get rid of problematic eyewitness memory and instead rely on physical evidence, such as DNA evidence, because we perceive it to be more reliable. One problem with this approach is that in 90 to 95% of cases DNA testing is not an available option (Innocence Project, 2014). Furthermore, physical evidence is not immune to error or bias; it can be contaminated or it can be interpreted incorrectly (Murphy & Thompson, 2010; Thompson, 2006). Adherence to established standards and procedures for the collection, storage, testing, and interpretation of that evidence helps minimize the incidence of errors. Eyewitness memory can be thought of as analogous to trace evidence. It is evidence that law enforcement must collect from the mind of the witness while exercising the utmost care not to contaminate it.

Memory researchers would have long ago been done with the context of the legal system when it was first established that eyewitness memory can be inaccurate and unreliable. But the literature does not support such a negative view of memory. If our memories were so poor we would have serious problems functioning in day to day to life and it would not be possible for the layperson to live blissfully unaware of the potential frailty of memory. Elizabeth Loftus’s work on false memory shows us that false memories are possible (e.g. a quarter of participants reported a false event in the famous “Lost in a Mall” study, Loftus & Pickrell, 1995) but not that false memories are so pervasive that we should never trust our memories. “We make no claims about the percentage of people who might be able to be misled in this way, only that these cases provide existence proof for the phenomenon of false memory formation” (Loftus, Coan, & Pickrell, 1996; p.207)

Long before Loftus even began her work on the misinformation effect and later false memories, Hugo Munsterberg (1908) had noted the potential for inaccuracy in eyewitness memory. We could have written off eyewitnesses as hopeless then and moved on to other problems. Instead, Munsterberg and eyewitness researchers since have spent decades trying to identify what factors influence accuracy in an effort to offer more practical solutions to minimize inaccuracies in eyewitness testimony and identifications while also contributing to a better understanding of the nature of memory as a whole.

While confidence, detail, and vividness do not predict accuracy, eyewitness researchers have been exploring the influence of countless other variables as well. We have identified some factors related to accuracy which we cannot control (termed estimator variables, see Wells, 1978) but are nonetheless helpful to be aware of because they signal which instances of eyewitness memory are more error-prone, such as, poor lighting, short exposure to culprit, stress, witness intoxication, the presence of a weapon, mismatch between witness and culprit race, etc. Other factors which can influence accuracy and are under the control of the legal system (termed system variables) have also been explored: whether witnesses are shown books of mug shots, making composite sketches, the specific lineup instructions given, the composition of the lineup such as the number of persons included and how the fillers were selected, the format in which the lineup is presented (e.g. simultaneously or sequentially), blind administration, etc. All of these variables are at play and this is what forms the complexity that requires greater familiarity with the primary research.

Regarding eyewitness identification research specifically Wells & Loftus (2003) say “The primary lesson of the eyewitness identification work is that mistaken identification rates can be very high under certain conditions and many of these conditions could actually be avoided by the use of scientific procedures for lineups”, p.150. This research has helped to inform important guidelines and recommendations for law enforcement agencies; a committee of experts assembled by The Justice Department published a set of guidelines for law enforcement (Technical Working Group for Eyewitness Evidence, 1999). Increasingly more jurisdictions (state and local) are adopting reforms aimed at improving the accuracy and reliability of eyewitness identifications and testimony; for example, approximately 32% of law enforcement agencies have switched to a sequential lineup procedure (Police Executive Research Forum, 2013).

Memory is certainly malleable and it is reconstructive, but this does not mean that eyewitness testimony can never be trusted or be valuable in court. It can … IF certain conditions are met.

REFERENCES

Garrett, B. (2011). Convicting the Innocent. Harvard University Press.

The Innocence Project (2014). Unreliable or Improper Forensic Science, retrieved from: http://www.innocenceproject.org/understand/Unreliable-Limited-Science.php

Loftus, E.F., Coan, J.A. & Pickrell, J.E. (1996) Manufacturing false memories using bits of reality. In L. M. Reder (Ed.) Implicit memory and metacognition. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 195-220.

Loftus, E., & Pickrell, J. (1995). The Formation of False Memories Psychiatric Annals, 25 (12), 720-725 DOI: 10.3928/0048-5713-19951201-07

Munsterberg, H. (1908). On the Witness Stand.

Murphy, E., & Thompson, W. C. (2010). Understanding potential errors and fallacies in forensic DNA statistics: An amicus brief in McDaniel v. Brown Criminal Law Bulletin, 46 (4), 709-757.

Police Executive Research Forum (PERF), & United States of America. (2013). National Survey of Eyewitness Identification Procedures in Law Enforcement Agencies.

Technical Working Group for Eyewitness Evidence. (1999). Eyewitness Evidence: A guide for Law Enforcement. Washington, DC: National
Institute of Justice.

Thompson, W.C. (2006). Tarnish on the “gold standard”: Understanding recent problems in forensic DNA testing The Champion, 30 (1), 10-16

Wells, G. (1978). Applied eyewitness-testimony research: System variables and estimator variables. Journal of Personality and Social Psychology, 36 (12), 1546-1557 DOI: 10.1037//0022-3514.36.12.1546

Wells, G. L. & Loftus, E.F. (2003). Eyewitness memory for people and events. A. M. Goldstein (Ed.) Handbook of Psychology. Vol 11 Forensic Psychology (I.B. Weiner, Editor-in-Chief). New York: John Wiley & Sons, pp 149-160

Workshop: Skepticism in the Classroom

Matt engages the audience. Photo by Dean Baird (minor retouching and cropping by me)

Matt recapped the most important concepts from his piece last year and presented more of his fun and interesting demonstrations. I used to think that cognitive psychologists had all of the fun because we study the interesting ways that our brains and minds fool us and can blow those minds by showing them. However, after some thought I realized that the physics teachers I know have the coolest, scariest, ickiest, and most surprising demonstrations. They deal with the physical world and there are almost as many bizarre things in the physical world as there are in the mind.

No, I do not remember what I was saying when I made this face. Photo by Dean Baird (minor retouching and cropping by me)

I was a bit nervous about this workshop because some of the material I presented is very different from my usual “Oew” and “Ah” and “aHA!” stuff. In addition, its connection to promoting skepticism is distant, at least on the surface. The title of my presentation was Deep Thoughts: Facilitating Critical Thinking at All Ages. In teaching critical thinking, the age of the student is extremely important in determining methods and focus. For adults, the biggest roadblock to critical thinking is overconfidence. This is just a nice way of saying “arrogance” or “closed-mindedness”. The irony is that we humans are so overconfident that we think the term applies to other people and not ourselves.

For young children, there are few roadblocks. What we should focus on is guiding cognitive development in a way that minimizes overconfidence. In my opinion, the best way to do this is to encourage the practice of consideration and deep thinking. This, I suggest, is accomplished through discussion of philosophical questions.

I have yet to read a review of the workshop. However, the immediate feedback I received was very positive and I heard my words flowing from the mouths of others all weekend, including on the stage. It is entirely possible that others have been thinking about the same issues, but I choose to take it as evidence that my ideas were discussed and found worthy of some consideration.

A Short TAM9 Review

Unfortunately, I was still tweaking my workshop presentation and was unable to attend the other workshops. I caught only some of the activism workshop – the one I needed the most – but luckily there is a wonderful manual available which was produced by Desiree Schell and Maria Walters. Last year’s reception, which kicks off the official meeting on Thursday night, featured music and live interviews. This seemed to defeat the ‘meet and greet’ purpose of most attendees. The reception this year returned to the usual format of conversation, but there were so many people that it was difficult to find anyone. Friday morning JREF president D.J. Grothe announced the final headcount. Attendees, organizers, and presenters at TAM9 From Outer Space totaled 1652, approximately 300 more people than last year, which was 200 more than the year before.

In general, the long list of speakers booked for this year included the most inspiring scientists and science communicators in the skeptical community. The original keynote speaker, Astrophysicist Neil deGrasse Tyson, is a personal hero of mine. His talk repeated much of what he covered in his very long and satisfying TAM6 talk, but I never tire of the material or his presentation style. Unfortunately, the other keynote speech, delivered by Richard Dawkins, was as boring (to me, anyway) as Tyson’s was entertaining. I have never found Dawkins to be a dynamic speaker, but this was particularly snore-worthy. He chose to spend much of his time describing his soon-to-be-released children’s book rather than discussing anything of note. Likewise, I find PZ Myers’ style a little bit dull, but I usually enjoy his talks simply because he chooses to talk about some of the most interesting topics. This year is no exception. His was one of the few talks that I missed, but I am looking forward to his discussion of alien anatomy when the JREF posts video of his talk (they committed to making all of the content available online).

Every other talk (not including the Sunday Paper Session, which varied in quality) was fantastic.

Some of the highlights for me:

• Carol Tavris delivered a speech about reducing cognitive dissonance by first considering the target’s vantage point (i.e., empathy). This was probably the best speech I have ever heard, and I have heard a LOT of speeches and talks.


Dylan Keenberg, Sabrina Gibson, and me Photo by Daniel Loxton (minor touch-ups and adjustments by me)

• Dylan Keenberg, a former student and collaborator of mine, delivered a wonderful Sunday talk describing one method for talking to others (Rogerian argumentation) which is highly likely to reduce both cognitive dissonance and misunderstandings. The most important aspect of this method is, once again, empathy. In order to more than simply fake empathy, though, one must be open to the possibility that one’s current understanding is wrong. My informal polling of TAM9 speakers and other community leaders tells me that I am justified in feeling extremely proud.
  
  

  Dylan Keenberg and Daniel Loxton Photo by Sabrina Gibson (minor touch-ups and cropping by me)

  Daniel Loxton’s discussion of these two talks (Tavris’s and Keenberg’s) as well as the two which specifically addressed activism (one by JREF Communications Sadie Crabtree and the other by union organizer and radio host Desiree Schell) is much more thoughtful, thorough, and interesting than what I could write at the moment.

• Daniel also wrote about the panel to discuss diversity. In a nutshell, the discussion was quite a mess for the first half, but the more they discussed the more each clarified, and in some cases, changed their views until they settled on a middle ground that I think all could embrace. Essentially, they agreed that applying skepticism to a more diverse set of problems/questions/domains would result in a more diverse community without compromising the integrity of skepticism as a movement. Political, moral, and social ideology are ‘outside the scope’ of skepticism because they remove objectivity. In addition, untestable claims (e.g., “Does God exist?”) are off-limits because they cannot be addressed scientifically.

  I am always thrilled to hear D.J. speak about such things from a stage because he tends to be clear, firm, and directly on-message. Last year, for example, he made a point of asking nearly every speaker to clearly define the scope of their organization and each answered with some form of “scientific skepticism”. This year, he elaborated on this by noting that he strives for a diversity of religious views.

  However, I did not leave TAM9 with the optimism that Daniel Loxton left with. One reason for this was that D.J. made those statements while discussing “Diversity in Skepticism” with Debbie Goddard, Greta Christina, Jamila Bey, and Hemant Mehta. Debbie Goddard is the campus outreach director for CFI, a secular organization with a branch devoted to skepticism (CSI). The panel’s moderator, Desiree Schell, is firmly rooted in the skeptical community as the host of Skeptically Speaking and an occasional blogger on Skeptic North. The other three panelists are closely identified with atheism and, in my opinion, have contributed little, if anything, to skepticism itself. I kept wondering who this “we” was in the discussion (e.g., “We could offer…”).

  The conflation of atheism and skepticism is a very serious problem with dire consequences. The most important of these is the degradation of the integrity of skepticism itself. The scientific method only works when scientists are open to interpreting any result objectively – to consider all evidence with an open mind and to hold all conclusions tentatively. The conclusion that there is no God cannot be arrived at empirically, so it cannot be “the result of properly-applied skepticism” as some claim. I am very worried about this trend to conflate these two for several reasons, including the manner in which the majority of atheists talk to and about the faithful.

• Bill Nye’s talk was condensed from the longer talk he gave at the Skeptic Society’s Science Symposium last month. In his position as the executive director of The Planetary Society he is concerned with science education and the consequences of failing in this area. For this reason, he is another hero to me.
• The panel discussion of the future of space exploration was almost as lively as the diversity panel would be two days later. Most notably, Neil deGrasse Tyson’s verbal sparring with Lawrence Krauss left Bill Nye and moderator Phil Plait with little room to get a word in. However, Pamela Gay managed to do so by literally shushing Tyson – three times! For that, if not for the plea during her solo talk for all in the audience to be activists for education, made her another hero. Phil’s talk last year still rings in my years, so the odd man out on that panel – Lawrence Krauss – was the only one on the stage that I would not walk a few miles, breaking a path in the snow, to hear speak.
• Speaking of heroes, there were two announcements at TAM9 which deserve to be noted. One was that The Richard Dawkins Foundation has committed to fund child care at meetings and conferences like TAM. The other involves everyone’s hero, Genie Scott. At the end of a talk in which she described the parallels between evolution denial and AGW denial (described and discussed in a great post by Donald Prothero), she announced that the NCSE is beginning an initiative to fight climate change denial in public education.
• Finally, two Jennifers, Jennifer Michael Hecht and Jennifer Ouellette, merged poetry and popular culture with skepticism and science, respectively, in the most uplifting and inspiring ways. Hecht condensed a normally hour-long history of doubt into half an hour by speaking quickly, but this only enhanced the talk. Jennifer held a cultural mirror up to science and space exploration, showing clips and images from A Trip to the Moon to Doctor Who and beyond. These were as, if not more, intellectually fulfilling as the talks given by the psychologists (okay, I’m biased) and neurologists (Elizabeth Loftus, Richard Wiseman, Susana Martinez-Conde, and Stephen Macknik all spoke). Wiseman even introduced me to a new favorite ‘suggested lyrics’ video, so I think that I will leave you with that.