(Spoiler alert: I succumbed!)
You surely remember the coffee study? I posted it only last week, Memories are made of this…
Okay, the study was actually on the effects of caffeine on the memory; Michael Yassa and associates were looking at how a dose of caffeine taken after a learning experience affected memory (even if the volunteer participants were not aware that they would be tested the following day on what they had remembered seeing during the “learning” experience).
My post was not about Yassa’s study itself; it was about the number of differences in press reports of the study: reports disagreed as to the number of volunteers, whether it was caffeine or coffee that the experimental group were given, and whether the study suggested improvement to long-term or to short-term memory.
Well, this week I realised that these discrepancies would be ideal as part of a workshop or as a classroom exercise as an illustration of some of the reasons WHY we cite our sources; they particularly emphasise the enabling a reader to check on or to follow-up sources, while allowing the writer to share the blame if s/he gets the facts wrong.
I went to the various online reports, taking screengrabs of headlines and sections of the text, straight-forward enough. I noted in particular that the number of volunteer participants varied tremendously:
44 according to the Guardian
60 according to the LA Times
73 according to the New York Times and the New York Daily News, and
160 according to Scientific American, New Scientist and Time magazine.
How could there be such discrepancy? From which source/s were the writers of these articles getting their information? Had they looked at the source, the original paper as published in Nature Neuroscience?
Memory told me that the number of participants was not given in the original study, not the section which was openly and freely available on or through the Nature Neuroscience website.
But I know my memory has lapses, so to the website I went, to check.
To find that this webpage has changed, slightly: a correction is posted, published the day after I posted last week’s observations on this study. “Corrected online 17 January 2014” it now says, and the correction is printed below, in full:
- Corrected online 17 January 2014
- In the version of this article initially published, in the first sentence in the Online Methods, the s.d. of the age of the subjects was missing and the number of female subjects was given as 280. The s.d. is 2 years and the number of female subjects is 80. The error has been corrected in the HTML and PDF versions of the article.
Curiouser and curiouser! It seems that the original paper included a misprint! The original paper suggested there were “280 female participants”. Were all participants female, or were there even more participants, male? Whether 280 or more, from where did the science correspondents get their figures, all well less than 280? Not from the paper as openly published and available for free on the Nature Neuroscience Brief Communication / Abstract or in the ReadCube extract, there are no mentions of numbers there. Nor is there mention in what appears to be the official Press Release from Johns Hopkins University, It’s all coming back to me now: Researchers find caffeine enhances memory.
Perhaps even more to the point, what did the research team actually say?
And that, Dear Reader, is the point at which I succumbed. I bought the paper! Not the full reprint, I have to admit; I bought the ReadCube version, which allows me to read online but not to download or print. But that’s enough.
It’s a pity, perhaps, that I did not succumb earlier, while the original was posted, before the correction was made. But never mind, we get to the original as corrected. The original mistake must have been very obvious – though not “very obvious” enough to the original proof-reader. On page 4 of the paper, in a section headed Online Methods, we read:
Subjects: 160 healthy, caffeine-naive subjects between the ages of 18 and 30 (mean age, 20; s.d. 2; 80 female subjects) participated in the study…
We are no nearer explaining discrepancies in those press reports, even with the full paper in hand. But we do have a valuable lesson in citation, and a reminder of research basics: go to the source.
We also have – at least, I have, now – the advantage of the original paper. The original Advance Online Publication of the paper. I wonder if there will be more discrepancies, when official publication is made? I must remember to watch out for it.
Pingback: By any other name… | Honesty, honestly…
From what I can tell, 160 participants volunteered for the study, but following selection – i.e. being caffeine naive, having a caffeine intake less than 500 mg per week, meeting certain criteria on a Mini Mental State Examination, and following the experimental protocol correctly – only 44 remained.