On 19 April 2016, Kendra Perkins wrote an article for the RefME blog with the title The CRAAP Test: An Easy & Fun Way to Evaluate Research Sources. RefME was taken over by Chegg and subsumed into the Cite This For Me service in 2017 and her original post is no longer available. Fortunately, you can still find the original post, preserved by the Wayback Machine (the Internet Archive).
Kendra had long been a fan of RefME and frequently recommended it in librarian listservs and forums such as iSkoodle. Towards the end of her RefME blog, she declared that she had compiled her list of References using the RefME referencing generator.
A year later, May 2017, Kendra posted an article in her own blog, The Inspired Librarian, a piece with the title Cite This for Me Changed my RefME Blog Post. Here she relates how her RefME post had been reposted on the Cite This For Me site. Her article now read:
The note towards the end of the article had also been changed:
making it seem that she had used Cite This For Me to compile her references. She says (in her The Inspired Librarian post)
I do not like the product, Cite This For Me. I wouldn’t suggest it’s use to anyone. Mendeley, Zotero or NoodleTools, in my opinion are far superior products. They produce much better quality citations with far less errors.
Cite This For Me edited my original blog post for RefME without my permission, and without my knowledge. I have been unsuccessful in contacting them so I am writing here to clarify.
In no way would Kendra endorse Cite This For Me and neither would I. What’s more, it is still as flawed as it was when I wrote A gift that kept on giving back in December 2016.
Just a month after that post I wrote By any other brand-name, not so sweet? inspired by an email from RefME which said it was being taken over by Chegg, allegedly to be amalgamated with Cite This For Me; the takeover would take effect on 28 February 2017.
Cite This For Me might not have replied directly to Kendra, but they did remove her post, it is no longer available on the Cite This For Me site. Fortunately, I Scrapbooked the Cite This For Me page when I first read her article – these last two images are taken from my own Scrapbook archive, which showed it as posted at http://www.citethisforme.com/blog/2016/04/19/the-craap-test-an-easy-fun-way-to-evaluate-research-sources
Double double-dipping – doubled and redoubled
I remembered Kendra’s experience when I was preparing for a recent academic honesty workshop. I wanted to use some of the statistics in a RefME blog/report on student attitudes to plagiarism. I blogged on it at the time, Self-serving survey? posted on 13 July 2016. The report was a masterpiece of misunderstanding and misdirection. This extract from my article illustrates at least six problems with the RefME report on the survey:
• the discussion of the surveys reads at times like an inadequate discussion of the surveys and at times like a press release produced by the RefME publicity bureau;
• the report manages to confuse and conflate incorrect or inconsistently formatted references with plagiarism and/or academic misconduct;
• the discussion grabs at different research and studies, and suggests (inter alia) that small-scale surveys can be regarded as universal truths;
• in grabbing at those different research reports and studies, the writer misreports some and fails to do the homework, to check on the source behind the source;
• the report, despite praising RefME for enabling correct and consistent referencing/ endnoting, manages to be incorrect, incomplete and/or inconsistent in at least 11 of its 13 references.
• a small matter of several, many, passages which reuse so much wording from source documents that it might be felt that quotation marks are required; some readers might even class these passages as plagiarism.
I explored each of these issues more deeply in my post.
The RefME article is no longer available (of course), and it took a little detective work to find it on the Cite This For Me site. A few differences were immediately apparent. Where the RefME report had carried the title Survey Reveals Unique Insights to US Students’ Attitudes Towards Plagiarism (the link leads to the Internet Archive’s capture of the article), on the CTFM site it has a new name, 10 Things We Discovered About Students’ Attitudes Towards Plagiarism. After Kendra’s experience, I was not surprised to find that the post had a new author – the original is recorded as by RefME, the CTFM post is “by” Cite This For Me.
At first glance, the main bulk of the report appeared to be the same, right down to the graphics. The change of title seemed to be the only difference.
But then I came across other differences and oddities: the percentages recorded in the section headed Survey reveals the most common referencing mistakes were slightly different to those I had quoted in my blog post. A glance at the reference list showed that while many of the footnotes were the same in both versions, several were different. A more careful reading revealed even more differences. I saw that the number of students taking part in the survey was different between the two reports. I was even more puzzled as I came across a sequence of passages which referenced problems in Australian universities. There had been no mention of Australia in the earlier report; that was about US students, obvious in the original title “Survey Reveals Unique Insights to US Students’ Attitudes Towards Plagiarism.” US students!
Searches on the Internet Archive revealed more discrepancies; the following links lead to the various archived versions.
The RefME article is dated 20 June 2016. The Internet Archive’s Wayback Machine captured it twice on 8 July 2016, a few weeks after its original publication:
The Wayback Machine captured the “original” Cite This For Me version five times;
all five versions are the RefME article, under new authorship; the last of these saves is dated 26 June 2017.
Apart from the domain name, the “original” Cite This For Me version had the same URL:
The revised Cite This For Me article has the different title, a different date and a different date in the URL:
Despite the date claimed, the Wayback Machine’s first capture of the latest incarnation of the article is dated 25 March 2017.
Does all this matter, what’s the point?
The point is that Cite This For Me appears to have committed piracy, has perhaps plagiarised itself and in consequence leaves us unsure as to the veracity of the data and the (second) report.
Whole chunks of the articles are identical and near identical as in these extracts (similarities highlighted):
The first half of Cite This For Me’s later version of the post is almost identical to the RefME report. There are just the few variations similar to those highlighted above. The differences are mainly of three kinds:
• the different numbers and percentages between the original and the later article,
• the different platforms, RefME replaced by Cite This For Me
• the different uses of “reference” and “citation” (between USA usage and usage in other countries).
I have remarked on the different usages of “references” and “citations” in another blog post, Language and labels; there I pointed out that RefME’s US site used the term “Citation Generator” while the UK site preferred “Referencing Generator.”
But how many surveys were there? Two – or four?
Does the CTFM report with its 4,979 respondents include RefME’s original 2,111 US students or are the samples completely different? Why is there no mention of the earlier report in the later article? Isn’t that what researchers do, refer to earlier studies and show how their own study is different, builds upon earlier work and affirms it or is different?
The main differences between the two articles are in the second half, for suddenly in the Cite This For Me version we are looking at reports of cheating in Australia, no mention of findings among US students.
It is worth noting that the first six footnotes and three later footnotes are identical in both versions (including the errors noted in Self-serving survey?). The CTFM article includes four new references in the second half, three of them referring to reports published in or about the situation in Australia. As with RefME’s set of footnotes, these new references include inconsistencies of style and incomplete information. Two of these include enough wordage copy-pasted from the original sources to raise thoughts of plagiarism (since there are no quotation marks to indicate a quotation) and one piece of summarised material manages to misrepresent the original and include misinterpretation and misinformation.
Here is an extract from Timna Jacks, Deakin University students kicked out for ‘contract cheating’, The Age 18 May 2016:
The IDP post used later in the article is no longer available but, thanks once more to the Internet Archive, we can read the original International Students in Australia, Blog, IDP. Thinking of cheating? Think again:
As well as the direct copy-paste here, it may strike that the IDP piece, written for an Australian audience, uses $1000 without qualification knowing that its readership will realise that they mean AU$1000. It seems likely that readers of the Cite This For Me article are more likely to think this is US$1000.
And then, from RMIT University, Academic integrity:
Once again, the Cite This For Me paraphrase is too close to the original for comfort. And, while “cheating in an exam” is a form of academic misconduct, it is not a form of plagiarism as suggested in the Cite This For Me article.
In my original article on the RefME report, I noted that the footnote references might have been more complete and consistent if the writer had used the RefME software to generate the references. As it was there were major inconsistencies, including failure to identify the authors of some of the sources used, and some entries using the date of access but not all. The same issues are noted in the four new references in the Cite This For Me article (entries # 7, 8, 9 and 13 if you want to follow these up for yourself).
In the earlier article, I pointed out that the RefME blogger’s report did not necessarily reflect the RefME reference/ citation generator itself. This does not hold true of Cite This For Me. The rehashed report really does reflect the problems with the software.
Moreover, are the two writers, the RefME blogger and the Cite This For Me blogger, one and the same person? If different people, then the CTFM writer has appropriated someone else’s work and presented it as his (or her) own. If there is just one person responsible for both versions of the article, then this approaches self-plagiarism, what the IB terms double-dipping, attempting to gain repeat credit for the same work.
The questions keep coming. Is the second CTFM report solely about student attitudes in Australia or is this now a worldwide study? For both articles, how were the students surveyed chosen? How representative a sample is it? Given the second report’s notes about difficulties faced by international students, how many international students are included, are there significant findings coming from their responses?
Given the lack of transparency, we are entitled to ask, how do we know that either survey report is genuine, how do we know that the figures have not simply been pulled out of the air?
It is all very disturbing. It all suggests we can have little confidence in Cite This For Me, even less confidence in the blog writer’s objectivity, his (or her) scholarship, his credibility, his lack of respect for his readership, his very integrity.
Kendra, you made a wise choice, disowning those revisions made to your article.