Smoke and mirrors

Print Friendly, PDF & Email

“Technological solutionism” – a term coined by Evgeny Morozov – offers us solutions to problems we often do not know we have. Some might feel that it sometimes creates new problems, too often without solving the problems it is designed to solve. So often and too often, it fails to do what it says on the tin.

On the other hand, technological solutionism can make big money for the companies behind the so-called solutions. It can blind us to other, often more workable, often more less expensive and more low-tech strategies, approaches and solutions.  Worse still, it can divert attention from the real problems, including situations which might cause the problems in the first place.

I have blogged before about technological solutions which promise far more than they deliver. Turnitin and EasyBib are the ones which come most readily to mind. You can name your own “favourites.”

And now, Microsoft has just released enhancements to Office 365. The announcement is made in an Office Blog article posted on 26 July 2016 with the snappy-catchy title New to Office 365 in July—new intelligent services Researcher and Editor in Word, Outlook Focused Inbox for desktop and Zoom in PowerPoint. The piece is written by Kirk Koenigsbauer. He is a corporate vice president for the Office team, heavy-hitting stuff indeed.  In this post, we’ll be looking just at Researcher and Editor.

In the blog, we read that

Researcher is a new service in Word that helps you find and incorporate reliable sources and content for your paper in fewer steps. Right within your Word document you can explore material related to your topic and add it—and its properly-formatted citation—in one click. Researcher uses the Bing Knowledge Graph to pull in the appropriate content from the web and provide structured, safe and credible information.

and that

Editor assists you with the finishing touches by providing an advanced proofing and editing service. Leveraging machine learning and natural language processing—mixed with input from our own team of linguists—Editor makes suggestions to help you improve your writing.

Powerful tools indeed.  If they work.

Given the first look that Microsoft gives us, they have a long way to go.

First, Researcher. The section heading in the blog reads

Introducing Researcher—accessing credible sources and cited content just got easier

That’s two promises in one: the promise of easy finding of “credible” sources and the promise of generating bibliographic citations for easy insertion a bibliography or list of works cited.  It is still up to the writer to write the work – which Editor will make it “easier” to check, proof-read and refine, and we shall come to this later.

The blog includes expandable screengrabs to illustrate the research and writing processes, and there is a video-clip embedded in the blog, available also as a YouTube stand-alone.

They are not encouraging.

“Researcher helps you get started,” we are told.

You can “Explore reliable sources without leaving Word,” because Researcher brings up Bing hits in a side-bar.

This of course assumes that Bing is better at knowing what you want, what you are looking for, what is relevant and reliable and credible and authoritative, better than you do.  You do not have to accept Bing’s suggestions, of course – but if you are using Researcher, perhaps you value speed and ease over slow intelligent thinking research? Hey! Let Bing do your thinking for you!

The video shows an illustrative example, the building of an essay on The Amazon Rainforest.

We are shown how (thanks to Bing)

we might find an article with the heading and publication details Climate Change 2014 – Impacts, Adaptation, and Vulnerability: Regional Aspects, Cambridge University Press, Dec. 29, 2014

We can then “pull in facts or quotes”

“and citations are added for” the piece we have just quoted.

Just don’t look too closely, or you will see that the citation/reference that is added reads:

Amazon Rainforest, Amazon Plants, Amazon River Animals. 2016. World Wildlife Fund 9 May 2016. <http://www.worldwildlifefund.org/biomes>

There is a disconnect there. The quotation is taken from Climate Change 2014 – Impacts, Adaptation, and Vulnerability: Regional Aspects but is attributed to a page on the World Wildlife Fund (WWF) website.

Funnily enough, the quotation can be found on the WWF website, at http://www.worldwildlife.org/biomes/tropical-and-subtropical-moist-broadleaf-forests – but this is not where our researcher – or Bing – found it, it is not where Researcher says it found it.  Nor is it the title that the WWF gives it, and it isn’t the URL either. The date of publication is different. What’s more, the WWF appears to have last used the URL http://www.worldwildlifefund.org/ in 2010 or early 2011. Thereafter, use of this URL redirects to http://www.worldwildlife.org/.  Thank you, Internet Archive.

But what is going on? A dubious piece of montage? Advertising magic? How can a search in May 2016 cite a site which ceased to exist 5 years ago?

 

 

 

The second reference generated by Researcher is – perhaps – more accurate,

 

 

 

We see only a small part of the reference in the video-clipbut we do see it in full in the Office blog (below):

It reads:

Field Museum Scientists Estimate 16,000 Tree Species in the Amazon. 17 October 2013. 9 May 2016. <http://www.eurekalert.org/pub_releases/2013-10/fm-fms101413.php>

which is just about right.

Of course, many referencing styles might not use italics for the title of the article (they might use quotation marks instead), and many referencing styles would use the corporate author, Field Museum, rather than the title, both in the reference and in the in-text citation.

 

 

The in-text citation? That’s in the video too. we have moved beyond our draft notes and are now looking at the finished product:

 

and let’s not quibble too much about the punctuation here?

The forest covers an area approximately as large as the 48 contiguous United States. (Field Museum Scientists Estimate 16,000 Tree Species in the Amazon) Contained within it …

and later, in this same paragraph and screengrab,

… the Amazon is also considered to be a distinct ecoregion. (Amazon Rainforest, Amazon Plants, Amazon …

Sorry. If we are talking academic writing, then perhaps it is not quibbling to suggest that the parenthetical citation belongs inside the sentence it is quoting or paraphrasing, not at the start of the next sentence. The periods/full-stops are in the wrong place. Something which Editor has missed?

And, hold on.  What documentation style is this writer using? The captures we are shown earlier in the video use a numbered footnoting system; the paper we are shown as illustration of the Editor’s capabilities are MLA – or similar – author-style in-text citation.  No date is given (as in APA or Chicago author-date). No page numbers are given. Page numbers are not possible with material taken from the web, but they do exist in printed matter. We see only a segment of the paraphrase from the Margulis source (this again is taken from the blog post, not from the video itself)

but it is clear. The parenthetical citation does not include a page number – yet the line and a bit that we can see is (probably) taken from page 47 of a 107 page document

which, for completeness, should perhaps carry the reference

Margulis, Sergio. Causes of Deforestation of the Brazilian Amazon. World Bank Working Paper; No. 22. Washington, DC: World Bank. 2004. 9 May 2016. https://openknowledge.worldbank.org/bitstream/handle/10986/15060/277150PAPER0wbwp0no1022.pdf?sequence=1

Hmmm. This is not the URL given in the Bibliography that we looked at earlier (and repeated below). The URL given in the Bibliography, http://www.treyresearch.net/Y07418PV/277150PAPER0wbwp0no1022.pdf takes us to Microsoft’s home page. Treyresearch indeed. Tres something, Jim, but it’s not research?

Not impressed

The change of referencing style between draft and final paper does not impress. The incompleteness of the referencing does not impress. The research going into this paper does not impress. The writing of this paper does not impress.

The video flits from feature to feature, page to page, making it difficult to see just what is going on. More likely, Microsoft does not want us to see what is going on?  The screengrabs on the Office blog page are more helpful; they do not move (and a click on them gives them full-size. Here is that last image yet again

Note the first paragraph we see here; it begins:

Some of what is lost is just burned down, to make paper, for example. A single paper manufacturing plant starts with burning down about 5600 square miles of forest. Another 2000 tons of rainforest wood a day are used to make electricity for each plant. The forests are also cleared for cattle ranches and the highways to service the ranches and the paper plants.

Now these are fairly exact figures, not exactly common knowledge. Shouldn’t there be a citation here? I rather think this section is based on the work of Leslie Taylor of the Raintree Group. We can find the “5,600 square miles of forest” and the “2000 tons of rainforest wood a day” statistics together in at least two places online, in a paper Saving the Rainforest: A Complex Problem and a Simple Solution and also in Rainforest facts:

The way Taylor writes it, it is just one particular paper manufacturing plant, set up in 1978, which burned down 5,600 square miles of rainforest (and replaced it with pulpwood trees), which every day uses 2,000 tons of rainforest wood to generate electricity.

The way the Microsoft researcher writes it, it seems that every paper manufacturing plant “starts with burning down about 5600 square miles of forest,” and they all need “another 2000 tons of rainforest wood a day are used to make electricity for each plant.”  Which – if this page is indeed the writer’s source – demonstrates the Microsoft writer’s misunderstanding and misreporting of the original source. And, whether it is Taylor who is the source or it is some other writer, where is the credit in the text for the source of the information?  (I am not counting the bibliographic reference, because this could be on the next page of the sample essay, and not included in the video.)

Is Researcher really fit for purpose? Are those responsible for the promotional content aware of what is involved in academic writing?  Are the software developers aware of what is involved in academic writing?

If they are aware, then the video and the blog present an illusion, smoke and mirrors. Don’t look too closely.

If they aren’t aware, then what are they doing? Can they be trusted? Can you trust this software with your research?

Either way, Researcher has a long way to go.

A brief word on Editor

Editor, “your new digital writing assistant,” provides spelling and grammar checking, and suggestions for alternative expression of words and ideas. The blog reports that Editor is set to get “better,” refinement is on the way:

This fall, it will expand upon Word’s current spelling and grammar tools to inform you why words or phrases may not be accurate—teaching at the same time it is correcting.

Ethical or not? Once upon a time, spell-checkers were frowned upon in academia, but are now widely accepted – not least because they are unreliable and users still have to think and choose, decide whether to accept the suggestions made. (Of course, one can auto-accept all changes suggested, but this is to risk looking foolish. As anyone who uses predictive text without double-checking is well aware.)

When it is technology making the decisions, fallible technology which is readily accessible to most users, the playing field is level. Just don’t stop thinking.

As for Researcher, “available (from late July) for Office 365 subscribers using Word 2016 on Windows desktops,” use at your own risk. And if use it you must, that same advice goes, don’t stop thinking.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.