Email updates

Keep up to date with the latest news and content from BMC Medical Informatics and Decision Making and BioMed Central.

Open Access Highly Accessed Research article

Is the coverage of google scholar enough to be used alone for systematic reviews

Jean-François Gehanno*, Laetitia Rollin and Stefan Darmoni

BMC Medical Informatics and Decision Making 2013, 13:7  doi:10.1186/1472-6947-13-7

PubMed Commons is an experimental system of commenting on PubMed abstracts, introduced in October 2013. Comments are displayed on the abstract page, but during the initial closed pilot, only registered users can read or post comments. Any researcher who is listed as an author of an article indexed by PubMed is entitled to participate in the pilot. If you would like to participate and need an invitation, please email, giving the PubMed ID of an article on which you are an author. For more information, see the PubMed Commons FAQ.

Of course GS will find what you already know exists

Tuulevi Ovaska   (2013-02-13 13:24)  University of Eastern Finland

Dear colleagues,

your article is very interesting but also rather confusing.

It would naturally be very efficient and truly economic if only one simple and freely available search engine could be used to search and find all studies needed for a systematic review.

You write that "GS was searched for all these studies one by one to assess the percentage of studies which could have been identified by searching only GS. ---All the 738 original studies included in the gold standard database were retrieved in GS (100%)."

Well, of course, you were able to retrieve them all as you already knew what they were and searched them one by one.

But would you have found them if you did not know them already? Would searching GS for the topic (instead of the studies already known) end in the result of the same studies? These are the important questions, imho.

Competing interests

None declared


Interesting but...short shrift to expert searching

Michelle Fiander   (2013-02-13 13:21)  A personal comment

That Google Scholar is a one stop source for searching because a selection of studies included in a selection of systematic reviews can be found in GS displays, I believe, faulty logic--perhaps an association fallacy. Whether or not I am correct regarding the logic, I am bit more concerned that the authors give very short shrift to the importance of the search interface (technology aspect) and knowledge of indexing (understanding gained by experience and training) which are both integral to searching for systematic reviews. While GS and other Web applications are absolutely fantastic (no argument from me on this point), neither the search interface, nor the scope and depth of GS is yet, in my opinion, sufficiently well defined or organized to support comprehensive searching for systematic reviews.

Competing interests

I am an Information Scientist, Trials Search Co-ordinator, for the Cochrane Collaboration Effective Practice and Organisation of Care Review Group and a member of the Cochrane Collaboration Steering Group.


Google Scholar as a single source for systematic reviews has not been justified

Alison Weightman   (2013-02-13 13:16)  Cardiff University email

This paper forms part of an important debate. Google Scholar (GS) is an increasingly powerful search tool that should rightly be considered by systematic review searchers. Research studies comparing GS to other established search tools are valuable.

However we feel the paper fails to address some really important issues, and believe that the conclusion that GS could be a single search source for systematic reviews has not been justified.

Being able to find a paper once you know about it is not the same as finding the paper in the first instance. Problems identified with this publication, on which we would welcome the authors' and journal editors' responses, are:

1. Google Scholar is constantly updated and it isn't possible to know whether the references included in it at the search date for this paper would have been there at the search date of the candidate systematic reviews. GS trawls many sites and may have picked up these references as a result of their inclusion in the reviews.

2. The lack of advanced search functions is acknowledged but there seems to be an implicit assumption in the paper that review authors would have chosen the right terms to pick up relevant publications, including citations without an abstract. Further, the very low precision rates measured by the authors of this paper from example searches (circa 0.1% precision) severely limit the current value of GS as a single search source for systematic reviews. It would ask a lot of systematic reviewers to trawl/sift 36,000 results in order to find 36 relevant papers.

3. The authors did not require the title to link to an abstract or full text in Google Scholar (GS), to be regarded as a study within GS; a link to a citation was considered sufficient. Each title identified via a citation would need to be searched for in other database(s) to find an abstract, or obtained in full text to assess relevance. This would entail a substantial additional workload for systematic review authors which is not addressed in the paper.

4. The paper has several sections that should have been edited for grammatically correct English. Errors are not surprising given that the authors are not writing in their native language, but it may call the editorial process into question.

Alison Weightman, Fiona Morgan, Mala Mann and Bernadette Coles
University Library Service, Cardiff University, UK

Competing interests

A competing interest exists when your professional judgment about a paper could possibly be influenced by considerations other than the paper's validity or importance. Detail possible competing interests here... No competing interests


Post a comment