• Searches that fail to use appropriate keywords and controlled vocabulary

• Searches that omit key databases

• Poorly documented searches that are not replicable

• Failure to develop a robust protocol in advance of beginning searches or abstract review

• Failure to engage librarian experts in search development

• Lack of peer review for search strategies

• Lack of dual review and an adjudication plan for title/abstract and full-text review

• Lack of justification for limits used in searching

• Lack of an advance plan for data extraction and review of articles for quality/bias

• Insufficient team member expertise on the topic of the review (such as when assigned to students)

• Unrealistic project timelines

• General failure to follow standards in methods and reporting

• Believing that GenAI can account for a lack of methodological expertise and governance

Systematic reviews, scoping reviews, and other knowledge syntheses have become a popular research approach, but many people misunderstand them. Sometimes well-meaning faculty will advise students or residents to complete a systematic review, or scoping review, as an “easy” research project without fully understanding the methods and resources required. These misunderstandings often lead to the development of poorly done “systematic reviews”, or "scoping reviews" that lack the required rigour defined in an appropriate protocol, lack well-developed and well-documented search strategies, and fail to follow other methodological and reporting standards that peer reviewers should look for when considering the manuscript for publication.

A systematic review or scoping review is not a suitable project for a single student/resident with no prior experience in this methodology. It also is generally not an appropriate project for a single student and a single faculty advisor who is not an expert on these methods. Most graduate students have not had the appropriate experience or training to independently develop robust protocols and search strategies for systematic reviews. A lack of familiarity with appropriate methods for knowledge synthesis projects is very likely to lead to wasted time, unpublishable work, biased, and poor quality outputs. A systematic review, or scoping review, is not an appropriate short-term project; systematic reviews and scoping review can take an average of a year or more to complete.

We recommend that students instructed to do a systematic review or scoping review without appropriate support share the above statement with their faculty. 

Source: This page has been based on, and adapted from our colleagues at the ETSU Medical Library guide: https://www.etsu.edu/medlib/services/systemic-review.php