Q: Teamwork/Work Division
A: This is something you’ll need to discuss and decide as a group – teamwork is a crucial skill in the workforce, including among research teams, so this is a great opportunity to get practice!
Q: Citation Chasing
A: Yes! This is a fantastic way to find additional research on your topic. For key articles on your topic, look at a) its reference list (review articles are great for this!), and b) who’s cited that particular paper in more recent publications. This is what’s known as “forward chaining” and “backward chaining” your references.
Q: Where to Search
A: Unfortunately no! Despite appearing to be an "all search" option, this tool does not capture everything and will often bring back loooots of irrelevant results, which is why we recommend searching within databases directly. Furthermore, many databases offer more thorough search abilities and filtering options, which allows you to create a more focused search. That's not to say you can't use it, just be aware of its limitations, especially for in-depth search methods such as rapid reviews.
Q: Number of References to Use/How Much Data to Include
A: This will depend on your topic and how much or how little published literature there is on your topic. There is no f you’re getting an unmanageable number of results from your searches, you might consider:
It’s certainly legitimate to exclude results that aren’t helpful based upon your evaluation and appraisal criteria, but be cautious about excluding results that don’t line up with your personal opinions and preferences – objectivity is key!
Q: How many articles is typically considered sufficient for a rapid review vs. a systematic review?
A: There is no magic number of studies distinguishing the two types of reviews; rather, the difference is in the goals and the process for how the review is conducted (e.g. search strategy, inclusion/exclusion criteria, and critical appraisal). Please refer to slide 5 on the EES3001 PowerPoint to compare the different types of knowledge synthesis and key differences between them.
Q: Rapid Review Documentation
A: Be as thorough as possible! Recall that key elements of rapid reviews are that they are transparent and replicable. Slides 23 and 28 from the EES3001 PowerPoint posted at the start of this guide cover some key elements to include during documentation, as well as further guidelines from the Collaboration for Environmental Evidence (CEE).
Q: Combining Search Materials
A: You wouldn’t need to link individual articles to specific databases, no – it’s sufficient to report the number of results obtained in total from all of the databases searched as part of your documentation. These can be reported using flow diagrams such as PRISMA or ROSES. CEE guidelines also provide some helpful details for the level of depth required when documenting and reporting your searches (see section 5.3 of Conducting a Search). To combine results from different databases, you can export your search results into citation managers (e.g. Zotero) and then de-duplicate articles within the software.
Q: Duplicate Results
A: Potentially, depending on your research topic and the resources searched. So for example, the Web of Science and Scopus databases have a lot of content overlap, but also unique results that would be found in one and not the other. The same applies to search engines like Google and Duck Duck Go. Frustrating for the researcher, but important for avoiding resource bias! Hopefully Zotero's deduplication feature helps with this :)
Q: Do we keep track of all grey literature searches? Especially the ones which are not credible but has embedded links to peer reviewed journals or other credible sources?
A: Documenting the resource and search strategy you used to get to those embedded results should be sufficient. So for example, say I did a Google search and found some quality references linked on a Wikipedia page. I would document the search engine I used (Google), the search string I used, the number of hits the Google search turned up, etc., and make note that citation chasing was used to find embedded articles from the resulting list.
The EES3001 PowerPoint has some more details on what to include when documenting your search.
Q: Should priority be given to articles with higher citation counts?
A: Citations can be great indicators of papers that are prominent in a field. Keep in mind that citation count is not necessarily an indication of quality and that time required to accumulate citations is also a factor. Ultimately your selections will be determined based on your inclusion/exclusion criteria and/or your critical appraisal criteria as determined in your protocol.
Q: Evaluating Results
A: The type and degree of screening criteria will vary depending on your topic and research protocol; for examples, we recommend looking over some sample protocols and reviewing the resources outlined in the EES3001 PowerPoint.
Q: Evidence Synthesis
A: Yes! Rapid Reviews may involve a mixed-methods approach.
Q: Remaining Unbiased/Relaying Implications of Research
A: This is tricky! As noted in class, you can certainly relay the implications for existing research, but must aim to do so in an unbiased way – recall, the goal of a systematic review is to inform, not to advise. You are providing the best available evidence (and their implications) so that policy makers can make an informed decision.
Q: Reporting/Paper Formatting/Examples of Systematic Reviews
A: Meander through some completed reviews on the Environmental Evidence page to get an idea of formatting and general content; look for examples that have “systematic review” in the title.
Q: Citation Style
A: It’s up to you, just be consistent!
Q: “Shelf-Life” of a Systematic Review
A: Great question! Once researchers conduct a systematic review, it’s not uncommon for that to become their “research baby” which they then tend to and care for as the literature continues to grow and evolve. So you may see researchers do an update on a previously-conducted SR (for example, 5 or 10 years down the line) to see what new literature has emerged on the topic – this is why documentation and transparency is so important, as the researcher must be able to recreate the search exactly as they did the first time!
University of Toronto Scarborough Library
1265 Military Trail, Toronto, ON M1C 1A4 Canada
About web accessibility. Tell us about a web accessibility problem.
About online privacy and data collection.
© University of Toronto. All rights reserved.