Searching for studies is different than screening for studies. When screening potential articles to be included in a knowledge synthesis, keep these pointers in mind:

• Think specifically, and at a granular level

• Decisions are based on the specific characteristics of each study, as defined by the scope of your topic, and that will answer your question, eligibility criteria should be consistent with these characteristics

• The process on how to handle uncertainty when screening should also be pre-determined, e.g. if the study is missing data, then include for now

• Reviewers should have a high minimum of 90% agreement among them in their screening process

*Our librarians do not provide consultations on screening, however we're happy to provide you with the information and resources below.

First level of screening

In the first stage of screening, a pair of reviewers from the review team will independently scan titles and abstracts of articles that were retrieved from a comprehensive (i.e. multiple source) search, and make decisions whether to include or exclude articles. To do this in a streamlined, unbiased, and method-driven way, reviewers should adhere to the pre-defined eligibility criteria, or guidance form.

Keep these tips in mind during the first stage of screening:

• Authors should generally be over-inclusive at this stage; a maybe is always a yes at this point

• Screen for outcomes with caution if at all; they're poorly reported in the title and abstract. Save those for full-text!

• Ask yourself: Is there enough information in the title and abstract to exclude this study at this stage, with absolute certainty?

Second level of screening

The second level of screening is a more rigourous, in-depth process in which the articles that were included in the first stage of screening are read in full-text. Similar to the first-level of screening, this is done independently by two reviewers from the review team, and the eligibility criteria that was used as a guideline for the first-level of screening is largely the same. However the second level of screening differs in these important ways:

• The reason(s) for exclusion must be recorded and reported

• You can now screen for outcome(s). Ask yourself: does the study report on the outcome(s) you're interested in?

• Although the eligibility criteria is the same, it will require additional detail (clarifying questions may arise during the first stage of screening)

(Adapted from source: Moher et al., 2009)

Screening does not come without challenges. The process can often be confusing, and coming to a consensus on whether an article "fits" the eligibility criteria can be hard. These are some common challenges faced during screening:

Internal challenges:

• Lack of methodological expertise

• Belief that AI or technology will solve problems, or fix a lack of methodological knowledge

• Poorly defined eligibility criteria

• Lack of training among the team

External challenges:

• Volume of the evidence

• Poor reporting of research

• Lack of clear how-to/practical guidance

• Subjective nature of methods

Guidance and training are two keys methods in mitigating challenges that arise during the screening process. 

Establishing a guidance form lessens reviewer confusion and anxiety during screening by providing a document that all team members can refer to. 

First level of screening - Steps to create a guidance form

1. Determine all eligibility criteria for your study based on your question & scope

2. Define each criteria extensively!

3. Write out all variations (nomenclature, spelling, meaning, definitions) for all criteria with examples

4. Play the "so what does that mean?" game to a fault, until the entire team is clear on all criteria and definitions, be annoying!

5. Once you feel confident on step 4, fill out the guidance form templates and adapt as necessary

First level of screening - Pilot testing

1. One person in the team should take the lead, usually the PI

2. Select a set of articles retrieved by your search, you want a good variety of articles with some that are difficult/vague, and some that are clear

3. Share articles among your team

4. Using the screening guidance forms, ask your team to determine individually if each record should be included or excluded with reasons

5. Collect the individual answers, and collate them blinding the name of each reviewer

6. Meet to discuss the results of the pilot and provide the correct answers with reasons, this is the chance for the team to air out any confusion and solve it!

7. Further clarify criteria based on the discussion

8. Apply guidance consistently throughout the screening process

Second level of screening - Steps to update a guidance form

1. Base guidance forms on the ones created for the first level of screening

2. Enhance forms by adding (never subtracting!) details that are hard to screen for, or determine at the first level

3. Determine explicit steps to be taken should studies warrant further clarification, and who will be responsible for it

4. Build a flowchart indicating in what order the reasons for exclusion will be determined, and reported

Second level of screening - Pilot testing

1. One person in the team should take the lead, usually the PI

2. Select a set of articles retrieved by your search, you want a good variety of articles with some that are difficult/vague, and some that are clear

3. Share articles among your team

4. Using the screening guidance forms, ask your team to determine individually if each record should be included or excluded with reasons

5. Collect the individual answers, and collate them blinding the name of each reviewer

6. Meet to discuss the results of the pilot and provide the correct answers with reasons, this is the chance for the team to air out any confusion and solve it!

7. Further clarify criteria based on the discussion

8. Apply guidance consistently throughout the screening

For a more comprehensive explanation on the process of screening, watch the webinar "Screening for studies in systematic reviews, scoping reviews, and other knowledge syntheses: Strategies for improvement" created and presented by Patricia Ayala, Research Services Librarian at Gerstein Science Information Centre. To view the slides, examples, and screening templates.

This work is openly licensed via CC BY-NC-SA 4.0. For information on this guide contact Patricia Ayala, Research Services Librarian at Gerstein Science Information Centre