A test set is a pre-existing collection of articles that would meet inclusion criteria for your search.​ 

Articles may be ‘randomly’ acquired through a variety of means, including but not limited to: 

• multiple Google Scholar searches 
• suggested references from a supervisor 
• references picked up from reference lists 

Test sets are not to be derived from a structured Medline search.  

Half of your test set becomes a development set​, and half becomes a validation set. 

Your development set is helpful to build your search synonyms. ​ How to use your development set:

1. Scan your development set for additional text words and subject headings to add to your search​ 
2. Utilize Text-Mining Tools such as these to find the most necessary search terms​: 

• PubReminer​ 
• Yale MeSH Analyzer​ 
• MeSH on Demand 

You will use your validation set after you have created your search strategy. Once the search is developed, follow the steps below to determine whether the pre-identified validation test set articles are being retrieved by the search. 

1. Run your search in Medline 
2. Search each article in the validation set on a separate line, followed by the fieldcode “.ti.” Note that if there are 0 results for one of the articles, it could be because the article doesn’t exist in the database. 
3. Combine all of the articles in your test set using the OR operator 
4. Use the OR operator to combine the test set with your final set of results 
5. If the resulting number is higher than your final set of results, use NOT to identify the outlier: (Search line with higher result) NOT (original search results line)  
6. Examine the outlier to determine why the search did not retrieve it, whether it is still relevant, and whether you would like to add additional search terms to your search.  

This work is openly licensed via CC BY-NC-SA 4.0. For information on this guide contact Erica Nekolaichuk, Faculty Liaison & Instruction Librarian at the Gerstein Science Information Centre.