Skip to Main Content

Research Guides

Comprehensive Searching in the Social Sciences

A test set (or validation set) is a short list of articles (10-15) not derived through a structured search that meet your planned inclusion criteria. Meaning, a short list of articles like the ones you hope to find during your comprehensive search.

The list should be assembled through a non-structured process that can include (but is not limited to) strategies like:

  • Multiple Google Scholar searches
  • Articles recommended by a supervisor
  • Articles picked up from reference lists or citation lists

The reason it’s important to build your test set through a non-structured process is that it creates a test set that’s free of the methodological approaches you need to test.

TIP: Generate this list before you begin to build your database search and set it aside until you are ready to test.

Testing your search

First retrieve your test set. Next, follow the steps below to determine if these pre-identified articles are being retrieved by your search:

  1. Run the search you are testing in your database
  2. Search each article in the test set using the field “Title.” If there are 0 results for one of the articles, it could be because the article isn’t indexed in the database.
  3. Combine all articles in your test set using the OR operator
  4. Use the OR operator to combine the test set with the search you are testing
  5. If the resulting number is higher than your final set of results, use NOT to identify the outlier: (Search line with higher result) NOT (original search results line)
  6. Examine the outlier to determine why the search did not retrieve it, whether it is still relevant, and whether you would like to add additional search terms to your search.