Skip to main content

Understanding Peer-Review

Concerns over Journal-Based Metrics

Research assessment is traditionally based on the citation count of journals.  There have been discussions of the appropriateness of such an approach over the years. 

In May 2013, a group of journal publishers and editors released the San Francisco Declaration on Research Assessment to express their concerns over journal-based metrics and put forth these recommendations:

  • the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;
  • the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and
  • the need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).

Thousands of individuals in different academic communities have signed the Declaration. 

Emergence of Alternative Metrics

Nowadays scholars communicate and share resources through various online avenues such as blogs, social networks, and digital repositories.  Their online activities offer a novel way to assess research impact.  In recent years, scholars have started aggregating the online activities to create alternative metrics (altmetrics) for research assessment.  As the altmetrics manifesto states: 

The speed of altmetrics presents the opportunity to create real-time recommendation and collaborative filtering systems: instead of subscribing to dozens of tables-of-contents, a researcher could get a feed of this week’s most significant work in her field. This becomes especially powerful when combined with quick “alt-publications” like blogs or preprint servers, shrinking the communication cycle from years to weeks or days. Faster, broader impact metrics could also play a role in funding and promotion decisions.

The video below features presentations about altmetrics and its uses.  The speakers are Jason Priem from ImpactStory, Kristi Holmes from Washington University in St. Louis, and Caitlin Aptowicz Trasande from Digital Science.  

 

Uses of Altmetrics and Article-Level Metrics

The Public Library of Science has explained why ALM is valuable to different stakeholders in the scholarly communication ecosystem: 

The SPARC primer points out that ALM can complement traditional metrics because it is more granular and more immediate.  It recommends that ALM be included in the faculty tenure and promotion process in light of the diverse ways scholars communicate.  Meanwhile, Heather Piwowar and Jason Priem assert that there are 10 benefits to scholars and scholarship when altmetrics are embedded in a CV.