EIFL webinar on Article Level Metrics
Participants from 17 countries attended EIFL's webinar on Article Level Metrics to learn about the new ways in which scholarly communication is measured.

You are here

Martin Fenner talks about open metrics for the timely, comprehensive and reproducible evaluation of the impact of research

37 participants from 17 countries (Belarus, Bosnia and Herzegovina, Botswana, Ethiopia, Germany, Latvia, Lithuania, Macedonia, Namibia, Palestine, Portugal, Serbia, Sudan, Spain, Uganda, the UK and Zimbabwe) attended instantpresenter EIFL's webinar on Article Level Metrics.

Martin Fenner (PLOS (article-level-metrics.plos.org) Article Level Metrics Project) started his presentation with a problem definition: "It has become more important where we publish than what we publish. And the tools we use for impact assessment are slow, limited and proprietary."

He talked about open metrics for the timely, comprehensive and reproducible evaluation of the impact of research. We need to open up (and standardize) the publication metadata, open up reference lists (and make them machine-readable), as well as links to data and other resources. We also need to open up what we measure and what is measured. And we need to open up content to be alm (Article Level Metrics)-able; open up the metrics data and the service and software to collect and analyze the metrics data.

Martin spoke about ALM FOSS he developed: Lagotto that allows a user to aggregate relevant performance data on research articles, including how often an article has been viewed, cited, saved, discussed and recommended; and ALM Reports. Using the Lagotto software, the Public Knowledge Project is providing an ALM service for publishers running Open Journal Systems (OJS).

Opening up data quality checks is also important as well as open up your mind to the limitations of metrics - any metric we use should have good reliability (consistency) and validity. And we need to open up discussion about this and NISO Alternative Assessment Metrics (Altmetrics) Initiative is the project for everybody interested to get involved.

Juan Pablo Alperin talked about PKP’s Article Level Metrics Service he runs for OJS-based journals. The main motivation behind this pilot service was to offer ALMs and altmetrics for developing countries.

Altmertics/ALM:

  • provide new ways to gain and demonstrate visibility;
  • help good articles to stand out from journal and researchers to stand out for good work.
  • help to count some forms of public impact;
  • provide incentive to gear research to public(local?) interests; and
  • move away from dependencies on measures biased against the South.
  • Altmetrics may not tell us much in the developing country context yet, but it's definitely worth trying.

PKP launched ALM using PLOS Article Level Metrics/Lagotto software giving priority to the developing world and to the small publishers. The plan is to make these metrics prominent
on article pages and hopefully to:

  • change culture around communicating research;
  • open minds to alternative measures of impact;
  • attract readers to a different group of journals; and
  • give an incentive to do research of public interest.

PKP runs this as a pilot project and ALM plugin has two parts:

  • Usage Metrics: any journal editor/publisher can show these
  • ALMs: these require a little bit of setup

Requirements: must upgrade to latest OJS (at least 2.4.3) - ALM plugin is already built in! For ALMs, the journals must assign DOIs (this is not required to show download stats).
And PKP has a new partnership with CrossRef and a new plugin that makes this easier in OJS 2.4.5: https://pkp.sfu.ca/crossref/.

If you are interested

  • upgrade to OJS 2.4.5 (2.4.3 OK, but 2.4.5 better);
  • if you want to collect ALMs, set up your DOIs in OJS (preferably to automatic deposit, see details: https://pkp.sfu.ca/crossref/);
  • go to (pkp-alm.lib.sfu.ca) and create an account (try to login, you’ll be redirected to Mozilla Persona Sign-on);
  • enter your API key into the Plugin Settings page;
  • email (juan@alperin.ca) with: the URL to your journal, your DOI prefix.

During the webinar there was good news from Martin - in the forthcoming version of Lagotto DOIs won't be required. And there was also good news from Juan Pablo: as a CrossRef Sponsoring Entity, PKP can also act as sponsor and representative for other journals to join CrossRef through PKP. Journals from low-income countries may be eligible for sponsorship, allowing these journals to become CrossRef members and deposit DOIs into the CrossRef system at no cost. You can find a list of eligible countries on the project page.

The following EIFL partner countries are eligible: Cambodia, Cameroon, Ethiopia, Kenya, Kyrgyz Republic, LAO PDR, Malawi, Mali, Myanmar, Nepal, Senegal, Sudan, Tanzania, Uganda, Zambia and Zimbabwe.

VIEW THE WEBINAR NOW

  • Watch the webinar recording here.
  • Download's Martin's and Juan Pablo's slides here.
Insofar as universities, funding agencies, and research assessment programs need to measure the impact of individual articles, they should use Article-Level Metrics, not journal-level metrics.
Ten years on from the Budapest Open Access Initiative: Setting the default to open