[eIFLoa] A Comparative Review of Research Assessment Regimes in Five Countries and the Role of Libraries in the Research Assessment Process

[Iryna Kuchma]

Key Perspectives Ltd. 2009. A Comparative Review of Research Assessment
Regimes in Five Countries and the Role of Libraries in the Research
Assessment Process. Report commissioned by OCLC Research. Published online
at: http://www.oclc.org/research/publications/library/2009/2009-09.pdf.

This study was designed to review research assessment regimes and the role
of research libraries within those assessment processes in five countries,
each of which takes a different approach to assessment: the Netherlands,
Ireland, the United Kingdom, Denmark and Australia.  At the beginning of the
project it was postulated that libraries occupy an interesting position
within the academy, both belonging to an institution yet to an extent
separated from it. There is—arguably—a set of 'research library values' that
remains independent of local, institutional values, enabling libraries to
occupy a unique and constructive role in the development and support of
research assessment processes. Libraries have an understanding of scholarly
communication processes, and they are currently in a state of rapid
transformation to keep pace with the way scholars work. They understand the
broad range of outputs and the publishing behaviour of scholars across
disciplines, and the methodological constraints, limitations and variances
that pertain to assessment exercises. This report provides an insight into
the extent to which research libraries have so far been able to leverage the
particular skills and experience their staff possess to position the library
at or near the operational and strategic centre of institutions’ responses
to the internal and national requirements of research assessment processes.

Research assessment is a process that involves many actors on the university
campus, and considers a range of data. The scope of this project was
therefore tightly defined and it set out to do the following:
• Investigate the characteristics of research assessment regimes in five
different countries and gather key stakeholders’ views about the advantages
and disadvantages of research assessment
• Discover stakeholders’ perceptions of the effectiveness of research
assessment including its advantages and disadvantages
• Analyse the effect of research assessment procedures on the values of the
academy
• Reveal the characteristics of research library involvement in research
assessment support
• Discover the extent to which research assessment forms part of
institutions’ strategic planning processes and the role libraries play in
planning for the future
• Draw out points of good or best practice for libraries in support of
national or institutional research assessment.

Overview of the results:
Research library involvement in research assessment:
Librarians have the skills and experience that enable them to make valuable
contributions to their institutions, helping to facilitate institutional
responses to the requirements of national and internal research assessment
systems. The extent to which they have contributed varies markedly by
country, reflecting not only the nature of the prevailing research
assessment regime but also the resources available within the library at the
point in time where research administrators were beginning to work on their
research assessment strategies. In Australia, for example, thanks to funding
from the Australian government, most institutions had a tried and tested
institutional repository in place at a time when they were gearing up for
the Research Quality Framework initiative. Thus libraries were well
positioned to strike up collaborative relationships with colleagues in the
research office. The particular nature of the current assessment initiative,
which has involved a qualitative review and ranking of journals, provided
further opportunities for Australian librarians to contribute based on their
knowledge of the publishing system and bibliometrics. Once libraries have
become embedded in an institution’s assessment system, not only does their
operational role appear to grow but librarians begin to play a greater role
in the planning process, all of which reinforces the central position of the
library within an institution.

Whereas the Australian experience demonstrates how institutional
repositories can be leveraged to play a greater role in facilitating the
research assessment process, the situation in other countries is less
positive. In the UK, libraries that possessed a useful institutional
repository at the time institutions were preparing for the latest Research
Assessment Exercise submission found their skills and infrastructure in
demand by the research office, reflecting the Australian experience. Those
libraries that could not offer technological solutions, however, have found
themselves to be more peripheral to the assessment process—in the beginning
at least. When research administrators discovered that they needed people
with experience in information gathering and metadata expertise, often the
library was called in, sometimes quite late in the process. Despite this
slow beginning, libraries in the UK are engaged with the assessment process.
Indeed, the plan to include bibliometrics in the forthcoming Research
Excellence Framework initiative has provided a new opportunity for
librarians to come to the fore, exploiting their long experience in this
field.

If institutional repositories, metadata and bibliometric expertise are key
to playing a central role in institutions’ responses to national research
assessment systems, then a number of libraries in Ireland are well
positioned to play their role if a national system of assessment was to be
instituted in that country. Some libraries are taking the lead in
integrating research information systems with well-founded institutional
repositories and are well advanced in terms of their plans to integrate
bibliometric data with their existing information systems. These information
assets and this infrastructure already play an important part in internal
research assessments and supporting researchers’ grant applications to
competitive funding sources. In the Netherlands, however, libraries’ role in
research assessment is constrained by the nature of the system - which is
run by faculties and which typically gather the evidence they need for
assessment purposes themselves. Many Dutch research libraries do, however,
take responsibility for running the national research information and
publications system, METIS, and this may offer the opportunity to become
more closely involved with the assessment process in due course.

In Denmark libraries have a history of collecting information for research
assessment and librarians were involved in developing the information
technology solution which underpins the new national research assessment
initiative. Because the Danish assessment system has a major bibliometric
component, librarians are building on their experience to offer various
levels of bibliometric analysis to the research community, while at the same
time investment in subject and institutional repositories continues.