Bibliometrics and research evaluation

Scientometrics studies the measurement and analysis of science, specifically research. Evaluating research calls for a criteria capable of measuring both the quantity and the quality of the research itself. Research evaluation is based on both a qualitative and quantitative approach using various methods such as bibliometric analysis, peer review, cost-benefit analysis, and the degree of internationalization.

• On a quantity level, scientific impact, patentability, and the presence of contracts with companies interested in research are measured in numerical terms. This measurement are based on citation analysis and usage analysis (bibliometrics). 

• On a quality level, the evaluation is based on peer review with clear and shared criteria for organized evaluation: 

  • peer-review ex ante: the peer judgment that happens before the publication of the article (which determines its publication)

  • peer-review ex post: th peer judgment that happenns after the publication of the work, typical of web 2.0 communities (social peer review)

Key to scientometrics is the evaluation and measurement of scientific publications: the part of scientometrics called bibliometrics. Scientometric analysis varies depending on the disciplinary field highlighting  the differences between hard sciences and humanities, where bibliometric techniques are difficult to apply.

Bibliometrics is applied to various scientific fields and uses mathematical and statistical techniques to analyze the scientific publications distribution patterns and assess their impact within scientific communities. Citation analysis and content analysis are common bibliometric methods.

Citation analysis examines the frequency of citations in scientific articles or texts. It uses bibliometric indices or indicators as analytical tools. 

Quantitative bibliometric analysis takes in account the following elements:

• number of publications by an author

• number of citations received by each publication (citation index)

• number of citations received by journals (Impact Factor)

• productivity and impact of publications by an individual author (H-index).

With the advent of web 2.0, we can find alternative metrics with indicators based on readership (usage data and citations in social media). 

At the national level, the evaluation of the research system is managed by Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca (ANVUR), which establishes criteria, depending on the different scientific fields, and issues the Abilitazione Scientifica Nazionale(National Scientific Qualification).

The research evaluation activity is based on a combination of quantitative indicators (including bibliometric indicators) and qualitative aspects (peer review analysis). The results are crucial for the distribution of a part of Fondo di Finanziamento Ordinario (FFO) to universities and research institutions.

Therefore, it is important to upload publications to the institutional repository IrisORA-Open Research Archive of Cineca. Publications in institutional archives, along with public engagement activities, are subject to analysis and evaluation by ANVUR.