Limits...
Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals.

Postma E - PLoS ONE (2007)

Bottom Line: I show that there are large differences in impact among evolutionary and non-evolutionary papers within journals; while the impact of evolutionary papers published in multidisciplinary journals is substantially overestimated by their overall impact factor, the impact of evolutionary papers in many of the more specialized, non-evolutionary journals is significantly underestimated.Importantly, however, their ranking remains largely unaffected.While journal impact factors may thus indeed provide a meaningful qualitative measure of impact, a fair quantitative comparison requires a more sophisticated journal classification system, together with multiple field-specific impact statistics per journal.

View Article: PubMed Central - PubMed

Affiliation: School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, Australia. e.postma@unsw.edu.au

ABSTRACT
Amongst the numerous problems associated with the use of impact factors as a measure of quality are the systematic differences in impact factors that exist among scientific fields. While in theory this can be circumvented by limiting comparisons to journals within the same field, for a diverse and multidisciplinary field like evolutionary biology, in which the majority of papers are published in journals that publish both evolutionary and non-evolutionary papers, this is impossible. However, a journal's overall impact factor may well be a poor predictor for the impact of its evolutionary papers. The extremely high impact factors of some multidisciplinary journals, for example, are by many believed to be driven mostly by publications from other fields. Despite plenty of speculation, however, we know as yet very little about the true impact of evolutionary papers in journals not specifically classified as evolutionary. Here I present, for a wide range of journals, an analysis of the number of evolutionary papers they publish and their average impact. I show that there are large differences in impact among evolutionary and non-evolutionary papers within journals; while the impact of evolutionary papers published in multidisciplinary journals is substantially overestimated by their overall impact factor, the impact of evolutionary papers in many of the more specialized, non-evolutionary journals is significantly underestimated. This suggests that, for evolutionary biologists, publishing in high-impact multidisciplinary journals should not receive as much weight as it does now, while evolutionary papers in more narrowly defined journals are currently undervalued. Importantly, however, their ranking remains largely unaffected. While journal impact factors may thus indeed provide a meaningful qualitative measure of impact, a fair quantitative comparison requires a more sophisticated journal classification system, together with multiple field-specific impact statistics per journal.

Show MeSH
The impact of evolutionary papers in evolutionary and non-evolutionary journals.A) The mean absolute impact of evolutionary papers for 39 journals from a range of different categories. The dotted line gives their overall impact. B) The impact of evolutionary articles relative to the impact of the average article for each of these journals. Relative impacts marked with an asterisk are significantly different from zero at the 5% level.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC1994586&req=5

pone-0000999-g001: The impact of evolutionary papers in evolutionary and non-evolutionary journals.A) The mean absolute impact of evolutionary papers for 39 journals from a range of different categories. The dotted line gives their overall impact. B) The impact of evolutionary articles relative to the impact of the average article for each of these journals. Relative impacts marked with an asterisk are significantly different from zero at the 5% level.

Mentions: Not surprisingly, there are highly significant differences among journals in the impact of the evolutionary papers they publish, as well as in their overall impact (Figure 1A). More interestingly, however, the size and/or direction of the difference between these two measures of impact varies among journals, as is indicated by the significant interaction between interaction between Evolutionary vs. Overall impact and Journal in Table 1. In other words, the overall impact of a journal does not necessarily provide a good predictor of the impact of the evolutionary papers it publishes.


Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals.

Postma E - PLoS ONE (2007)

The impact of evolutionary papers in evolutionary and non-evolutionary journals.A) The mean absolute impact of evolutionary papers for 39 journals from a range of different categories. The dotted line gives their overall impact. B) The impact of evolutionary articles relative to the impact of the average article for each of these journals. Relative impacts marked with an asterisk are significantly different from zero at the 5% level.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC1994586&req=5

pone-0000999-g001: The impact of evolutionary papers in evolutionary and non-evolutionary journals.A) The mean absolute impact of evolutionary papers for 39 journals from a range of different categories. The dotted line gives their overall impact. B) The impact of evolutionary articles relative to the impact of the average article for each of these journals. Relative impacts marked with an asterisk are significantly different from zero at the 5% level.
Mentions: Not surprisingly, there are highly significant differences among journals in the impact of the evolutionary papers they publish, as well as in their overall impact (Figure 1A). More interestingly, however, the size and/or direction of the difference between these two measures of impact varies among journals, as is indicated by the significant interaction between interaction between Evolutionary vs. Overall impact and Journal in Table 1. In other words, the overall impact of a journal does not necessarily provide a good predictor of the impact of the evolutionary papers it publishes.

Bottom Line: I show that there are large differences in impact among evolutionary and non-evolutionary papers within journals; while the impact of evolutionary papers published in multidisciplinary journals is substantially overestimated by their overall impact factor, the impact of evolutionary papers in many of the more specialized, non-evolutionary journals is significantly underestimated.Importantly, however, their ranking remains largely unaffected.While journal impact factors may thus indeed provide a meaningful qualitative measure of impact, a fair quantitative comparison requires a more sophisticated journal classification system, together with multiple field-specific impact statistics per journal.

View Article: PubMed Central - PubMed

Affiliation: School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, Australia. e.postma@unsw.edu.au

ABSTRACT
Amongst the numerous problems associated with the use of impact factors as a measure of quality are the systematic differences in impact factors that exist among scientific fields. While in theory this can be circumvented by limiting comparisons to journals within the same field, for a diverse and multidisciplinary field like evolutionary biology, in which the majority of papers are published in journals that publish both evolutionary and non-evolutionary papers, this is impossible. However, a journal's overall impact factor may well be a poor predictor for the impact of its evolutionary papers. The extremely high impact factors of some multidisciplinary journals, for example, are by many believed to be driven mostly by publications from other fields. Despite plenty of speculation, however, we know as yet very little about the true impact of evolutionary papers in journals not specifically classified as evolutionary. Here I present, for a wide range of journals, an analysis of the number of evolutionary papers they publish and their average impact. I show that there are large differences in impact among evolutionary and non-evolutionary papers within journals; while the impact of evolutionary papers published in multidisciplinary journals is substantially overestimated by their overall impact factor, the impact of evolutionary papers in many of the more specialized, non-evolutionary journals is significantly underestimated. This suggests that, for evolutionary biologists, publishing in high-impact multidisciplinary journals should not receive as much weight as it does now, while evolutionary papers in more narrowly defined journals are currently undervalued. Importantly, however, their ranking remains largely unaffected. While journal impact factors may thus indeed provide a meaningful qualitative measure of impact, a fair quantitative comparison requires a more sophisticated journal classification system, together with multiple field-specific impact statistics per journal.

Show MeSH