Effectiveness of journal ranking schemes as a tool for locating information.
Bottom Line:
Here, we systematically evaluate the effectiveness of journals, through the work of editors and reviewers, at evaluating unpublished research.Our model enables us to quantify both the typical impact and the range of impacts of papers published in a journal.Finally, we propose a journal-ranking scheme that maximizes the efficiency of locating high impact research.
View Article:
PubMed Central - PubMed
Affiliation: Department of Physics and Astronomy, Northwestern University, Evanston, Illinois, USA.
ABSTRACT
Show MeSH
Background: The rise of electronic publishing, preprint archives, blogs, and wikis is raising concerns among publishers, editors, and scientists about the present day relevance of academic journals and traditional peer review. These concerns are especially fuelled by the ability of search engines to automatically identify and sort information. It appears that academic journals can only remain relevant if acceptance of research for publication within a journal allows readers to infer immediate, reliable information on the value of that research. Methodology/principal findings: Here, we systematically evaluate the effectiveness of journals, through the work of editors and reviewers, at evaluating unpublished research. We find that the distribution of the number of citations to a paper published in a given journal in a specific year converges to a steady state after a journal-specific transient time, and demonstrate that in the steady state the logarithm of the number of citations has a journal-specific typical value. We then develop a model for the asymptotic number of citations accrued by papers published in a journal that closely matches the data. Conclusions/significance: Our model enables us to quantify both the typical impact and the range of impacts of papers published in a journal. Finally, we propose a journal-ranking scheme that maximizes the efficiency of locating high impact research. |
Related In:
Results -
Collection
getmorefigures.php?uid=PMC2244807&req=5
Mentions: One can interpret γ as the value of q at which one can expect a paper to get cited once (Figure 2A). More generally, one could write n = floor(10q+ε−γ), where ε∈N(0,σε), to account for external influences to the number of citations. For example, assuming γ = 0 and q = 3, one would get n = 794 for ε = −0.1 and n = 1258 for ε = 0.1. However, if ε is independent of J, will not be significantly affected by ε. Thus, even though the number of citations to individual papers may change, the mean for a journal will not. To demonstrate the agreement between our model and the data, in Figure 2 we plot the moments of the empirical distributions for each journal together with the predictions of our model for those quantities. It is visually apparent that the model provides a close description of the data. |
View Article: PubMed Central - PubMed
Affiliation: Department of Physics and Astronomy, Northwestern University, Evanston, Illinois, USA.
Background: The rise of electronic publishing, preprint archives, blogs, and wikis is raising concerns among publishers, editors, and scientists about the present day relevance of academic journals and traditional peer review. These concerns are especially fuelled by the ability of search engines to automatically identify and sort information. It appears that academic journals can only remain relevant if acceptance of research for publication within a journal allows readers to infer immediate, reliable information on the value of that research.
Methodology/principal findings: Here, we systematically evaluate the effectiveness of journals, through the work of editors and reviewers, at evaluating unpublished research. We find that the distribution of the number of citations to a paper published in a given journal in a specific year converges to a steady state after a journal-specific transient time, and demonstrate that in the steady state the logarithm of the number of citations has a journal-specific typical value. We then develop a model for the asymptotic number of citations accrued by papers published in a journal that closely matches the data.
Conclusions/significance: Our model enables us to quantify both the typical impact and the range of impacts of papers published in a journal. Finally, we propose a journal-ranking scheme that maximizes the efficiency of locating high impact research.