Limits...
Differences in citation frequency of clinical and basic science papers in cardiovascular research.

Opthof T - Med Biol Eng Comput (2011)

Bottom Line: It is also demonstrated that the groups of clinical and basic cardiovascular papers are also heterogeneous concerning citation frequency.It is concluded that none of the existing citation indicators appreciates these differences.At this moment these indicators should not be used for quality assessment of individual scientists and scientific niches with small numbers of scientists.

View Article: PubMed Central - PubMed

ABSTRACT
In this article, a critical analysis is performed on differences in citation frequency of basic and clinical cardiovascular papers. It appears that the latter papers are cited at about 40% higher frequency. The differences between the largest number of citations of the most cited papers are even larger. It is also demonstrated that the groups of clinical and basic cardiovascular papers are also heterogeneous concerning citation frequency. It is concluded that none of the existing citation indicators appreciates these differences. At this moment these indicators should not be used for quality assessment of individual scientists and scientific niches with small numbers of scientists.

Show MeSH
Comparison of peer esteem of 12 chemistry groups of a Dutch University with three citation indicators. There is no difference between the peer esteem ‘good’ and ‘excellent’. Data from Table 1 in Ref. [30]
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3104007&req=5

Fig7: Comparison of peer esteem of 12 chemistry groups of a Dutch University with three citation indicators. There is no difference between the peer esteem ‘good’ and ‘excellent’. Data from Table 1 in Ref. [30]

Mentions: Figure 7 shows an analysis of peer reviewer’s quality assessments of 12 chemistry departments of a Dutch University and its correlation with the h-index and two citation indicators developed by the CWTS (CPP/JCSm and CPP/FCSm; both parameters have recently been adapted [32] in response to criticism [17]). The h-index was explained above; both other parameters relate citation of a set of papers to the same type of papers published in the same years in the same journals or sets of journals. The ordinate is labelled with ‘arbitrary units’. It has a different meaning for the three parameters: the h-index and two citation parameters of the CWTS. Thus, the differences between the three sets of parameters have no meaning. The data were collected from Table 1 in Ref. [30]. The original study [30] aimed at a comparison between these three citation parameters, not on the relation between peer review parameters and the three citation parameters. This small amount of data was analyzed here in an alternative way. The result is obvious. The pivotal issue is the absence of any difference between research labelled by peers as ‘good’ or ‘excellent’ for whatever citation parameter, including the h-index (see also [18]) . There is one other problem in relating peer judgment with citation parameters. This problem also applies to the judgment on grant proposals. What is the dependent and what is the independent parameter? Moreover, it appears important whether or not peers are informed by the organization asking for their advice on citation data. Even in the case that such information is NOT provided, modern retrieval tools make it possible that peers search for these data themselves and that this influences a judgment which would otherwise have been based only on reading a paper or proposal. This problem has been emphasized previously by Moed [14]. Recently, Van den Besselaar and Leydesdorff [28] pointed out that the best non-granted biomedical research proposals of ‘NWO’ (Dutch Organization for Fundamental Research) actually had higher bibliometrical parameters than the granted proposals, emphasizing again that citation data are not suitable for the distinction between what is considered as ‘good’ and ‘excellent’. Finally, Spaan [25] has paid attention to the fact that citation parameters that are related to the total years of scientific activity may be unfair to women in general and also to scientists with a ‘break’ in their career, f.e. due to a period of extensive clinical training. Also, a major move from one field to another may put creative scientists with a temporary decrease in scientific output at a disadvantage. This may not be good news for scientific innovation.Fig. 7


Differences in citation frequency of clinical and basic science papers in cardiovascular research.

Opthof T - Med Biol Eng Comput (2011)

Comparison of peer esteem of 12 chemistry groups of a Dutch University with three citation indicators. There is no difference between the peer esteem ‘good’ and ‘excellent’. Data from Table 1 in Ref. [30]
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3104007&req=5

Fig7: Comparison of peer esteem of 12 chemistry groups of a Dutch University with three citation indicators. There is no difference between the peer esteem ‘good’ and ‘excellent’. Data from Table 1 in Ref. [30]
Mentions: Figure 7 shows an analysis of peer reviewer’s quality assessments of 12 chemistry departments of a Dutch University and its correlation with the h-index and two citation indicators developed by the CWTS (CPP/JCSm and CPP/FCSm; both parameters have recently been adapted [32] in response to criticism [17]). The h-index was explained above; both other parameters relate citation of a set of papers to the same type of papers published in the same years in the same journals or sets of journals. The ordinate is labelled with ‘arbitrary units’. It has a different meaning for the three parameters: the h-index and two citation parameters of the CWTS. Thus, the differences between the three sets of parameters have no meaning. The data were collected from Table 1 in Ref. [30]. The original study [30] aimed at a comparison between these three citation parameters, not on the relation between peer review parameters and the three citation parameters. This small amount of data was analyzed here in an alternative way. The result is obvious. The pivotal issue is the absence of any difference between research labelled by peers as ‘good’ or ‘excellent’ for whatever citation parameter, including the h-index (see also [18]) . There is one other problem in relating peer judgment with citation parameters. This problem also applies to the judgment on grant proposals. What is the dependent and what is the independent parameter? Moreover, it appears important whether or not peers are informed by the organization asking for their advice on citation data. Even in the case that such information is NOT provided, modern retrieval tools make it possible that peers search for these data themselves and that this influences a judgment which would otherwise have been based only on reading a paper or proposal. This problem has been emphasized previously by Moed [14]. Recently, Van den Besselaar and Leydesdorff [28] pointed out that the best non-granted biomedical research proposals of ‘NWO’ (Dutch Organization for Fundamental Research) actually had higher bibliometrical parameters than the granted proposals, emphasizing again that citation data are not suitable for the distinction between what is considered as ‘good’ and ‘excellent’. Finally, Spaan [25] has paid attention to the fact that citation parameters that are related to the total years of scientific activity may be unfair to women in general and also to scientists with a ‘break’ in their career, f.e. due to a period of extensive clinical training. Also, a major move from one field to another may put creative scientists with a temporary decrease in scientific output at a disadvantage. This may not be good news for scientific innovation.Fig. 7

Bottom Line: It is also demonstrated that the groups of clinical and basic cardiovascular papers are also heterogeneous concerning citation frequency.It is concluded that none of the existing citation indicators appreciates these differences.At this moment these indicators should not be used for quality assessment of individual scientists and scientific niches with small numbers of scientists.

View Article: PubMed Central - PubMed

ABSTRACT
In this article, a critical analysis is performed on differences in citation frequency of basic and clinical cardiovascular papers. It appears that the latter papers are cited at about 40% higher frequency. The differences between the largest number of citations of the most cited papers are even larger. It is also demonstrated that the groups of clinical and basic cardiovascular papers are also heterogeneous concerning citation frequency. It is concluded that none of the existing citation indicators appreciates these differences. At this moment these indicators should not be used for quality assessment of individual scientists and scientific niches with small numbers of scientists.

Show MeSH