Differences in citation frequency of clinical and basic science papers in cardiovascular research.
Bottom Line: It is also demonstrated that the groups of clinical and basic cardiovascular papers are also heterogeneous concerning citation frequency.It is concluded that none of the existing citation indicators appreciates these differences.At this moment these indicators should not be used for quality assessment of individual scientists and scientific niches with small numbers of scientists.
In this article, a critical analysis is performed on differences in citation frequency of basic and clinical cardiovascular papers. It appears that the latter papers are cited at about 40% higher frequency. The differences between the largest number of citations of the most cited papers are even larger. It is also demonstrated that the groups of clinical and basic cardiovascular papers are also heterogeneous concerning citation frequency. It is concluded that none of the existing citation indicators appreciates these differences. At this moment these indicators should not be used for quality assessment of individual scientists and scientific niches with small numbers of scientists.
Mentions: Figure 7 shows an analysis of peer reviewer’s quality assessments of 12 chemistry departments of a Dutch University and its correlation with the h-index and two citation indicators developed by the CWTS (CPP/JCSm and CPP/FCSm; both parameters have recently been adapted  in response to criticism ). The h-index was explained above; both other parameters relate citation of a set of papers to the same type of papers published in the same years in the same journals or sets of journals. The ordinate is labelled with ‘arbitrary units’. It has a different meaning for the three parameters: the h-index and two citation parameters of the CWTS. Thus, the differences between the three sets of parameters have no meaning. The data were collected from Table 1 in Ref. . The original study  aimed at a comparison between these three citation parameters, not on the relation between peer review parameters and the three citation parameters. This small amount of data was analyzed here in an alternative way. The result is obvious. The pivotal issue is the absence of any difference between research labelled by peers as ‘good’ or ‘excellent’ for whatever citation parameter, including the h-index (see also ) . There is one other problem in relating peer judgment with citation parameters. This problem also applies to the judgment on grant proposals. What is the dependent and what is the independent parameter? Moreover, it appears important whether or not peers are informed by the organization asking for their advice on citation data. Even in the case that such information is NOT provided, modern retrieval tools make it possible that peers search for these data themselves and that this influences a judgment which would otherwise have been based only on reading a paper or proposal. This problem has been emphasized previously by Moed . Recently, Van den Besselaar and Leydesdorff  pointed out that the best non-granted biomedical research proposals of ‘NWO’ (Dutch Organization for Fundamental Research) actually had higher bibliometrical parameters than the granted proposals, emphasizing again that citation data are not suitable for the distinction between what is considered as ‘good’ and ‘excellent’. Finally, Spaan  has paid attention to the fact that citation parameters that are related to the total years of scientific activity may be unfair to women in general and also to scientists with a ‘break’ in their career, f.e. due to a period of extensive clinical training. Also, a major move from one field to another may put creative scientists with a temporary decrease in scientific output at a disadvantage. This may not be good news for scientific innovation.Fig. 7