Limits...
Methodology capture: discriminating between the "best" and the rest of community practice.

Eales JM, Pinney JW, Stevens RD, Robertson DL - BMC Bioinformatics (2008)

Bottom Line: We have identified a structured community of phylogenetic researchers performing analyses that are customary in their own local community and significantly different from those in other areas.We propose that the practice of expert authors from the field of evolutionary biology is the closest to contemporary best practice in phylogenetic experimental design.Capturing best practice is, however, a complex task and should also acknowledge the differences between fields such as the specific context of the analysis.

View Article: PubMed Central - HTML - PubMed

Affiliation: Faculty of Life Sciences, University of Manchester, Manchester, UK. james.eales@postgrad.manchester.ac.uk

ABSTRACT

Background: The methodologies we use both enable and help define our research. However, as experimental complexity has increased the choice of appropriate methodologies has become an increasingly difficult task. This makes it difficult to keep track of available bioinformatics software, let alone the most suitable protocols in a specific research area. To remedy this we present an approach for capturing methodology from literature in order to identify and, thus, define best practice within a field.

Results: Our approach is to implement data extraction techniques on the full-text of scientific articles to obtain the set of experimental protocols used by an entire scientific discipline, molecular phylogenetics. Our methodology for identifying methodologies could in principle be applied to any scientific discipline, whether or not computer-based. We find a number of issues related to the nature of best practice, as opposed to community practice. We find that there is much heterogeneity in the use of molecular phylogenetic methods and software, some of which is related to poor specification of protocols. We also find that phylogenetic practice exhibits field-specific tendencies that have increased through time, despite the generic nature of the available software. We used the practice of highly published and widely collaborative researchers ("expert" researchers) to analyse the influence of authority on community practice. We find expert authors exhibit patterns of practice common to their field and therefore act as useful field-specific practice indicators.

Conclusion: We have identified a structured community of phylogenetic researchers performing analyses that are customary in their own local community and significantly different from those in other areas. Best practice information can help to bridge such subtle differences by increasing communication of protocols to a wider audience. We propose that the practice of expert authors from the field of evolutionary biology is the closest to contemporary best practice in phylogenetic experimental design. Capturing best practice is, however, a complex task and should also acknowledge the differences between fields such as the specific context of the analysis.

Show MeSH
Usage of protocols by field and through time. Protocol networks for the years 1996, 2000 and 2005. Nodes represent individual protocols and are sized according to the number of times they were used. Each node is also a pie chart describing the proportion of all uses of that protocol by each field group. Edges denote an F-measure value of greater than 0.75 phylogenetic term similarity between the protocols. The networks shown are the largest connected component after the F-measure threshold was applied.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2553348&req=5

Figure 2: Usage of protocols by field and through time. Protocol networks for the years 1996, 2000 and 2005. Nodes represent individual protocols and are sized according to the number of times they were used. Each node is also a pie chart describing the proportion of all uses of that protocol by each field group. Edges denote an F-measure value of greater than 0.75 phylogenetic term similarity between the protocols. The networks shown are the largest connected component after the F-measure threshold was applied.

Mentions: Given that a best practice proposal should be able to cater for all users of phylogenetics, we assessed the differences/similarities between these fields and how they have developed through time. Furthermore, we can use this to assess whether there is methodological communication between fields. To do this we calculate the proportion of articles from each journal group that contained each of the protocols implemented in each year and generate a series of networks (Figure 2) that map the methodological choices made by authors from three different fields during the last 10 years (Additional file 1). These networks indicate that, while there is overlap, a significant shift in methodological preference has occurred between fields. We have used calculations of the network assortativity coefficient (r) [28] to highlight changes in methodological choice. In this case a larger r-value indicates field-specific method choice. Overall network assortativity and some field-field assortativity comparisons, specifically, Evolutionary Biology/Microbiology and Evolutionary Biology/Virology, have increased throughout this period (Figure 3). Compared to random networks, there is a significantly different increase in overall network assortativity (Figure 3). This is presumably due to the larger increase in assortativity between the Evolutionary Biology field and the other two fields (Figure 3). There was no change in assortativity between the Microbiology and Virology fields, with the values being inside the 95% confidence interval from 1996–1998 and 2000, and when outside the 95% confidence interval only varying between -0.04 and 0.04.


Methodology capture: discriminating between the "best" and the rest of community practice.

Eales JM, Pinney JW, Stevens RD, Robertson DL - BMC Bioinformatics (2008)

Usage of protocols by field and through time. Protocol networks for the years 1996, 2000 and 2005. Nodes represent individual protocols and are sized according to the number of times they were used. Each node is also a pie chart describing the proportion of all uses of that protocol by each field group. Edges denote an F-measure value of greater than 0.75 phylogenetic term similarity between the protocols. The networks shown are the largest connected component after the F-measure threshold was applied.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2553348&req=5

Figure 2: Usage of protocols by field and through time. Protocol networks for the years 1996, 2000 and 2005. Nodes represent individual protocols and are sized according to the number of times they were used. Each node is also a pie chart describing the proportion of all uses of that protocol by each field group. Edges denote an F-measure value of greater than 0.75 phylogenetic term similarity between the protocols. The networks shown are the largest connected component after the F-measure threshold was applied.
Mentions: Given that a best practice proposal should be able to cater for all users of phylogenetics, we assessed the differences/similarities between these fields and how they have developed through time. Furthermore, we can use this to assess whether there is methodological communication between fields. To do this we calculate the proportion of articles from each journal group that contained each of the protocols implemented in each year and generate a series of networks (Figure 2) that map the methodological choices made by authors from three different fields during the last 10 years (Additional file 1). These networks indicate that, while there is overlap, a significant shift in methodological preference has occurred between fields. We have used calculations of the network assortativity coefficient (r) [28] to highlight changes in methodological choice. In this case a larger r-value indicates field-specific method choice. Overall network assortativity and some field-field assortativity comparisons, specifically, Evolutionary Biology/Microbiology and Evolutionary Biology/Virology, have increased throughout this period (Figure 3). Compared to random networks, there is a significantly different increase in overall network assortativity (Figure 3). This is presumably due to the larger increase in assortativity between the Evolutionary Biology field and the other two fields (Figure 3). There was no change in assortativity between the Microbiology and Virology fields, with the values being inside the 95% confidence interval from 1996–1998 and 2000, and when outside the 95% confidence interval only varying between -0.04 and 0.04.

Bottom Line: We have identified a structured community of phylogenetic researchers performing analyses that are customary in their own local community and significantly different from those in other areas.We propose that the practice of expert authors from the field of evolutionary biology is the closest to contemporary best practice in phylogenetic experimental design.Capturing best practice is, however, a complex task and should also acknowledge the differences between fields such as the specific context of the analysis.

View Article: PubMed Central - HTML - PubMed

Affiliation: Faculty of Life Sciences, University of Manchester, Manchester, UK. james.eales@postgrad.manchester.ac.uk

ABSTRACT

Background: The methodologies we use both enable and help define our research. However, as experimental complexity has increased the choice of appropriate methodologies has become an increasingly difficult task. This makes it difficult to keep track of available bioinformatics software, let alone the most suitable protocols in a specific research area. To remedy this we present an approach for capturing methodology from literature in order to identify and, thus, define best practice within a field.

Results: Our approach is to implement data extraction techniques on the full-text of scientific articles to obtain the set of experimental protocols used by an entire scientific discipline, molecular phylogenetics. Our methodology for identifying methodologies could in principle be applied to any scientific discipline, whether or not computer-based. We find a number of issues related to the nature of best practice, as opposed to community practice. We find that there is much heterogeneity in the use of molecular phylogenetic methods and software, some of which is related to poor specification of protocols. We also find that phylogenetic practice exhibits field-specific tendencies that have increased through time, despite the generic nature of the available software. We used the practice of highly published and widely collaborative researchers ("expert" researchers) to analyse the influence of authority on community practice. We find expert authors exhibit patterns of practice common to their field and therefore act as useful field-specific practice indicators.

Conclusion: We have identified a structured community of phylogenetic researchers performing analyses that are customary in their own local community and significantly different from those in other areas. Best practice information can help to bridge such subtle differences by increasing communication of protocols to a wider audience. We propose that the practice of expert authors from the field of evolutionary biology is the closest to contemporary best practice in phylogenetic experimental design. Capturing best practice is, however, a complex task and should also acknowledge the differences between fields such as the specific context of the analysis.

Show MeSH