Limits...
A multi-center study benchmarks software tools for label-free proteome quantification

View Article: PubMed Central - PubMed

ABSTRACT

The consistent and accurate quantification of proteins by mass spectrometry (MS)-based proteomics depends on the performance of instruments, acquisition methods and data analysis software. In collaboration with the software developers, we evaluated OpenSWATH, SWATH2.0, Skyline, Spectronaut and DIA-Umpire, five of the most widely used software methods for processing data from SWATH-MS (sequential window acquisition of all theoretical fragment ion spectra), a method that uses data-independent acquisition (DIA) for label-free protein quantification. We analyzed high-complexity test datasets from hybrid proteome samples of defined quantitative composition acquired on two different MS instruments using different SWATH isolation windows setups. For consistent evaluation we developed LFQbench, an R-package to calculate metrics of precision and accuracy in label-free quantitative MS, and report the identification performance, robustness and specificity of each software tool. Our reference datasets enabled developers to improve their software tools. After optimization, all tools provided highly convergent identification and reliable quantification performance, underscoring their robustness for label-free quantitative proteomics.

No MeSH data available.


Retention time differences and correlation of reported peak intensities between all software tools for the respective matching precursors.Retention time outliers (upper right panels) are plotted in the color of the outlier software tool (see color legend in the diagonal panels). Diagonal panels show the total number and percentage (to the total number of common detected peptides) of outliers of each respective software tool. Outliers have been defined as producing a standard deviation of the peak retention time greater than 0.2 minutes relative to all other software tools detecting that precursor, after removing ambiguous cases, in which more than one software tool produce a greater standard deviation in the peak retention time. The correlation of reported peak intensities is displayed at the lower left panels. The retention time outliers are also marked in the respective correlation plots.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC5120688&req=5

Figure 4: Retention time differences and correlation of reported peak intensities between all software tools for the respective matching precursors.Retention time outliers (upper right panels) are plotted in the color of the outlier software tool (see color legend in the diagonal panels). Diagonal panels show the total number and percentage (to the total number of common detected peptides) of outliers of each respective software tool. Outliers have been defined as producing a standard deviation of the peak retention time greater than 0.2 minutes relative to all other software tools detecting that precursor, after removing ambiguous cases, in which more than one software tool produce a greater standard deviation in the peak retention time. The correlation of reported peak intensities is displayed at the lower left panels. The retention time outliers are also marked in the respective correlation plots.

Mentions: The analysis of common peptides provides a unique opportunity to assess the correctness of the peak picking of each tool. Analyzing one of the injections, we found that all tools pick the same peak (based on retention time) in more than 98% of the cases. All library-based tools each had less than 0.3% of outliers, and DIA-Umpire has approximately 1%. This emphasizes the robustness of SWATH, as even orthogonal identification methods (library-based vs. pseudo-spectra database search) agree in about 99% of the cases (Figure 4). Peptide intensities reported by library-based tools show a very high correlation (R^2: 0.93 – 0.97). The observed differences between DIA-Umpire and any library-based tool (R^2: 0.73 – 0.75) in the first iteration (Supplementary Figure 15) were reduced in the second iteration (R^2: 0.76 – 0.80) (Figure 4). These differences are likely due to the selection of different fragments used for quantification, as about 30% of the top two most intense fragments reported by DIA-Umpire were not included in the DDA library (Supplementary Figure 16). Since DIA-Umpire relies on correct matching of MS1 precursors with their fragments, even high intensity precursors may not be identified by DIA-Umpire due to interferences in the MS1 space (see Supplementary Figure 22). Notably, these differences did not negatively affect the relative quantification accuracy of DIA-Umpire (Figure 2).


A multi-center study benchmarks software tools for label-free proteome quantification
Retention time differences and correlation of reported peak intensities between all software tools for the respective matching precursors.Retention time outliers (upper right panels) are plotted in the color of the outlier software tool (see color legend in the diagonal panels). Diagonal panels show the total number and percentage (to the total number of common detected peptides) of outliers of each respective software tool. Outliers have been defined as producing a standard deviation of the peak retention time greater than 0.2 minutes relative to all other software tools detecting that precursor, after removing ambiguous cases, in which more than one software tool produce a greater standard deviation in the peak retention time. The correlation of reported peak intensities is displayed at the lower left panels. The retention time outliers are also marked in the respective correlation plots.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC5120688&req=5

Figure 4: Retention time differences and correlation of reported peak intensities between all software tools for the respective matching precursors.Retention time outliers (upper right panels) are plotted in the color of the outlier software tool (see color legend in the diagonal panels). Diagonal panels show the total number and percentage (to the total number of common detected peptides) of outliers of each respective software tool. Outliers have been defined as producing a standard deviation of the peak retention time greater than 0.2 minutes relative to all other software tools detecting that precursor, after removing ambiguous cases, in which more than one software tool produce a greater standard deviation in the peak retention time. The correlation of reported peak intensities is displayed at the lower left panels. The retention time outliers are also marked in the respective correlation plots.
Mentions: The analysis of common peptides provides a unique opportunity to assess the correctness of the peak picking of each tool. Analyzing one of the injections, we found that all tools pick the same peak (based on retention time) in more than 98% of the cases. All library-based tools each had less than 0.3% of outliers, and DIA-Umpire has approximately 1%. This emphasizes the robustness of SWATH, as even orthogonal identification methods (library-based vs. pseudo-spectra database search) agree in about 99% of the cases (Figure 4). Peptide intensities reported by library-based tools show a very high correlation (R^2: 0.93 – 0.97). The observed differences between DIA-Umpire and any library-based tool (R^2: 0.73 – 0.75) in the first iteration (Supplementary Figure 15) were reduced in the second iteration (R^2: 0.76 – 0.80) (Figure 4). These differences are likely due to the selection of different fragments used for quantification, as about 30% of the top two most intense fragments reported by DIA-Umpire were not included in the DDA library (Supplementary Figure 16). Since DIA-Umpire relies on correct matching of MS1 precursors with their fragments, even high intensity precursors may not be identified by DIA-Umpire due to interferences in the MS1 space (see Supplementary Figure 22). Notably, these differences did not negatively affect the relative quantification accuracy of DIA-Umpire (Figure 2).

View Article: PubMed Central - PubMed

ABSTRACT

The consistent and accurate quantification of proteins by mass spectrometry (MS)-based proteomics depends on the performance of instruments, acquisition methods and data analysis software. In collaboration with the software developers, we evaluated OpenSWATH, SWATH2.0, Skyline, Spectronaut and DIA-Umpire, five of the most widely used software methods for processing data from SWATH-MS (sequential window acquisition of all theoretical fragment ion spectra), a method that uses data-independent acquisition (DIA) for label-free protein quantification. We analyzed high-complexity test datasets from hybrid proteome samples of defined quantitative composition acquired on two different MS instruments using different SWATH isolation windows setups. For consistent evaluation we developed LFQbench, an R-package to calculate metrics of precision and accuracy in label-free quantitative MS, and report the identification performance, robustness and specificity of each software tool. Our reference datasets enabled developers to improve their software tools. After optimization, all tools provided highly convergent identification and reliable quantification performance, underscoring their robustness for label-free quantitative proteomics.

No MeSH data available.