Limits...
A manually curated ChIP-seq benchmark demonstrates room for improvement in current peak-finder programs.

Rye MB, Sætrom P, Drabløs F - Nucleic Acids Res. (2010)

Bottom Line: Second, none of the programs returned peak-regions that corresponded to the actual resolution in ChIP-seq data.Our results showed that ChIP-seq peaks should be narrowed down to 100-400 bp, which is sufficient to identify unique peaks and binding sites.Based on these results, we propose a meta-approach that gives improved peak definitions.

View Article: PubMed Central - PubMed

Affiliation: Department of Cancer Research and Molecular Medicine, Norwegian University of Science and Technology, NO-7489 Trondheim, Norway. morten.rye@ntnu.no

ABSTRACT
Chromatin immunoprecipitation (ChIP) followed by high throughput sequencing (ChIP-seq) is rapidly becoming the method of choice for discovering cell-specific transcription factor binding locations genome wide. By aligning sequenced tags to the genome, binding locations appear as peaks in the tag profile. Several programs have been designed to identify such peaks, but program evaluation has been difficult due to the lack of benchmark data sets. We have created benchmark data sets for three transcription factors by manually evaluating a selection of potential binding regions that cover typical variation in peak size and appearance. Performance of five programs on this benchmark showed, first, that external control or background data was essential to limit the number of false positive peaks from the programs. However, >80% of these peaks could be manually filtered out by visual inspection alone, without using additional background data, showing that peak shape information is not fully exploited in the evaluated programs. Second, none of the programs returned peak-regions that corresponded to the actual resolution in ChIP-seq data. Our results showed that ChIP-seq peaks should be narrowed down to 100-400 bp, which is sufficient to identify unique peaks and binding sites. Based on these results, we propose a meta-approach that gives improved peak definitions.

Show MeSH

Related in: MedlinePlus

Performance of the different programs in the manually evaluated regions for the three transcription factors NRSF (A and D), SRF (B and E) and Max (C and F). The plots show how the FDR increases with percentage recovery of true peaks. Note the difference in scale on the FDR-axes for plots (A–C) compared to (D–F). Plots (A–C) show results when no background data is used in the analysis, whereas D–F show results when additional background data is included. The latter plots also show results from the meta-approach described in this study.
© Copyright Policy - creative-commons
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3045577&req=5

Figure 2: Performance of the different programs in the manually evaluated regions for the three transcription factors NRSF (A and D), SRF (B and E) and Max (C and F). The plots show how the FDR increases with percentage recovery of true peaks. Note the difference in scale on the FDR-axes for plots (A–C) compared to (D–F). Plots (A–C) show results when no background data is used in the analysis, whereas D–F show results when additional background data is included. The latter plots also show results from the meta-approach described in this study.

Mentions: The evaluation curves for each program in Figure 2 were created by identifying overlaps between the program-defined regions and the manually evaluated regions. The program-defined regions were first sorted in descending order according to their score, meaning that the most confident regions appeared at the beginning of the list. The list was then traversed, and the level of false positives (program regions overlapping with evaluation regions not classified as true or ambiguous peaks) was calculated each time a new true positive (program region overlapping with a true peak) was encountered. The number of false positives thus accumulates as more true positives are found.Figure 2.


A manually curated ChIP-seq benchmark demonstrates room for improvement in current peak-finder programs.

Rye MB, Sætrom P, Drabløs F - Nucleic Acids Res. (2010)

Performance of the different programs in the manually evaluated regions for the three transcription factors NRSF (A and D), SRF (B and E) and Max (C and F). The plots show how the FDR increases with percentage recovery of true peaks. Note the difference in scale on the FDR-axes for plots (A–C) compared to (D–F). Plots (A–C) show results when no background data is used in the analysis, whereas D–F show results when additional background data is included. The latter plots also show results from the meta-approach described in this study.
© Copyright Policy - creative-commons
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3045577&req=5

Figure 2: Performance of the different programs in the manually evaluated regions for the three transcription factors NRSF (A and D), SRF (B and E) and Max (C and F). The plots show how the FDR increases with percentage recovery of true peaks. Note the difference in scale on the FDR-axes for plots (A–C) compared to (D–F). Plots (A–C) show results when no background data is used in the analysis, whereas D–F show results when additional background data is included. The latter plots also show results from the meta-approach described in this study.
Mentions: The evaluation curves for each program in Figure 2 were created by identifying overlaps between the program-defined regions and the manually evaluated regions. The program-defined regions were first sorted in descending order according to their score, meaning that the most confident regions appeared at the beginning of the list. The list was then traversed, and the level of false positives (program regions overlapping with evaluation regions not classified as true or ambiguous peaks) was calculated each time a new true positive (program region overlapping with a true peak) was encountered. The number of false positives thus accumulates as more true positives are found.Figure 2.

Bottom Line: Second, none of the programs returned peak-regions that corresponded to the actual resolution in ChIP-seq data.Our results showed that ChIP-seq peaks should be narrowed down to 100-400 bp, which is sufficient to identify unique peaks and binding sites.Based on these results, we propose a meta-approach that gives improved peak definitions.

View Article: PubMed Central - PubMed

Affiliation: Department of Cancer Research and Molecular Medicine, Norwegian University of Science and Technology, NO-7489 Trondheim, Norway. morten.rye@ntnu.no

ABSTRACT
Chromatin immunoprecipitation (ChIP) followed by high throughput sequencing (ChIP-seq) is rapidly becoming the method of choice for discovering cell-specific transcription factor binding locations genome wide. By aligning sequenced tags to the genome, binding locations appear as peaks in the tag profile. Several programs have been designed to identify such peaks, but program evaluation has been difficult due to the lack of benchmark data sets. We have created benchmark data sets for three transcription factors by manually evaluating a selection of potential binding regions that cover typical variation in peak size and appearance. Performance of five programs on this benchmark showed, first, that external control or background data was essential to limit the number of false positive peaks from the programs. However, >80% of these peaks could be manually filtered out by visual inspection alone, without using additional background data, showing that peak shape information is not fully exploited in the evaluated programs. Second, none of the programs returned peak-regions that corresponded to the actual resolution in ChIP-seq data. Our results showed that ChIP-seq peaks should be narrowed down to 100-400 bp, which is sufficient to identify unique peaks and binding sites. Based on these results, we propose a meta-approach that gives improved peak definitions.

Show MeSH
Related in: MedlinePlus