Limits...
Validation of 3D EM Reconstructions: The Phantom in the Noise.

Heymann JB - AIMS Biophys (2015)

Bottom Line: This poses the risk of inappropriate data processing with dubious results.How can we test that a map is a coherent structure present in the images selected from the micrographs?Adopting such a test can aid the microscopist in assessing the usefulness of the micrographs taken before committing to lengthy processing with questionable outcomes.

View Article: PubMed Central - HTML - PubMed

Affiliation: National Institute of Arthritis and Musculoskeletal and Skin Diseases, National Institutes of Health, 50 South Dr, Bethesda, MD 20892, USA.

ABSTRACT

Validation is a necessity to trust the structures solved by electron microscopy by single particle techniques. The impressive achievements in single particle reconstruction fuel its expansion beyond a small community of image processing experts. This poses the risk of inappropriate data processing with dubious results. Nowhere is it more clearly illustrated than in the recovery of a reference density map from pure noise aligned to that map-a phantom in the noise. Appropriate use of existing validating methods such as resolution-limited alignment and the processing of independent data sets ("gold standard") avoid this pitfall. However, these methods can be undermined by biases introduced in various subtle ways. How can we test that a map is a coherent structure present in the images selected from the micrographs? In stead of viewing the phantom emerging from noise as a cautionary tale, it should be used as a defining baseline. Any map is always recoverable from noise images, provided a sufficient number of images are aligned and used in reconstruction. However, with smaller numbers of images, the expected coherence in the real particle images should yield better reconstructions than equivalent numbers of noise or background images, even without masking or imposing resolution limits as potential biases. The validation test proposed is therefore a simple alignment of a limited number of micrograph and noise images against the final reconstruction as reference, demonstrating that the micrograph images yield a better reconstruction. I examine synthetic cases to relate the resolution of a reconstruction to the alignment error as a function of the signal-to-noise ratio. I also administered the test to real cases of publicly available data. Adopting such a test can aid the microscopist in assessing the usefulness of the micrographs taken before committing to lengthy processing with questionable outcomes.

No MeSH data available.


(a) A projection of lumazine synthase (left) corrupted with different levels of noise as indicated by the SNRimp values (1–0.01). (b–d) The errors in alignment were determined for projections of synthetic maps of proteinase K (PK; black disks) and lumazine synthase (LS; blue diamonds) with different imposed SNR values. View (b) and in-plane (c) rotational errors show a rapid change between SNR values of 0.01 and 0.1. (d) Translational errors show a gradual change with the SNR. (e) Resolution estimates of reconstructions of PK from 5000 images, and LS from 100 images (6000 asymmetric units).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4440490&req=5

Figure 1: (a) A projection of lumazine synthase (left) corrupted with different levels of noise as indicated by the SNRimp values (1–0.01). (b–d) The errors in alignment were determined for projections of synthetic maps of proteinase K (PK; black disks) and lumazine synthase (LS; blue diamonds) with different imposed SNR values. View (b) and in-plane (c) rotational errors show a rapid change between SNR values of 0.01 and 0.1. (d) Translational errors show a gradual change with the SNR. (e) Resolution estimates of reconstructions of PK from 5000 images, and LS from 100 images (6000 asymmetric units).

Mentions: Two synthetic cases were chosen with different sizes and symmetries. Proteinase K (PK) is a 30 kDa monomer with no symmetry [22], while lumazine synthase (LS) is a 1.03 MDa 60-mer with icosahedral symmetry [23]. Density maps of these were calculated from the crystal structures, randomly projected, gaussian noise added to the projections (Figure 1a), and the resultant images aligned to the density maps using all information up to Nyquist frequency (2 Å in these cases). The measure of resolution chosen is the threshold of 0.5 in the FSC curve (FSC0.5), both as a conservative choice and because it is the most commonly reported in the EMDB (http://www.ebi.ac.uk/pdbe/emdb/).


Validation of 3D EM Reconstructions: The Phantom in the Noise.

Heymann JB - AIMS Biophys (2015)

(a) A projection of lumazine synthase (left) corrupted with different levels of noise as indicated by the SNRimp values (1–0.01). (b–d) The errors in alignment were determined for projections of synthetic maps of proteinase K (PK; black disks) and lumazine synthase (LS; blue diamonds) with different imposed SNR values. View (b) and in-plane (c) rotational errors show a rapid change between SNR values of 0.01 and 0.1. (d) Translational errors show a gradual change with the SNR. (e) Resolution estimates of reconstructions of PK from 5000 images, and LS from 100 images (6000 asymmetric units).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4440490&req=5

Figure 1: (a) A projection of lumazine synthase (left) corrupted with different levels of noise as indicated by the SNRimp values (1–0.01). (b–d) The errors in alignment were determined for projections of synthetic maps of proteinase K (PK; black disks) and lumazine synthase (LS; blue diamonds) with different imposed SNR values. View (b) and in-plane (c) rotational errors show a rapid change between SNR values of 0.01 and 0.1. (d) Translational errors show a gradual change with the SNR. (e) Resolution estimates of reconstructions of PK from 5000 images, and LS from 100 images (6000 asymmetric units).
Mentions: Two synthetic cases were chosen with different sizes and symmetries. Proteinase K (PK) is a 30 kDa monomer with no symmetry [22], while lumazine synthase (LS) is a 1.03 MDa 60-mer with icosahedral symmetry [23]. Density maps of these were calculated from the crystal structures, randomly projected, gaussian noise added to the projections (Figure 1a), and the resultant images aligned to the density maps using all information up to Nyquist frequency (2 Å in these cases). The measure of resolution chosen is the threshold of 0.5 in the FSC curve (FSC0.5), both as a conservative choice and because it is the most commonly reported in the EMDB (http://www.ebi.ac.uk/pdbe/emdb/).

Bottom Line: This poses the risk of inappropriate data processing with dubious results.How can we test that a map is a coherent structure present in the images selected from the micrographs?Adopting such a test can aid the microscopist in assessing the usefulness of the micrographs taken before committing to lengthy processing with questionable outcomes.

View Article: PubMed Central - HTML - PubMed

Affiliation: National Institute of Arthritis and Musculoskeletal and Skin Diseases, National Institutes of Health, 50 South Dr, Bethesda, MD 20892, USA.

ABSTRACT

Validation is a necessity to trust the structures solved by electron microscopy by single particle techniques. The impressive achievements in single particle reconstruction fuel its expansion beyond a small community of image processing experts. This poses the risk of inappropriate data processing with dubious results. Nowhere is it more clearly illustrated than in the recovery of a reference density map from pure noise aligned to that map-a phantom in the noise. Appropriate use of existing validating methods such as resolution-limited alignment and the processing of independent data sets ("gold standard") avoid this pitfall. However, these methods can be undermined by biases introduced in various subtle ways. How can we test that a map is a coherent structure present in the images selected from the micrographs? In stead of viewing the phantom emerging from noise as a cautionary tale, it should be used as a defining baseline. Any map is always recoverable from noise images, provided a sufficient number of images are aligned and used in reconstruction. However, with smaller numbers of images, the expected coherence in the real particle images should yield better reconstructions than equivalent numbers of noise or background images, even without masking or imposing resolution limits as potential biases. The validation test proposed is therefore a simple alignment of a limited number of micrograph and noise images against the final reconstruction as reference, demonstrating that the micrograph images yield a better reconstruction. I examine synthetic cases to relate the resolution of a reconstruction to the alignment error as a function of the signal-to-noise ratio. I also administered the test to real cases of publicly available data. Adopting such a test can aid the microscopist in assessing the usefulness of the micrographs taken before committing to lengthy processing with questionable outcomes.

No MeSH data available.