Limits...
Of gastro and the gold standard: evaluation and policy implications of norovirus test performance for outbreak detection.

Fisman DN, Greer AL, Brouhanski G, Drews SJ - J Transl Med (2009)

Bottom Line: If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested.Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

View Article: PubMed Central - HTML - PubMed

Affiliation: Division of Epidemiology and Surveillance, Ontario Agency for Health Protection and Promotion, Toronto, Canada. david.fisman@gmail.com

ABSTRACT

Background: The norovirus group (NVG) of caliciviruses are the etiological agents of most institutional outbreaks of gastroenteritis in North America and Europe. Identification of NVG is complicated by the non-culturable nature of this virus, and the absence of a diagnostic gold standard makes traditional evaluation of test characteristics problematic.

Methods: We evaluated 189 specimens derived from 440 acute gastroenteritis outbreaks investigated in Ontario in 2006-07. Parallel testing for NVG was performed with real-time reverse-transcriptase polymerase chain reaction (RT2-PCR), enzyme immunoassay (EIA) and electron microscopy (EM). Test characteristics (sensitivity and specificity) were estimated using latent class models and composite reference standard methods. The practical implications of test characteristics were evaluated using binomial probability models.

Results: Latent class modelling estimated sensitivities of RT2-PCR, EIA, and EM as 100%, 86%, and 17% respectively; specificities were 84%, 92%, and 100%; estimates obtained using a composite reference standard were similar. If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested. Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.

Conclusion: Our findings support the characterization of EM as lacking sensitivity for NVG outbreaks. The high sensitivity of RT2-PCR and EIA permit identification of NVG outbreaks with testing of limited numbers of clinical specimens. Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Show MeSH

Related in: MedlinePlus

Probability of One or More Positive Test Results by Specimens Tested, Under Varying Assumptions Regarding Proportion of True Positive Specimens. Curves are constructed based on a binomial distribution. Each contour represents a different proportion of true positive test specimens. Graph (A) represents estimates generated based on high (100%) sensitivity estimated for real-time reverse-transcriptase polymerase chain reaction using both latent class modeling (LCM) and composite reference standard (CRS) methods. Graph (B) presents estimates generated using LCM estimates for enzyme immunoassay (EIA) sensitivity (86%). A graph using EIA sensitivity estimates from CRS would be similar to graph (A) due to high (97%) sensitivity estimates using the latter approach.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2667494&req=5

Figure 3: Probability of One or More Positive Test Results by Specimens Tested, Under Varying Assumptions Regarding Proportion of True Positive Specimens. Curves are constructed based on a binomial distribution. Each contour represents a different proportion of true positive test specimens. Graph (A) represents estimates generated based on high (100%) sensitivity estimated for real-time reverse-transcriptase polymerase chain reaction using both latent class modeling (LCM) and composite reference standard (CRS) methods. Graph (B) presents estimates generated using LCM estimates for enzyme immunoassay (EIA) sensitivity (86%). A graph using EIA sensitivity estimates from CRS would be similar to graph (A) due to high (97%) sensitivity estimates using the latter approach.

Mentions: We assessed the likelihood that an individual specimen contained NVG material by comparing submitted specimen numbers in identified outbreaks to the number of specimens testing positive by RT2-PCR in those same outbreaks (Table 3). Depending on the presence or absence of EM confirmation of a given outbreak, the proportion of specimens testing positive in apparent outbreaks varied from approximately 58–72% (with 95% confidence intervals as low as 54% and as high as 76%). As such, it would be estimated that using highly sensitive methods such as RT2-PCR an outbreak will be identified with greater than 98% certainty with the submission of five stool specimens during an outbreak investigation, even if only 50% of specimens contain detectable norovirus. With slightly less sensitive but more specific test methods such as EIA, similar projections are generated (Figures 3A and 3B).


Of gastro and the gold standard: evaluation and policy implications of norovirus test performance for outbreak detection.

Fisman DN, Greer AL, Brouhanski G, Drews SJ - J Transl Med (2009)

Probability of One or More Positive Test Results by Specimens Tested, Under Varying Assumptions Regarding Proportion of True Positive Specimens. Curves are constructed based on a binomial distribution. Each contour represents a different proportion of true positive test specimens. Graph (A) represents estimates generated based on high (100%) sensitivity estimated for real-time reverse-transcriptase polymerase chain reaction using both latent class modeling (LCM) and composite reference standard (CRS) methods. Graph (B) presents estimates generated using LCM estimates for enzyme immunoassay (EIA) sensitivity (86%). A graph using EIA sensitivity estimates from CRS would be similar to graph (A) due to high (97%) sensitivity estimates using the latter approach.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2667494&req=5

Figure 3: Probability of One or More Positive Test Results by Specimens Tested, Under Varying Assumptions Regarding Proportion of True Positive Specimens. Curves are constructed based on a binomial distribution. Each contour represents a different proportion of true positive test specimens. Graph (A) represents estimates generated based on high (100%) sensitivity estimated for real-time reverse-transcriptase polymerase chain reaction using both latent class modeling (LCM) and composite reference standard (CRS) methods. Graph (B) presents estimates generated using LCM estimates for enzyme immunoassay (EIA) sensitivity (86%). A graph using EIA sensitivity estimates from CRS would be similar to graph (A) due to high (97%) sensitivity estimates using the latter approach.
Mentions: We assessed the likelihood that an individual specimen contained NVG material by comparing submitted specimen numbers in identified outbreaks to the number of specimens testing positive by RT2-PCR in those same outbreaks (Table 3). Depending on the presence or absence of EM confirmation of a given outbreak, the proportion of specimens testing positive in apparent outbreaks varied from approximately 58–72% (with 95% confidence intervals as low as 54% and as high as 76%). As such, it would be estimated that using highly sensitive methods such as RT2-PCR an outbreak will be identified with greater than 98% certainty with the submission of five stool specimens during an outbreak investigation, even if only 50% of specimens contain detectable norovirus. With slightly less sensitive but more specific test methods such as EIA, similar projections are generated (Figures 3A and 3B).

Bottom Line: If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested.Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

View Article: PubMed Central - HTML - PubMed

Affiliation: Division of Epidemiology and Surveillance, Ontario Agency for Health Protection and Promotion, Toronto, Canada. david.fisman@gmail.com

ABSTRACT

Background: The norovirus group (NVG) of caliciviruses are the etiological agents of most institutional outbreaks of gastroenteritis in North America and Europe. Identification of NVG is complicated by the non-culturable nature of this virus, and the absence of a diagnostic gold standard makes traditional evaluation of test characteristics problematic.

Methods: We evaluated 189 specimens derived from 440 acute gastroenteritis outbreaks investigated in Ontario in 2006-07. Parallel testing for NVG was performed with real-time reverse-transcriptase polymerase chain reaction (RT2-PCR), enzyme immunoassay (EIA) and electron microscopy (EM). Test characteristics (sensitivity and specificity) were estimated using latent class models and composite reference standard methods. The practical implications of test characteristics were evaluated using binomial probability models.

Results: Latent class modelling estimated sensitivities of RT2-PCR, EIA, and EM as 100%, 86%, and 17% respectively; specificities were 84%, 92%, and 100%; estimates obtained using a composite reference standard were similar. If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested. Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.

Conclusion: Our findings support the characterization of EM as lacking sensitivity for NVG outbreaks. The high sensitivity of RT2-PCR and EIA permit identification of NVG outbreaks with testing of limited numbers of clinical specimens. Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Show MeSH
Related in: MedlinePlus