Limits...
Performance map of a cluster detection test using extended power.

Guttmann A, Ouchchane L, Li X, Perthus I, Gaudart J, Demongeot J, Boire JY - Int J Health Geogr (2013)

Bottom Line: Consistent with previous studies, the performance of the spatial scan statistic increased with the baseline incidence of disease, the size of the at-risk population and the strength of the cluster (i.e., the relative risk).Performance was heterogeneous, however, even for very similar clusters (i.e., similar with respect to the aforementioned factors), suggesting the influence of other factors.The performance map we propose enables epidemiologists to assess cluster detection tests across an entire study region.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Biostatistics, Medical Informatics and Communication Technologies, Clermont University Hospital, Clermont-Ferrand F-63000, France. aline.guttmann@udamail.fr.

ABSTRACT

Background: Conventional power studies possess limited ability to assess the performance of cluster detection tests. In particular, they cannot evaluate the accuracy of the cluster location, which is essential in such assessments. Furthermore, they usually estimate power for one or a few particular alternative hypotheses and thus cannot assess performance over an entire region. Takahashi and Tango developed the concept of extended power that indicates both the rate of hypothesis rejection and the accuracy of the cluster location. We propose a systematic assessment method, using here extended power, to produce a map showing the performance of cluster detection tests over an entire region.

Methods: To explore the behavior of a cluster detection test on identical cluster types at any possible location, we successively applied four different spatial and epidemiological parameters. These parameters determined four cluster collections, each covering the entire study region. We simulated 1,000 datasets for each cluster and analyzed them with Kulldorff's spatial scan statistic. From the area under the extended power curve, we constructed a map for each parameter set showing the performance of the test across the entire region.

Results: Consistent with previous studies, the performance of the spatial scan statistic increased with the baseline incidence of disease, the size of the at-risk population and the strength of the cluster (i.e., the relative risk). Performance was heterogeneous, however, even for very similar clusters (i.e., similar with respect to the aforementioned factors), suggesting the influence of other factors.

Conclusions: The area under the extended power curve is a single measure of performance and, although needing further exploration, it is suitable to conduct a systematic spatial evaluation of performance. The performance map we propose enables epidemiologists to assess cluster detection tests across an entire study region.

Show MeSH

Related in: MedlinePlus

AUCEP of Kulldorff’s spatial scan and locations of three simulated clusters for four combinations of two relative risk (RR) and two annual incidence of birth defects: low RR = 3 and high RR = 6; low incidence = 0.48% births and high incidence = 2.26% births.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4016504&req=5

Figure 4: AUCEP of Kulldorff’s spatial scan and locations of three simulated clusters for four combinations of two relative risk (RR) and two annual incidence of birth defects: low RR = 3 and high RR = 6; low incidence = 0.48% births and high incidence = 2.26% births.

Mentions: CDT performance increased monotonically with the at-risk population size (Figure 3). We noted a stronger heterogeneity of CDT performance for the clusters with the largest populations, especially at intermediate risk levels (Figure 3); by this, we mean that clusters with nearly the same population size led to slightly different test performance behaviors. For example, Figure 4 shows test performance in detecting three clusters centered on SUs “43770” (red cluster in the figure), “03700” (blue cluster) and “03420” (green cluster), which had population sizes of 544, 558 and 545 births (mean number over 8 years), respectively. At the lowest risk level, the red cluster was the only one even marginally detected, whereas under other configurations, the blue cluster was best detected. The worst detection performance was exhibited with respect to the green cluster, particularly at intermediate risk levels. We note that the green cluster was the only borderland cluster.


Performance map of a cluster detection test using extended power.

Guttmann A, Ouchchane L, Li X, Perthus I, Gaudart J, Demongeot J, Boire JY - Int J Health Geogr (2013)

AUCEP of Kulldorff’s spatial scan and locations of three simulated clusters for four combinations of two relative risk (RR) and two annual incidence of birth defects: low RR = 3 and high RR = 6; low incidence = 0.48% births and high incidence = 2.26% births.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4016504&req=5

Figure 4: AUCEP of Kulldorff’s spatial scan and locations of three simulated clusters for four combinations of two relative risk (RR) and two annual incidence of birth defects: low RR = 3 and high RR = 6; low incidence = 0.48% births and high incidence = 2.26% births.
Mentions: CDT performance increased monotonically with the at-risk population size (Figure 3). We noted a stronger heterogeneity of CDT performance for the clusters with the largest populations, especially at intermediate risk levels (Figure 3); by this, we mean that clusters with nearly the same population size led to slightly different test performance behaviors. For example, Figure 4 shows test performance in detecting three clusters centered on SUs “43770” (red cluster in the figure), “03700” (blue cluster) and “03420” (green cluster), which had population sizes of 544, 558 and 545 births (mean number over 8 years), respectively. At the lowest risk level, the red cluster was the only one even marginally detected, whereas under other configurations, the blue cluster was best detected. The worst detection performance was exhibited with respect to the green cluster, particularly at intermediate risk levels. We note that the green cluster was the only borderland cluster.

Bottom Line: Consistent with previous studies, the performance of the spatial scan statistic increased with the baseline incidence of disease, the size of the at-risk population and the strength of the cluster (i.e., the relative risk).Performance was heterogeneous, however, even for very similar clusters (i.e., similar with respect to the aforementioned factors), suggesting the influence of other factors.The performance map we propose enables epidemiologists to assess cluster detection tests across an entire study region.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Biostatistics, Medical Informatics and Communication Technologies, Clermont University Hospital, Clermont-Ferrand F-63000, France. aline.guttmann@udamail.fr.

ABSTRACT

Background: Conventional power studies possess limited ability to assess the performance of cluster detection tests. In particular, they cannot evaluate the accuracy of the cluster location, which is essential in such assessments. Furthermore, they usually estimate power for one or a few particular alternative hypotheses and thus cannot assess performance over an entire region. Takahashi and Tango developed the concept of extended power that indicates both the rate of hypothesis rejection and the accuracy of the cluster location. We propose a systematic assessment method, using here extended power, to produce a map showing the performance of cluster detection tests over an entire region.

Methods: To explore the behavior of a cluster detection test on identical cluster types at any possible location, we successively applied four different spatial and epidemiological parameters. These parameters determined four cluster collections, each covering the entire study region. We simulated 1,000 datasets for each cluster and analyzed them with Kulldorff's spatial scan statistic. From the area under the extended power curve, we constructed a map for each parameter set showing the performance of the test across the entire region.

Results: Consistent with previous studies, the performance of the spatial scan statistic increased with the baseline incidence of disease, the size of the at-risk population and the strength of the cluster (i.e., the relative risk). Performance was heterogeneous, however, even for very similar clusters (i.e., similar with respect to the aforementioned factors), suggesting the influence of other factors.

Conclusions: The area under the extended power curve is a single measure of performance and, although needing further exploration, it is suitable to conduct a systematic spatial evaluation of performance. The performance map we propose enables epidemiologists to assess cluster detection tests across an entire study region.

Show MeSH
Related in: MedlinePlus