Limits...
The most dangerous hospital or the most dangerous equation?

Tu YK, Gilthorpe MS - BMC Health Serv Res (2007)

Bottom Line: A close examination of the information reveals a pattern which is consistent with a statistical phenomenon, discovered by the French mathematician de Moivre nearly 300 years ago, described in every introductory statistics textbook: namely that variation in performance indicators is expected to be greater in small Trusts and smaller in large Trusts.Therefore, it is not surprising to note that small hospitals are more likely to occur at the top and the bottom of league tables, whilst mortality rates are independent of hospital sizes.This statistical phenomenon needs to be taken into account in the comparison of hospital Trusts performance, especially with regard to policy decisions.

View Article: PubMed Central - HTML - PubMed

Affiliation: Biostatistics Unit, Centre for Epidemiology and Biostatistics, University of Leeds, 30/32 Hyde Terrace, Leeds, LS2 9LN, UK. y.k.tu@leeds.ac.uk

ABSTRACT

Background: Hospital mortality rates are one of the most frequently selected indicators for measuring the performance of NHS Trusts. A recent article in a national newspaper named the hospital with the highest or lowest mortality in the 2005/6 financial year; a report by the organization Dr Foster Intelligence provided information with regard to the performance of all NHS Trusts in England.

Methods: Basic statistical theory and computer simulations were used to explore the relationship between the variations in the performance of NHS Trusts and the sizes of the Trusts. Data of hospital standardised mortality ratio (HSMR) of 152 English NHS Trusts for 2005/6 were re-analysed.

Results: A close examination of the information reveals a pattern which is consistent with a statistical phenomenon, discovered by the French mathematician de Moivre nearly 300 years ago, described in every introductory statistics textbook: namely that variation in performance indicators is expected to be greater in small Trusts and smaller in large Trusts. From a statistical viewpoint, the number of deaths in a hospital is not in proportion to the size of the hospital, but is proportional to the square root of its size. Therefore, it is not surprising to note that small hospitals are more likely to occur at the top and the bottom of league tables, whilst mortality rates are independent of hospital sizes.

Conclusion: This statistical phenomenon needs to be taken into account in the comparison of hospital Trusts performance, especially with regard to policy decisions.

Show MeSH
A funnel plot of the relationship between the expected number of events and the ratio of observed to expected number of events in the simulated dataset of 1,000 hospitals. The blue lines (top and the bottom of the panel) represent respectively the upper and lower 95% confidence limits of the observed/expected ratio.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2194686&req=5

Figure 1: A funnel plot of the relationship between the expected number of events and the ratio of observed to expected number of events in the simulated dataset of 1,000 hospitals. The blue lines (top and the bottom of the panel) represent respectively the upper and lower 95% confidence limits of the observed/expected ratio.

Mentions: A funnel plot in Figure 1 shows a simulated dataset of 1,000 hospitals in which the number of surgical procedures (Y) in each year has a mean of 2,500 and a standard deviation (SD) of 700. The mortality rate is assumed to be 5% across all sizes of hospital, so the number of expected deaths (X) has a mean of 125 and SD of 35. A random variable with zero mean and SD proportional to the square root of X is simulated to represent the variation/fluctuation in the observed mean number of deaths, and this is added to X. The vertical axis in Figure 1 is the ratio of observed number of deaths (Z) over the expected number of deaths (X), and the horizontal axis is X. If we fit a linear regression model to the data, the regression slope will be close to zero, indicating that the observed to expected ratio is independent of the expected number of deaths (i.e. hospital size), yet variation in the 95% confidence interval of these ratios (represented by the blue lines both top and the bottom of the figure) is inversely related to hospital size. Although this simulation assumes no relationship between mortality and the number of surgeries undertaken, a few hospitals are below the lower confidence limit or above the upper confidence limit, as would be expected due to chance alone 5% of the time, indicating that their performance is either alarmingly poor or extremely good. We would therefore still need to be cautious in identifying the poor or good performers using funnel plots or quality control charts, given that chance is involved. In the report published in the "How healthy is your hospital?" readers can find that the report's graphs follow a very similar pattern [2].


The most dangerous hospital or the most dangerous equation?

Tu YK, Gilthorpe MS - BMC Health Serv Res (2007)

A funnel plot of the relationship between the expected number of events and the ratio of observed to expected number of events in the simulated dataset of 1,000 hospitals. The blue lines (top and the bottom of the panel) represent respectively the upper and lower 95% confidence limits of the observed/expected ratio.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2194686&req=5

Figure 1: A funnel plot of the relationship between the expected number of events and the ratio of observed to expected number of events in the simulated dataset of 1,000 hospitals. The blue lines (top and the bottom of the panel) represent respectively the upper and lower 95% confidence limits of the observed/expected ratio.
Mentions: A funnel plot in Figure 1 shows a simulated dataset of 1,000 hospitals in which the number of surgical procedures (Y) in each year has a mean of 2,500 and a standard deviation (SD) of 700. The mortality rate is assumed to be 5% across all sizes of hospital, so the number of expected deaths (X) has a mean of 125 and SD of 35. A random variable with zero mean and SD proportional to the square root of X is simulated to represent the variation/fluctuation in the observed mean number of deaths, and this is added to X. The vertical axis in Figure 1 is the ratio of observed number of deaths (Z) over the expected number of deaths (X), and the horizontal axis is X. If we fit a linear regression model to the data, the regression slope will be close to zero, indicating that the observed to expected ratio is independent of the expected number of deaths (i.e. hospital size), yet variation in the 95% confidence interval of these ratios (represented by the blue lines both top and the bottom of the figure) is inversely related to hospital size. Although this simulation assumes no relationship between mortality and the number of surgeries undertaken, a few hospitals are below the lower confidence limit or above the upper confidence limit, as would be expected due to chance alone 5% of the time, indicating that their performance is either alarmingly poor or extremely good. We would therefore still need to be cautious in identifying the poor or good performers using funnel plots or quality control charts, given that chance is involved. In the report published in the "How healthy is your hospital?" readers can find that the report's graphs follow a very similar pattern [2].

Bottom Line: A close examination of the information reveals a pattern which is consistent with a statistical phenomenon, discovered by the French mathematician de Moivre nearly 300 years ago, described in every introductory statistics textbook: namely that variation in performance indicators is expected to be greater in small Trusts and smaller in large Trusts.Therefore, it is not surprising to note that small hospitals are more likely to occur at the top and the bottom of league tables, whilst mortality rates are independent of hospital sizes.This statistical phenomenon needs to be taken into account in the comparison of hospital Trusts performance, especially with regard to policy decisions.

View Article: PubMed Central - HTML - PubMed

Affiliation: Biostatistics Unit, Centre for Epidemiology and Biostatistics, University of Leeds, 30/32 Hyde Terrace, Leeds, LS2 9LN, UK. y.k.tu@leeds.ac.uk

ABSTRACT

Background: Hospital mortality rates are one of the most frequently selected indicators for measuring the performance of NHS Trusts. A recent article in a national newspaper named the hospital with the highest or lowest mortality in the 2005/6 financial year; a report by the organization Dr Foster Intelligence provided information with regard to the performance of all NHS Trusts in England.

Methods: Basic statistical theory and computer simulations were used to explore the relationship between the variations in the performance of NHS Trusts and the sizes of the Trusts. Data of hospital standardised mortality ratio (HSMR) of 152 English NHS Trusts for 2005/6 were re-analysed.

Results: A close examination of the information reveals a pattern which is consistent with a statistical phenomenon, discovered by the French mathematician de Moivre nearly 300 years ago, described in every introductory statistics textbook: namely that variation in performance indicators is expected to be greater in small Trusts and smaller in large Trusts. From a statistical viewpoint, the number of deaths in a hospital is not in proportion to the size of the hospital, but is proportional to the square root of its size. Therefore, it is not surprising to note that small hospitals are more likely to occur at the top and the bottom of league tables, whilst mortality rates are independent of hospital sizes.

Conclusion: This statistical phenomenon needs to be taken into account in the comparison of hospital Trusts performance, especially with regard to policy decisions.

Show MeSH