Limits...
A systematic review of the quality of homeopathic clinical trials.

Jonas WB, Anderson RL, Crawford CC, Lyons JS - BMC Complement Altern Med (2001)

Bottom Line: Of these, 79% were from peer-reviewed journals, 29% used a placebo control, 51% used random assignment, and 86% failed to consider potentially confounding variables.The main validity problems were in measurement where 96% did not report the proportion of subjects screened, and 64% did not report attrition rate. 17% of subjects dropped out in studies where this was reported.Compared to research on conventional therapies the overall quality of studies in homeopathy was worse and only slightly improved in more recent years.

View Article: PubMed Central - HTML - PubMed

Affiliation: Samueli Institute for Information Biology and Department of Family Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA. wjonas@siib.org

ABSTRACT

Background: While a number of reviews of homeopathic clinical trials have been done, all have used methods dependent on allopathic diagnostic classifications foreign to homeopathic practice. In addition, no review has used established and validated quality criteria allowing direct comparison of the allopathic and homeopathic literature.

Methods: In a systematic review, we compared the quality of clinical-trial research in homeopathy to a sample of research on conventional therapies using a validated and system-neutral approach. All clinical trials on homeopathic treatments with parallel treatment groups published between 1945-1995 in English were selected. All were evaluated with an established set of 33 validity criteria previously validated on a broad range of health interventions across differing medical systems. Criteria covered statistical conclusion, internal, construct and external validity. Reliability of criteria application is greater than 0.95.

Results: 59 studies met the inclusion criteria. Of these, 79% were from peer-reviewed journals, 29% used a placebo control, 51% used random assignment, and 86% failed to consider potentially confounding variables. The main validity problems were in measurement where 96% did not report the proportion of subjects screened, and 64% did not report attrition rate. 17% of subjects dropped out in studies where this was reported. There was practically no replication of or overlap in the conditions studied and most studies were relatively small and done at a single-site. Compared to research on conventional therapies the overall quality of studies in homeopathy was worse and only slightly improved in more recent years.

Conclusions: Clinical homeopathic research is clearly in its infancy with most studies using poor sampling and measurement techniques, few subjects, single sites and no replication. Many of these problems are correctable even within a "holistic" paradigm given sufficient research expertise, support and methods.

Show MeSH
Average % of studies Not Meeting Validity
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC64638&req=5

Figure 1: Average % of studies Not Meeting Validity

Mentions: Significant threats to validity occurred in 30–40% of studies in every major category of validity measured. (Figure 1) Especially high were problems with external validity since most studies had small sample sizes and few were multi-centered. Thus, the results of research in homeopathic medicine is less likely to be generalizable beyond study populations than the results of research in conventional medicine. Surprisingly, internal validity quality scores were very similar between homeopathic and conventional studies, indicating that homeopathic research is not appreciably worse than conventional research when it comes to controlling for bias and systematic error. (Figure 1) Compared to conventional studies using these same quality criteria the homeopathic studies had a higher number of validity threats in 21 categories (63.6%), and the same in 7 (21%) categories. Conventional studies scored worse than homeopathic studies in 5 (15%) categories. How homeopathic and conventional studies compare on all validity threats is shown in Table 3. We were unable to do a subset analysis of conventional studies that used only parallel control groups in order to confirm the independence of validity threats from study design in this sample.


A systematic review of the quality of homeopathic clinical trials.

Jonas WB, Anderson RL, Crawford CC, Lyons JS - BMC Complement Altern Med (2001)

Average % of studies Not Meeting Validity
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC64638&req=5

Figure 1: Average % of studies Not Meeting Validity
Mentions: Significant threats to validity occurred in 30–40% of studies in every major category of validity measured. (Figure 1) Especially high were problems with external validity since most studies had small sample sizes and few were multi-centered. Thus, the results of research in homeopathic medicine is less likely to be generalizable beyond study populations than the results of research in conventional medicine. Surprisingly, internal validity quality scores were very similar between homeopathic and conventional studies, indicating that homeopathic research is not appreciably worse than conventional research when it comes to controlling for bias and systematic error. (Figure 1) Compared to conventional studies using these same quality criteria the homeopathic studies had a higher number of validity threats in 21 categories (63.6%), and the same in 7 (21%) categories. Conventional studies scored worse than homeopathic studies in 5 (15%) categories. How homeopathic and conventional studies compare on all validity threats is shown in Table 3. We were unable to do a subset analysis of conventional studies that used only parallel control groups in order to confirm the independence of validity threats from study design in this sample.

Bottom Line: Of these, 79% were from peer-reviewed journals, 29% used a placebo control, 51% used random assignment, and 86% failed to consider potentially confounding variables.The main validity problems were in measurement where 96% did not report the proportion of subjects screened, and 64% did not report attrition rate. 17% of subjects dropped out in studies where this was reported.Compared to research on conventional therapies the overall quality of studies in homeopathy was worse and only slightly improved in more recent years.

View Article: PubMed Central - HTML - PubMed

Affiliation: Samueli Institute for Information Biology and Department of Family Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA. wjonas@siib.org

ABSTRACT

Background: While a number of reviews of homeopathic clinical trials have been done, all have used methods dependent on allopathic diagnostic classifications foreign to homeopathic practice. In addition, no review has used established and validated quality criteria allowing direct comparison of the allopathic and homeopathic literature.

Methods: In a systematic review, we compared the quality of clinical-trial research in homeopathy to a sample of research on conventional therapies using a validated and system-neutral approach. All clinical trials on homeopathic treatments with parallel treatment groups published between 1945-1995 in English were selected. All were evaluated with an established set of 33 validity criteria previously validated on a broad range of health interventions across differing medical systems. Criteria covered statistical conclusion, internal, construct and external validity. Reliability of criteria application is greater than 0.95.

Results: 59 studies met the inclusion criteria. Of these, 79% were from peer-reviewed journals, 29% used a placebo control, 51% used random assignment, and 86% failed to consider potentially confounding variables. The main validity problems were in measurement where 96% did not report the proportion of subjects screened, and 64% did not report attrition rate. 17% of subjects dropped out in studies where this was reported. There was practically no replication of or overlap in the conditions studied and most studies were relatively small and done at a single-site. Compared to research on conventional therapies the overall quality of studies in homeopathy was worse and only slightly improved in more recent years.

Conclusions: Clinical homeopathic research is clearly in its infancy with most studies using poor sampling and measurement techniques, few subjects, single sites and no replication. Many of these problems are correctable even within a "holistic" paradigm given sufficient research expertise, support and methods.

Show MeSH