Limits...
When should potentially false research findings be considered acceptable?

Djulbegovic B, Hozo I - PLoS Med. (2007)

Bottom Line: Ioannidis estimated that most published research findings are false, but he did not indicate when, if at all, potentially false research results may be considered as acceptable to society.A new model indicates that the probability above which research results should be accepted depends on the expected payback from the research (the benefits) and the inadvertent consequences (the harms).Our acceptance of research findings changes as a function of what we call "acceptable regret," i.e., our tolerance of making a wrong decision in accepting the research hypothesis.

View Article: PubMed Central - PubMed

Affiliation: Department of Interdisciplinary Oncology, H. Lee Moffitt Cancer Center and Research Institute, University of South Florida, Tampa, Florida, United States of America. Benjamin.Djulbegovic@moffitt.org

ABSTRACT
Ioannidis estimated that most published research findings are false, but he did not indicate when, if at all, potentially false research results may be considered as acceptable to society. We combined our two previously published models to calculate the probability above which research findings may become acceptable. A new model indicates that the probability above which research results should be accepted depends on the expected payback from the research (the benefits) and the inadvertent consequences (the harms). This probability may dramatically change depending on our willingness to tolerate error in accepting false research findings. Our acceptance of research findings changes as a function of what we call "acceptable regret," i.e., our tolerance of making a wrong decision in accepting the research hypothesis. We illustrate our findings by providing a new framework for early stopping rules in clinical research (i.e., when should we accept early findings from a clinical trial indicating the benefits as true?). Obtaining absolute "truth" in research is impossible, and so society has to decide when less-than-perfect results may become acceptable.

Show MeSH
The Threshold Probability Above (Pt in Red) Which We Should Accept Findings of Research Hypothesis as Being TrueThe horizontal yellow line indicates the actual conditional probability that the research hypothesis is true in the case of positive findings. This means that for benefit/harm ratios above the threshold (1.5 in this example), the research hypothesis can be accepted.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC1808081&req=5

pmed-0040026-g001: The Threshold Probability Above (Pt in Red) Which We Should Accept Findings of Research Hypothesis as Being TrueThe horizontal yellow line indicates the actual conditional probability that the research hypothesis is true in the case of positive findings. This means that for benefit/harm ratios above the threshold (1.5 in this example), the research hypothesis can be accepted.

Mentions: Figure 1 shows the threshold probability of “truth” (i.e., the probability above which the research findings may be accepted) as a function of B/H associated with the research results. The graph shows that as long as the probability of “accepted truth” (a horizontal line) is above the threshold probability curve, the research findings may be accepted. The higher the B/H ratio, the less certain we need to be of the truthfulness of the research results in order to accept them.


When should potentially false research findings be considered acceptable?

Djulbegovic B, Hozo I - PLoS Med. (2007)

The Threshold Probability Above (Pt in Red) Which We Should Accept Findings of Research Hypothesis as Being TrueThe horizontal yellow line indicates the actual conditional probability that the research hypothesis is true in the case of positive findings. This means that for benefit/harm ratios above the threshold (1.5 in this example), the research hypothesis can be accepted.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC1808081&req=5

pmed-0040026-g001: The Threshold Probability Above (Pt in Red) Which We Should Accept Findings of Research Hypothesis as Being TrueThe horizontal yellow line indicates the actual conditional probability that the research hypothesis is true in the case of positive findings. This means that for benefit/harm ratios above the threshold (1.5 in this example), the research hypothesis can be accepted.
Mentions: Figure 1 shows the threshold probability of “truth” (i.e., the probability above which the research findings may be accepted) as a function of B/H associated with the research results. The graph shows that as long as the probability of “accepted truth” (a horizontal line) is above the threshold probability curve, the research findings may be accepted. The higher the B/H ratio, the less certain we need to be of the truthfulness of the research results in order to accept them.

Bottom Line: Ioannidis estimated that most published research findings are false, but he did not indicate when, if at all, potentially false research results may be considered as acceptable to society.A new model indicates that the probability above which research results should be accepted depends on the expected payback from the research (the benefits) and the inadvertent consequences (the harms).Our acceptance of research findings changes as a function of what we call "acceptable regret," i.e., our tolerance of making a wrong decision in accepting the research hypothesis.

View Article: PubMed Central - PubMed

Affiliation: Department of Interdisciplinary Oncology, H. Lee Moffitt Cancer Center and Research Institute, University of South Florida, Tampa, Florida, United States of America. Benjamin.Djulbegovic@moffitt.org

ABSTRACT
Ioannidis estimated that most published research findings are false, but he did not indicate when, if at all, potentially false research results may be considered as acceptable to society. We combined our two previously published models to calculate the probability above which research findings may become acceptable. A new model indicates that the probability above which research results should be accepted depends on the expected payback from the research (the benefits) and the inadvertent consequences (the harms). This probability may dramatically change depending on our willingness to tolerate error in accepting false research findings. Our acceptance of research findings changes as a function of what we call "acceptable regret," i.e., our tolerance of making a wrong decision in accepting the research hypothesis. We illustrate our findings by providing a new framework for early stopping rules in clinical research (i.e., when should we accept early findings from a clinical trial indicating the benefits as true?). Obtaining absolute "truth" in research is impossible, and so society has to decide when less-than-perfect results may become acceptable.

Show MeSH