Limits...
Optimizing the use of expert panel reference diagnoses in diagnostic studies of multidimensional syndromes.

Handels RL, Wolfs CA, Aalten P, Bossuyt PM, Joore MA, Leentjens AF, Severens JL, Verhey FR - BMC Neurol (2014)

Bottom Line: This did not increase after the consensus discussion meeting.The process evaluation led to several recommendations for improvement of the protocol.A protocol for attaining a reference diagnosis based on expert panel consensus was shown feasible in research practice.

View Article: PubMed Central - PubMed

ABSTRACT

Background: In the absence of a gold standard, a panel of experts can be invited to assign a reference diagnosis for use in research. Available literature offers limited guidance on assembling and working with an expert panel for this purpose. We aimed to develop a protocol for an expert panel consensus diagnosis and evaluated its applicability in a pilot project.

Methods: An adjusted Delphi method was used, which started with the assessment of clinical vignettes by 3 experts individually, followed by a consensus discussion meeting to solve diagnostic discrepancies. A panel facilitator ensured that all experts were able to express their views, and encouraged the use of argumentation to arrive at a specific diagnosis, until consensus was reached by all experts. Eleven vignettes of patients suspected of having a primary neurodegenerative disease were presented to the experts. Clinical information was provided stepwise and included medical history, neurological, physical and cognitive function, brain MRI scan, and follow-up assessments over 2 years. After the consensus discussion meeting, the procedure was evaluated by the experts.

Results: The average degree of consensus for the reference diagnosis increased from 52% after individual assessment of the vignettes to 94% after the consensus discussion meeting. Average confidence in the diagnosis after individual assessment was 85%. This did not increase after the consensus discussion meeting. The process evaluation led to several recommendations for improvement of the protocol.

Conclusion: A protocol for attaining a reference diagnosis based on expert panel consensus was shown feasible in research practice.

No MeSH data available.


Related in: MedlinePlus

Single panel approach (option 1 and 2) and partly independent approaches (option 3) to evaluate diagnostic tests for AD.* = possible diagnostic review bias. † = possible test review bias.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4195860&req=5

Fig2: Single panel approach (option 1 and 2) and partly independent approaches (option 3) to evaluate diagnostic tests for AD.* = possible diagnostic review bias. † = possible test review bias.

Mentions: Review bias may occur in two forms. Test review bias occurs when the reference diagnosis is known while either the biomarker-driven or care-as-usual diagnosis is being set. Vice versa, when either the biomarker-driven diagnosis or care-as-usual diagnosis is known while the reference diagnosis is being set it is called diagnostic review bias [39]. Options 1 and 2 from Figure 2 graphically represent both situations. In an ideal situation, all three diagnoses are assessed by three independent expert panels. Due to limited time and resources a decision rule could be applied that combines the care-as-usual diagnosis with the patient’s biomarker profile using pre-defined cut-off values in a decision rule (see Figure 2 option 3 for an overview) [2]. From our pilot evaluation questionnaire (question 16 of Additional file 2) the experts rated potential review bias an average of 5.7 on a scale of 0 to 10.Figure 2


Optimizing the use of expert panel reference diagnoses in diagnostic studies of multidimensional syndromes.

Handels RL, Wolfs CA, Aalten P, Bossuyt PM, Joore MA, Leentjens AF, Severens JL, Verhey FR - BMC Neurol (2014)

Single panel approach (option 1 and 2) and partly independent approaches (option 3) to evaluate diagnostic tests for AD.* = possible diagnostic review bias. † = possible test review bias.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4195860&req=5

Fig2: Single panel approach (option 1 and 2) and partly independent approaches (option 3) to evaluate diagnostic tests for AD.* = possible diagnostic review bias. † = possible test review bias.
Mentions: Review bias may occur in two forms. Test review bias occurs when the reference diagnosis is known while either the biomarker-driven or care-as-usual diagnosis is being set. Vice versa, when either the biomarker-driven diagnosis or care-as-usual diagnosis is known while the reference diagnosis is being set it is called diagnostic review bias [39]. Options 1 and 2 from Figure 2 graphically represent both situations. In an ideal situation, all three diagnoses are assessed by three independent expert panels. Due to limited time and resources a decision rule could be applied that combines the care-as-usual diagnosis with the patient’s biomarker profile using pre-defined cut-off values in a decision rule (see Figure 2 option 3 for an overview) [2]. From our pilot evaluation questionnaire (question 16 of Additional file 2) the experts rated potential review bias an average of 5.7 on a scale of 0 to 10.Figure 2

Bottom Line: This did not increase after the consensus discussion meeting.The process evaluation led to several recommendations for improvement of the protocol.A protocol for attaining a reference diagnosis based on expert panel consensus was shown feasible in research practice.

View Article: PubMed Central - PubMed

ABSTRACT

Background: In the absence of a gold standard, a panel of experts can be invited to assign a reference diagnosis for use in research. Available literature offers limited guidance on assembling and working with an expert panel for this purpose. We aimed to develop a protocol for an expert panel consensus diagnosis and evaluated its applicability in a pilot project.

Methods: An adjusted Delphi method was used, which started with the assessment of clinical vignettes by 3 experts individually, followed by a consensus discussion meeting to solve diagnostic discrepancies. A panel facilitator ensured that all experts were able to express their views, and encouraged the use of argumentation to arrive at a specific diagnosis, until consensus was reached by all experts. Eleven vignettes of patients suspected of having a primary neurodegenerative disease were presented to the experts. Clinical information was provided stepwise and included medical history, neurological, physical and cognitive function, brain MRI scan, and follow-up assessments over 2 years. After the consensus discussion meeting, the procedure was evaluated by the experts.

Results: The average degree of consensus for the reference diagnosis increased from 52% after individual assessment of the vignettes to 94% after the consensus discussion meeting. Average confidence in the diagnosis after individual assessment was 85%. This did not increase after the consensus discussion meeting. The process evaluation led to several recommendations for improvement of the protocol.

Conclusion: A protocol for attaining a reference diagnosis based on expert panel consensus was shown feasible in research practice.

No MeSH data available.


Related in: MedlinePlus