Limits...
Development and application of a novel metric to assess effectiveness of biomedical data.

Bloom GC, Eschrich S, Han G, Hang G, Schabath MB, Bhansali N, Hoerter AM, Morgan S, Fenstermacher DA - BMJ Open (2013)

Bottom Line: An effectiveness or E-score was obtained by calculating the conditional probabilities of the p-value and A-score within the given data set with their product equaling the effectiveness score (E-score).Conversely, elements surgery-site, histologic-type and pathological-TNM stage were down-ranked in comparison to their p values based on lower A-scores caused by significantly higher acquisition costs.Results show that an element's underlying data quality is an important consideration in addition to p value correlation to outcome when determining the element's clinical or research utility in a study.

View Article: PubMed Central - PubMed

Affiliation: Department of Biomedical Informatics, H Lee Moffitt Cancer and Research Institute, Tampa, Florida, USA.

ABSTRACT

Objective: Design a metric to assess the comparative effectiveness of biomedical data elements within a study that incorporates their statistical relatedness to a given outcome variable as well as a measurement of the quality of their underlying data.

Materials and methods: The cohort consisted of 874 patients with adenocarcinoma of the lung, each with 47 clinical data elements. The p value for each element was calculated using the Cox proportional hazard univariable regression model with overall survival as the endpoint. An attribute or A-score was calculated by quantification of an element's four quality attributes; Completeness, Comprehensiveness, Consistency and Overall-cost. An effectiveness or E-score was obtained by calculating the conditional probabilities of the p-value and A-score within the given data set with their product equaling the effectiveness score (E-score).

Results: The E-score metric provided information about the utility of an element beyond an outcome-related p value ranking. E-scores for elements age-at-diagnosis, gender and tobacco-use showed utility above what their respective p values alone would indicate due to their relative ease of acquisition, that is, higher A-scores. Conversely, elements surgery-site, histologic-type and pathological-TNM stage were down-ranked in comparison to their p values based on lower A-scores caused by significantly higher acquisition costs.

Conclusions: A novel metric termed E-score was developed which incorporates standard statistics with data quality metrics and was tested on elements from a large lung cohort. Results show that an element's underlying data quality is an important consideration in addition to p value correlation to outcome when determining the element's clinical or research utility in a study.

No MeSH data available.


Related in: MedlinePlus

Graph of the five survey-type elements showing changes in effectiveness-score with varying attribute-score.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3753524&req=5

BMJOPEN2013003220F3: Graph of the five survey-type elements showing changes in effectiveness-score with varying attribute-score.

Mentions: We wanted to test the behaviour of the E-score metric for a set of procedure elements with varying A-scores. FigureĀ 3 is a graph of the E-score values for four selected procedure elements for a series of 10 simulated A-scores between 10 and 100. As in figure 2 the actual p value calculated from the data was used here (see table 2 for the p values of each element) and only the A-score was simulated.


Development and application of a novel metric to assess effectiveness of biomedical data.

Bloom GC, Eschrich S, Han G, Hang G, Schabath MB, Bhansali N, Hoerter AM, Morgan S, Fenstermacher DA - BMJ Open (2013)

Graph of the five survey-type elements showing changes in effectiveness-score with varying attribute-score.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3753524&req=5

BMJOPEN2013003220F3: Graph of the five survey-type elements showing changes in effectiveness-score with varying attribute-score.
Mentions: We wanted to test the behaviour of the E-score metric for a set of procedure elements with varying A-scores. FigureĀ 3 is a graph of the E-score values for four selected procedure elements for a series of 10 simulated A-scores between 10 and 100. As in figure 2 the actual p value calculated from the data was used here (see table 2 for the p values of each element) and only the A-score was simulated.

Bottom Line: An effectiveness or E-score was obtained by calculating the conditional probabilities of the p-value and A-score within the given data set with their product equaling the effectiveness score (E-score).Conversely, elements surgery-site, histologic-type and pathological-TNM stage were down-ranked in comparison to their p values based on lower A-scores caused by significantly higher acquisition costs.Results show that an element's underlying data quality is an important consideration in addition to p value correlation to outcome when determining the element's clinical or research utility in a study.

View Article: PubMed Central - PubMed

Affiliation: Department of Biomedical Informatics, H Lee Moffitt Cancer and Research Institute, Tampa, Florida, USA.

ABSTRACT

Objective: Design a metric to assess the comparative effectiveness of biomedical data elements within a study that incorporates their statistical relatedness to a given outcome variable as well as a measurement of the quality of their underlying data.

Materials and methods: The cohort consisted of 874 patients with adenocarcinoma of the lung, each with 47 clinical data elements. The p value for each element was calculated using the Cox proportional hazard univariable regression model with overall survival as the endpoint. An attribute or A-score was calculated by quantification of an element's four quality attributes; Completeness, Comprehensiveness, Consistency and Overall-cost. An effectiveness or E-score was obtained by calculating the conditional probabilities of the p-value and A-score within the given data set with their product equaling the effectiveness score (E-score).

Results: The E-score metric provided information about the utility of an element beyond an outcome-related p value ranking. E-scores for elements age-at-diagnosis, gender and tobacco-use showed utility above what their respective p values alone would indicate due to their relative ease of acquisition, that is, higher A-scores. Conversely, elements surgery-site, histologic-type and pathological-TNM stage were down-ranked in comparison to their p values based on lower A-scores caused by significantly higher acquisition costs.

Conclusions: A novel metric termed E-score was developed which incorporates standard statistics with data quality metrics and was tested on elements from a large lung cohort. Results show that an element's underlying data quality is an important consideration in addition to p value correlation to outcome when determining the element's clinical or research utility in a study.

No MeSH data available.


Related in: MedlinePlus