Limits...
Concordance and predictive value of two adverse drug event data sets.

Cami A, Reis BY - BMC Med Inform Decis Mak (2014)

Bottom Line: A minor difference of 1.1% was found in the AUROC of PPN when SIDER 2012 was used for validation instead of Lexi-comp 2010.In conclusion, the ADE and drug counts in Lexi-comp and SIDER data sets were highly correlated and the choice of validation set did not greatly affect the overall prediction performance of PPN.Our results also suggest that it is important to be aware of the differences that exist among ADE data sets, especially in modeling applications focused on specific drug and ADE categories.

View Article: PubMed Central - HTML - PubMed

Affiliation: Division of Emergency Medicine, Boston Children's Hospital, 1 Autumn Street, 5th Floor, Boston, MA 02215, USA. aurel.cami@childrens.harvard.edu.

ABSTRACT

Background: Accurate prediction of adverse drug events (ADEs) is an important means of controlling and reducing drug-related morbidity and mortality. Since no single "gold standard" ADE data set exists, a range of different drug safety data sets are currently used for developing ADE prediction models. There is a critical need to assess the degree of concordance between these various ADE data sets and to validate ADE prediction models against multiple reference standards.

Methods: We systematically evaluated the concordance of two widely used ADE data sets - Lexi-comp from 2010 and SIDER from 2012. The strength of the association between ADE (drug) counts in Lexi-comp and SIDER was assessed using Spearman rank correlation, while the differences between the two data sets were characterized in terms of drug categories, ADE categories and ADE frequencies. We also performed a comparative validation of the Predictive Pharmacosafety Networks (PPN) model using both ADE data sets. The predictive power of PPN using each of the two validation sets was assessed using the area under Receiver Operating Characteristic curve (AUROC).

Results: The correlations between the counts of ADEs and drugs in the two data sets were 0.84 (95% CI: 0.82-0.86) and 0.92 (95% CI: 0.91-0.93), respectively. Relative to an earlier snapshot of Lexi-comp from 2005, Lexi-comp 2010 and SIDER 2012 introduced a mean of 1,973 and 4,810 new drug-ADE associations per year, respectively. The difference between these two data sets was most pronounced for Nervous System and Anti-infective drugs, Gastrointestinal and Nervous System ADEs, and postmarketing ADEs. A minor difference of 1.1% was found in the AUROC of PPN when SIDER 2012 was used for validation instead of Lexi-comp 2010.

Conclusions: In conclusion, the ADE and drug counts in Lexi-comp and SIDER data sets were highly correlated and the choice of validation set did not greatly affect the overall prediction performance of PPN. Our results also suggest that it is important to be aware of the differences that exist among ADE data sets, especially in modeling applications focused on specific drug and ADE categories.

Show MeSH

Related in: MedlinePlus

Overview of the study framework. First, data were integrated from multiple sources, including data on drug-ADE associations from two different sources (Lexi-comp and SIDER), drug and ADE taxonomies, and intrinsic drug properties. Next, a number of steps to standardize and integrate these data were carried out. The strength of the association between the counts of ADEs and drugs in Lexi-comp and SIDER data sets and the differences between these two data sets were assessed. Finally, a PPN model was trained using a 2005 version of Lexi-comp and validated using both Lexi-comp 2010 and SIDER 2012 as reference standards.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4150549&req=5

Figure 1: Overview of the study framework. First, data were integrated from multiple sources, including data on drug-ADE associations from two different sources (Lexi-comp and SIDER), drug and ADE taxonomies, and intrinsic drug properties. Next, a number of steps to standardize and integrate these data were carried out. The strength of the association between the counts of ADEs and drugs in Lexi-comp and SIDER data sets and the differences between these two data sets were assessed. Finally, a PPN model was trained using a 2005 version of Lexi-comp and validated using both Lexi-comp 2010 and SIDER 2012 as reference standards.

Mentions: FigureĀ 1 shows an overview of the study framework. First, we integrated data from multiple sources, including data on drug-ADE associations from two snapshots of Lexi-comp and one snapshot of SIDER, drug and ADE taxonomies, and intrinsic drug properties. Next, we carried out a number of steps to standardize and integrate these data, including mapping the Lexi-comp and SIDER ADE names to MedDRA High Level Terms (HLTs), standardizing the drug names in Lexi-comp and SIDER, and constructing bi-partite network representations of the drug-ADE associations. Next, we assessed the strength of the association between the counts of ADEs (per drug) and drugs (per ADE) in Lexi-comp and SIDER data sets. We also identified the difference between the two data sets and characterized it in terms of drug categories, ADE categories and ADE frequencies. Finally, we trained a PPN model using a 2005 version of Lexi-comp data and validated the prediction performance of PPN using both Lexi-comp 2010 and SIDER 2012 as reference standards.


Concordance and predictive value of two adverse drug event data sets.

Cami A, Reis BY - BMC Med Inform Decis Mak (2014)

Overview of the study framework. First, data were integrated from multiple sources, including data on drug-ADE associations from two different sources (Lexi-comp and SIDER), drug and ADE taxonomies, and intrinsic drug properties. Next, a number of steps to standardize and integrate these data were carried out. The strength of the association between the counts of ADEs and drugs in Lexi-comp and SIDER data sets and the differences between these two data sets were assessed. Finally, a PPN model was trained using a 2005 version of Lexi-comp and validated using both Lexi-comp 2010 and SIDER 2012 as reference standards.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4150549&req=5

Figure 1: Overview of the study framework. First, data were integrated from multiple sources, including data on drug-ADE associations from two different sources (Lexi-comp and SIDER), drug and ADE taxonomies, and intrinsic drug properties. Next, a number of steps to standardize and integrate these data were carried out. The strength of the association between the counts of ADEs and drugs in Lexi-comp and SIDER data sets and the differences between these two data sets were assessed. Finally, a PPN model was trained using a 2005 version of Lexi-comp and validated using both Lexi-comp 2010 and SIDER 2012 as reference standards.
Mentions: FigureĀ 1 shows an overview of the study framework. First, we integrated data from multiple sources, including data on drug-ADE associations from two snapshots of Lexi-comp and one snapshot of SIDER, drug and ADE taxonomies, and intrinsic drug properties. Next, we carried out a number of steps to standardize and integrate these data, including mapping the Lexi-comp and SIDER ADE names to MedDRA High Level Terms (HLTs), standardizing the drug names in Lexi-comp and SIDER, and constructing bi-partite network representations of the drug-ADE associations. Next, we assessed the strength of the association between the counts of ADEs (per drug) and drugs (per ADE) in Lexi-comp and SIDER data sets. We also identified the difference between the two data sets and characterized it in terms of drug categories, ADE categories and ADE frequencies. Finally, we trained a PPN model using a 2005 version of Lexi-comp data and validated the prediction performance of PPN using both Lexi-comp 2010 and SIDER 2012 as reference standards.

Bottom Line: A minor difference of 1.1% was found in the AUROC of PPN when SIDER 2012 was used for validation instead of Lexi-comp 2010.In conclusion, the ADE and drug counts in Lexi-comp and SIDER data sets were highly correlated and the choice of validation set did not greatly affect the overall prediction performance of PPN.Our results also suggest that it is important to be aware of the differences that exist among ADE data sets, especially in modeling applications focused on specific drug and ADE categories.

View Article: PubMed Central - HTML - PubMed

Affiliation: Division of Emergency Medicine, Boston Children's Hospital, 1 Autumn Street, 5th Floor, Boston, MA 02215, USA. aurel.cami@childrens.harvard.edu.

ABSTRACT

Background: Accurate prediction of adverse drug events (ADEs) is an important means of controlling and reducing drug-related morbidity and mortality. Since no single "gold standard" ADE data set exists, a range of different drug safety data sets are currently used for developing ADE prediction models. There is a critical need to assess the degree of concordance between these various ADE data sets and to validate ADE prediction models against multiple reference standards.

Methods: We systematically evaluated the concordance of two widely used ADE data sets - Lexi-comp from 2010 and SIDER from 2012. The strength of the association between ADE (drug) counts in Lexi-comp and SIDER was assessed using Spearman rank correlation, while the differences between the two data sets were characterized in terms of drug categories, ADE categories and ADE frequencies. We also performed a comparative validation of the Predictive Pharmacosafety Networks (PPN) model using both ADE data sets. The predictive power of PPN using each of the two validation sets was assessed using the area under Receiver Operating Characteristic curve (AUROC).

Results: The correlations between the counts of ADEs and drugs in the two data sets were 0.84 (95% CI: 0.82-0.86) and 0.92 (95% CI: 0.91-0.93), respectively. Relative to an earlier snapshot of Lexi-comp from 2005, Lexi-comp 2010 and SIDER 2012 introduced a mean of 1,973 and 4,810 new drug-ADE associations per year, respectively. The difference between these two data sets was most pronounced for Nervous System and Anti-infective drugs, Gastrointestinal and Nervous System ADEs, and postmarketing ADEs. A minor difference of 1.1% was found in the AUROC of PPN when SIDER 2012 was used for validation instead of Lexi-comp 2010.

Conclusions: In conclusion, the ADE and drug counts in Lexi-comp and SIDER data sets were highly correlated and the choice of validation set did not greatly affect the overall prediction performance of PPN. Our results also suggest that it is important to be aware of the differences that exist among ADE data sets, especially in modeling applications focused on specific drug and ADE categories.

Show MeSH
Related in: MedlinePlus