Limits...
Enhancing adverse drug event detection in electronic health records using molecular structure similarity: application to pancreatitis.

Vilar S, Harpaz R, Santana L, Uriarte E, Friedman C - PLoS ONE (2012)

Bottom Line: A reference standard of drugs known to be causally associated with the adverse event pancreatitis was used to create a MFBM.Candidate drugs associated with these signals were then assessed using the MFBM to find the most promising candidates based on structural similarity.The method proposed in this paper provides evidence of being a promising adjunct to existing automated ADE detection and analysis approaches.

View Article: PubMed Central - PubMed

Affiliation: Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, United States of America. sav7003@dbmi.columbia.edu

ABSTRACT

Background: Adverse drug events (ADEs) detection and assessment is at the center of pharmacovigilance. Data mining of systems, such as FDA's Adverse Event Reporting System (AERS) and more recently, Electronic Health Records (EHRs), can aid in the automatic detection and analysis of ADEs. Although different data mining approaches have been shown to be valuable, it is still crucial to improve the quality of the generated signals.

Objective: To leverage structural similarity by developing molecular fingerprint-based models (MFBMs) to strengthen ADE signals generated from EHR data.

Methods: A reference standard of drugs known to be causally associated with the adverse event pancreatitis was used to create a MFBM. Electronic Health Records (EHRs) from the New York Presbyterian Hospital were mined to generate structured data. Disproportionality Analysis (DPA) was applied to the data, and 278 possible signals related to the ADE pancreatitis were detected. Candidate drugs associated with these signals were then assessed using the MFBM to find the most promising candidates based on structural similarity.

Results: The use of MFBM as a means to strengthen or prioritize signals generated from the EHR significantly improved the detection accuracy of ADEs related to pancreatitis. MFBM also highlights the etiology of the ADE by identifying structurally similar drugs, which could follow a similar mechanism of action.

Conclusion: The method proposed in this paper provides evidence of being a promising adjunct to existing automated ADE detection and analysis approaches.

Show MeSH

Related in: MedlinePlus

Receiver Operating Characteristic (ROC) (a) and Precision-Recall (b) curves evaluating the set of 278 EHR ADE candidates with OR05 and different MFBMs.It is worth noting that although OR05 algorithm is very useful to originate the first set of 278 candidate drugs related to pancreatitis (99 out of 278 drugs were already included in the pancreatitis reference standard set), the precision of the method is constant within this set. However, an improvement of the precision in top positions can be achieved using MFBM (in the graphic: black-OR05, red-MACCS, green-GpiDAPH3, yellow-TGT, blue-TGD).
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3404072&req=5

pone-0041471-g002: Receiver Operating Characteristic (ROC) (a) and Precision-Recall (b) curves evaluating the set of 278 EHR ADE candidates with OR05 and different MFBMs.It is worth noting that although OR05 algorithm is very useful to originate the first set of 278 candidate drugs related to pancreatitis (99 out of 278 drugs were already included in the pancreatitis reference standard set), the precision of the method is constant within this set. However, an improvement of the precision in top positions can be achieved using MFBM (in the graphic: black-OR05, red-MACCS, green-GpiDAPH3, yellow-TGT, blue-TGD).

Mentions: The EHR from New York Presbyterian Hospital was mined looking for associations between drugs and the adverse event pancreatitis, and 278 drugs were found to be associated with the ADE using the DPA method by itself. Of those, 99 drugs were already included in the pancreatitis reference standard dataset established previously (see Tables S1 and S2). The precision of the method was calculated as the ratio of true positive cases divided by all the positive cases (Precision  =  TP/(TP+FP)). The overall precision of the EHR analysis is 0.36. The method was compared to random results using the DrugBank database [12], containing 1660 approved drugs (small drugs, biotech and nutraceuticals). Based on expectation, if a random subset of 278 drugs in DrugBank was selected, 42 drugs included in the reference standard would be found. The estimated precision of a method that randomly selects drug candidates was 0.15. The p-value for the probability that Disproportionality Analysis (DPA) identified 99 reference standard drugs in the subset of 278 candidates is very unlikely (p<.001). Table 1 shows the performance of DPA at different top positions. These results point out the usefulness of the application of EHR in the detection of adverse events in drugs, since 99 out of 278 associations were found in the pancreatitis reference standard database. However, if the Precision-Recall and Receiver Operating Characteristic (ROC) curves are plotted for the 278 candidates using OR05 as the scoring function, it is possible to observe that the precision of the method barely improves in top positions (see Table 1 and Figure 2). Nevertheless, as it is explained in the next section, an improvement in ADE detection is still possible through the combination of DPA with MFBM techniques.


Enhancing adverse drug event detection in electronic health records using molecular structure similarity: application to pancreatitis.

Vilar S, Harpaz R, Santana L, Uriarte E, Friedman C - PLoS ONE (2012)

Receiver Operating Characteristic (ROC) (a) and Precision-Recall (b) curves evaluating the set of 278 EHR ADE candidates with OR05 and different MFBMs.It is worth noting that although OR05 algorithm is very useful to originate the first set of 278 candidate drugs related to pancreatitis (99 out of 278 drugs were already included in the pancreatitis reference standard set), the precision of the method is constant within this set. However, an improvement of the precision in top positions can be achieved using MFBM (in the graphic: black-OR05, red-MACCS, green-GpiDAPH3, yellow-TGT, blue-TGD).
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3404072&req=5

pone-0041471-g002: Receiver Operating Characteristic (ROC) (a) and Precision-Recall (b) curves evaluating the set of 278 EHR ADE candidates with OR05 and different MFBMs.It is worth noting that although OR05 algorithm is very useful to originate the first set of 278 candidate drugs related to pancreatitis (99 out of 278 drugs were already included in the pancreatitis reference standard set), the precision of the method is constant within this set. However, an improvement of the precision in top positions can be achieved using MFBM (in the graphic: black-OR05, red-MACCS, green-GpiDAPH3, yellow-TGT, blue-TGD).
Mentions: The EHR from New York Presbyterian Hospital was mined looking for associations between drugs and the adverse event pancreatitis, and 278 drugs were found to be associated with the ADE using the DPA method by itself. Of those, 99 drugs were already included in the pancreatitis reference standard dataset established previously (see Tables S1 and S2). The precision of the method was calculated as the ratio of true positive cases divided by all the positive cases (Precision  =  TP/(TP+FP)). The overall precision of the EHR analysis is 0.36. The method was compared to random results using the DrugBank database [12], containing 1660 approved drugs (small drugs, biotech and nutraceuticals). Based on expectation, if a random subset of 278 drugs in DrugBank was selected, 42 drugs included in the reference standard would be found. The estimated precision of a method that randomly selects drug candidates was 0.15. The p-value for the probability that Disproportionality Analysis (DPA) identified 99 reference standard drugs in the subset of 278 candidates is very unlikely (p<.001). Table 1 shows the performance of DPA at different top positions. These results point out the usefulness of the application of EHR in the detection of adverse events in drugs, since 99 out of 278 associations were found in the pancreatitis reference standard database. However, if the Precision-Recall and Receiver Operating Characteristic (ROC) curves are plotted for the 278 candidates using OR05 as the scoring function, it is possible to observe that the precision of the method barely improves in top positions (see Table 1 and Figure 2). Nevertheless, as it is explained in the next section, an improvement in ADE detection is still possible through the combination of DPA with MFBM techniques.

Bottom Line: A reference standard of drugs known to be causally associated with the adverse event pancreatitis was used to create a MFBM.Candidate drugs associated with these signals were then assessed using the MFBM to find the most promising candidates based on structural similarity.The method proposed in this paper provides evidence of being a promising adjunct to existing automated ADE detection and analysis approaches.

View Article: PubMed Central - PubMed

Affiliation: Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, United States of America. sav7003@dbmi.columbia.edu

ABSTRACT

Background: Adverse drug events (ADEs) detection and assessment is at the center of pharmacovigilance. Data mining of systems, such as FDA's Adverse Event Reporting System (AERS) and more recently, Electronic Health Records (EHRs), can aid in the automatic detection and analysis of ADEs. Although different data mining approaches have been shown to be valuable, it is still crucial to improve the quality of the generated signals.

Objective: To leverage structural similarity by developing molecular fingerprint-based models (MFBMs) to strengthen ADE signals generated from EHR data.

Methods: A reference standard of drugs known to be causally associated with the adverse event pancreatitis was used to create a MFBM. Electronic Health Records (EHRs) from the New York Presbyterian Hospital were mined to generate structured data. Disproportionality Analysis (DPA) was applied to the data, and 278 possible signals related to the ADE pancreatitis were detected. Candidate drugs associated with these signals were then assessed using the MFBM to find the most promising candidates based on structural similarity.

Results: The use of MFBM as a means to strengthen or prioritize signals generated from the EHR significantly improved the detection accuracy of ADEs related to pancreatitis. MFBM also highlights the etiology of the ADE by identifying structurally similar drugs, which could follow a similar mechanism of action.

Conclusion: The method proposed in this paper provides evidence of being a promising adjunct to existing automated ADE detection and analysis approaches.

Show MeSH
Related in: MedlinePlus