Limits...
mQC: A Heuristic Quality-Control Metric for High-Throughput Drug Combination Screening

View Article: PubMed Central - PubMed

ABSTRACT

Quality control (QC) metrics are critical in high throughput screening (HTS) platforms to ensure reliability and confidence in assay data and downstream analyses. Most reported HTS QC metrics are designed for plate level or single well level analysis. With the advent of high throughput combination screening there is a need for QC metrics that quantify the quality of combination response matrices. We introduce a predictive, interpretable, matrix-level QC metric, mQC, based on a mix of data-derived and heuristic features. mQC accurately reproduces the expert assessment of combination response quality and correctly identifies unreliable response matrices that can lead to erroneous or misleading characterization of synergy. When combined with the plate-level QC metric, Z’, mQC provides a more appropriate determination of the quality of a drug combination screen. Retrospective analysis on a number of completed combination screens further shows that mQC is able to identify problematic screens whereas plate-level QC was not able to. In conclusion, our data indicates that mQC is a reliable QC filter that can be used to identify problematic drug combinations matrices and prevent further analysis on erroneously active combinations as well as for troubleshooting failed screens. The R source code of mQC is available at http://matrix.ncats.nih.gov/mQC.

No MeSH data available.


Proposed QC guideline for drug combination screening.(A) Each combination screening is represented by two independent points: a red point (Z’ as X-axis value and percentage of “Good” matrices as Y-axis), and a green point (Z’ as X-axis and percentage of “Good” plus “Medium” quality matrices as Y-axis). The distribution associated with Z’, Good%, Good + Medium% are beside the scatter plot. The dashed lines indicate the best practice cutoff for Z’ and mQC levels given a screening. (B) The best practice workflow for quality control of a cHTS campaign.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC5121902&req=5

f5: Proposed QC guideline for drug combination screening.(A) Each combination screening is represented by two independent points: a red point (Z’ as X-axis value and percentage of “Good” matrices as Y-axis), and a green point (Z’ as X-axis and percentage of “Good” plus “Medium” quality matrices as Y-axis). The distribution associated with Z’, Good%, Good + Medium% are beside the scatter plot. The dashed lines indicate the best practice cutoff for Z’ and mQC levels given a screening. (B) The best practice workflow for quality control of a cHTS campaign.

Mentions: This large-scale analysis was in line with our initial hypothesis that Z’ alone is insufficient to indicate the overall quality of a response matrix. In addition, from comparing Z’ and mQC using a subset which has trackable plate-level data (totally 119,287 blocks available in Supplementary Dataset S3), we observed weak correlation between Z’ and mQC using Spearman correlation (ρgood = 0.23, ρmedium = −0.008, ρbad = −0.38 when Z’ is aggregated by screen, Fig. 5A). Z’ and SSMD also have poor correlation with mQC if we analyze the QC breakdown by plate (Supplementary Fig. S2). Noticing the fact that Z’ or SSMD may not hold if the controls are placed on one side in the presence of dramatic plate effect, we also calculated Z’ (sample) and SSMD (sample) using the block DMSO controls and original positive controls. However, we are still unable to find a reasonable correlation between plate-level QC metrics (Z’ (sample) or SSMD (sample)) and mQC (Supplementary Fig. S3), although Z’ (sample) and SSMD (sample) only achieves a mediocre correlation with Z’ (plate) or SSMD (plate) (Supplementary Fig. S4). Therefore, it is reasonable to define a combined criterion as the basis of a QC guideline for cHTS. The conventional criterion for a good HTS is Z’ > 0.5, and here we found that ~85% screenings met this QC requirement. Based on this 85th quantile that defines an excellent HTS assay based on plate-level quality, the corresponding matrix-level mQC criterion should be “screen with >60% Good response matrices and >90% Good or Medium response matrices” (the horizontal dashed lines in Fig. 5A). Herein we suggest that the quality of a cHTS campaign be judged by both plate-level and matrix-level QC metrics: (1) Z’ > 0.5 and (2) >60% “Good” response matrices and (3) >90% “Good” or “Medium” response matrices (Fig. 5B). If only plate-level QC is satisfied, it suggests that major matrix-level issues are involved, such as low cell viability, wrong time points, unstable readout, problems in chemical selection/handling/concentration, etc. Otherwise, it suggests a failed control or biased layout as a majority of response matrices satisfy the matrix-level QC criteria.


mQC: A Heuristic Quality-Control Metric for High-Throughput Drug Combination Screening
Proposed QC guideline for drug combination screening.(A) Each combination screening is represented by two independent points: a red point (Z’ as X-axis value and percentage of “Good” matrices as Y-axis), and a green point (Z’ as X-axis and percentage of “Good” plus “Medium” quality matrices as Y-axis). The distribution associated with Z’, Good%, Good + Medium% are beside the scatter plot. The dashed lines indicate the best practice cutoff for Z’ and mQC levels given a screening. (B) The best practice workflow for quality control of a cHTS campaign.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC5121902&req=5

f5: Proposed QC guideline for drug combination screening.(A) Each combination screening is represented by two independent points: a red point (Z’ as X-axis value and percentage of “Good” matrices as Y-axis), and a green point (Z’ as X-axis and percentage of “Good” plus “Medium” quality matrices as Y-axis). The distribution associated with Z’, Good%, Good + Medium% are beside the scatter plot. The dashed lines indicate the best practice cutoff for Z’ and mQC levels given a screening. (B) The best practice workflow for quality control of a cHTS campaign.
Mentions: This large-scale analysis was in line with our initial hypothesis that Z’ alone is insufficient to indicate the overall quality of a response matrix. In addition, from comparing Z’ and mQC using a subset which has trackable plate-level data (totally 119,287 blocks available in Supplementary Dataset S3), we observed weak correlation between Z’ and mQC using Spearman correlation (ρgood = 0.23, ρmedium = −0.008, ρbad = −0.38 when Z’ is aggregated by screen, Fig. 5A). Z’ and SSMD also have poor correlation with mQC if we analyze the QC breakdown by plate (Supplementary Fig. S2). Noticing the fact that Z’ or SSMD may not hold if the controls are placed on one side in the presence of dramatic plate effect, we also calculated Z’ (sample) and SSMD (sample) using the block DMSO controls and original positive controls. However, we are still unable to find a reasonable correlation between plate-level QC metrics (Z’ (sample) or SSMD (sample)) and mQC (Supplementary Fig. S3), although Z’ (sample) and SSMD (sample) only achieves a mediocre correlation with Z’ (plate) or SSMD (plate) (Supplementary Fig. S4). Therefore, it is reasonable to define a combined criterion as the basis of a QC guideline for cHTS. The conventional criterion for a good HTS is Z’ > 0.5, and here we found that ~85% screenings met this QC requirement. Based on this 85th quantile that defines an excellent HTS assay based on plate-level quality, the corresponding matrix-level mQC criterion should be “screen with >60% Good response matrices and >90% Good or Medium response matrices” (the horizontal dashed lines in Fig. 5A). Herein we suggest that the quality of a cHTS campaign be judged by both plate-level and matrix-level QC metrics: (1) Z’ > 0.5 and (2) >60% “Good” response matrices and (3) >90% “Good” or “Medium” response matrices (Fig. 5B). If only plate-level QC is satisfied, it suggests that major matrix-level issues are involved, such as low cell viability, wrong time points, unstable readout, problems in chemical selection/handling/concentration, etc. Otherwise, it suggests a failed control or biased layout as a majority of response matrices satisfy the matrix-level QC criteria.

View Article: PubMed Central - PubMed

ABSTRACT

Quality control (QC) metrics are critical in high throughput screening (HTS) platforms to ensure reliability and confidence in assay data and downstream analyses. Most reported HTS QC metrics are designed for plate level or single well level analysis. With the advent of high throughput combination screening there is a need for QC metrics that quantify the quality of combination response matrices. We introduce a predictive, interpretable, matrix-level QC metric, mQC, based on a mix of data-derived and heuristic features. mQC accurately reproduces the expert assessment of combination response quality and correctly identifies unreliable response matrices that can lead to erroneous or misleading characterization of synergy. When combined with the plate-level QC metric, Z’, mQC provides a more appropriate determination of the quality of a drug combination screen. Retrospective analysis on a number of completed combination screens further shows that mQC is able to identify problematic screens whereas plate-level QC was not able to. In conclusion, our data indicates that mQC is a reliable QC filter that can be used to identify problematic drug combinations matrices and prevent further analysis on erroneously active combinations as well as for troubleshooting failed screens. The R source code of mQC is available at http://matrix.ncats.nih.gov/mQC.

No MeSH data available.