Limits...
Quality assessment of observational studies in a drug-safety systematic review, comparison of two tools: the Newcastle-Ottawa Scale and the RTI item bank.

Margulis AV, Pladevall M, Riera-Guardia N, Varas-Lorenzo C, Hazell L, Berkman ND, Viswanathan M, Perez-Gutthann S - Clin Epidemiol (2014)

Bottom Line: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB's developers.

View Article: PubMed Central - PubMed

Affiliation: RTI Health Solutions, Barcelona, Spain.

ABSTRACT

Background: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.

Methods: We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.

Results: The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).

Conclusion: The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB's developers.

No MeSH data available.


Risk of bias by domain (in bold) and question in nine case-control studies using the Newcastle–Ottawa Scale.Note: Numbers on the green bar represent the number of studies with low risk of bias over the number of studies assessed.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4199858&req=5

f1-clep-6-359: Risk of bias by domain (in bold) and question in nine case-control studies using the Newcastle–Ottawa Scale.Note: Numbers on the green bar represent the number of studies with low risk of bias over the number of studies assessed.

Mentions: NOS scores for the observational studies included in our systematic review ranged from 5 to 9 (the range of possible scores goes from 0 through 9), with a median and mode of 8 (25th percentile [p25] =7; p75 =8). A summary of the risk of bias as assessed using the NOS for case-control and cohort studies is shown in Figures 1 and 2. All studies earned a star for comparability with regards to age and sex, which we considered the most important factors for adjustment because our study eligibility criteria required, as a minimum, adjustment for age and sex. All except one included study earned a second star for additional adjustment. Among case-control studies, evaluation of the nonresponse rate was the question with the lowest count of stars, with only 22% of the studies having a low risk of bias. Among cohort studies, the lowest count of stars was for the question evaluating the presence of the outcome at the start of follow-up, with 46% of studies showing low risk of bias.


Quality assessment of observational studies in a drug-safety systematic review, comparison of two tools: the Newcastle-Ottawa Scale and the RTI item bank.

Margulis AV, Pladevall M, Riera-Guardia N, Varas-Lorenzo C, Hazell L, Berkman ND, Viswanathan M, Perez-Gutthann S - Clin Epidemiol (2014)

Risk of bias by domain (in bold) and question in nine case-control studies using the Newcastle–Ottawa Scale.Note: Numbers on the green bar represent the number of studies with low risk of bias over the number of studies assessed.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4199858&req=5

f1-clep-6-359: Risk of bias by domain (in bold) and question in nine case-control studies using the Newcastle–Ottawa Scale.Note: Numbers on the green bar represent the number of studies with low risk of bias over the number of studies assessed.
Mentions: NOS scores for the observational studies included in our systematic review ranged from 5 to 9 (the range of possible scores goes from 0 through 9), with a median and mode of 8 (25th percentile [p25] =7; p75 =8). A summary of the risk of bias as assessed using the NOS for case-control and cohort studies is shown in Figures 1 and 2. All studies earned a star for comparability with regards to age and sex, which we considered the most important factors for adjustment because our study eligibility criteria required, as a minimum, adjustment for age and sex. All except one included study earned a second star for additional adjustment. Among case-control studies, evaluation of the nonresponse rate was the question with the lowest count of stars, with only 22% of the studies having a low risk of bias. Among cohort studies, the lowest count of stars was for the question evaluating the presence of the outcome at the start of follow-up, with 46% of studies showing low risk of bias.

Bottom Line: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB's developers.

View Article: PubMed Central - PubMed

Affiliation: RTI Health Solutions, Barcelona, Spain.

ABSTRACT

Background: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.

Methods: We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.

Results: The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).

Conclusion: The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB's developers.

No MeSH data available.