Limits...
Context matters: measuring implementation climate among individuals and groups.

Jacobs SR, Weiner BJ, Bunger AC - Implement Sci (2014)

Bottom Line: It has been noted that implementation climate is positively associated with implementation effectiveness.The results are context-specific.In addition, more opportunities are needed to validate this measure and understand how well it predicts and explains implementation effectiveness.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Health Policy and Management, Gillings School of Global Public Health, 1101 McGavran-Greenberg Hall, CB #7411, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7411, USA. jacobssr@live.unc.edu.

ABSTRACT

Background: It has been noted that implementation climate is positively associated with implementation effectiveness. However, issues surrounding the measurement of implementation climate, or the extent to which organizational members perceive that innovation use is expected, supported and rewarded by their organization remain. Specifically, it is unclear whether implementation climate can be measured as a global construct, whether individual or group-referenced items should be used, and whether implementation climate can be assessed at the group or organizational level.

Methods: This research includes two cross-sectional studies with data collected via surveys at the individual level. The first study assessed the implementation climate perceptions of physicians participating in the National Cancer Institute's (NCI) Community Clinical Oncology Program (CCOP), and the second study assessed the perceptions of children's behavioral health clinicians implementing a treatment innovation. To address if implementation climate is a global construct, we used confirmatory factor analysis. To address how implementation climate should be measured and at what level, we followed a five-step framework outlined by van Mierlo and colleagues. This framework includes exploratory factor analysis and correlations to assess differences between individual and group-referenced items and intraclass correlations, interrater agreements, and exploratory factor analysis to determine if implementation climate can be assessed at the organizational level.

Results: The confirmatory factor analysis demonstrated that implementation climate is a global construct consisting of items related to expectations, support and rewards. There are mixed results, however, as to whether implementation climate should be measured using individual or group-referenced items. In our first study, where physicians were geographically dispersed and practice independently, there were no differences based on the type of items used, and implementation climate was an individual level construct. However, in the second study, in which clinicians practice in a central location and interact more frequently, group-referenced items may be appropriate. In addition, implementation climate could be considered an organizational level construct.

Conclusions: The results are context-specific. Researchers should carefully consider the study setting when measuring implementation climate. In addition, more opportunities are needed to validate this measure and understand how well it predicts and explains implementation effectiveness.

Show MeSH

Related in: MedlinePlus

Example of second order CFA model for group-referenced items.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4012549&req=5

Figure 3: Example of second order CFA model for group-referenced items.

Mentions: To determine if implementation climate can be measured as global construct consisting of expectations, rewards and support, we ran a series of CFAs to fit a second-order factor structure to the data in each study, with separate analyses for the individually and group-referenced items (Figures 2 and 3). In both studies, the second-order CFA model for the individually referenced items converged and demonstrated strong fit (Table 2). Based on model fit and the resulting modifications indices (i.e., the minimum that the chi-square statistic is expected to decrease if the corresponding parameter is no longer assumed to be fixed at zero), we did not need to make any post-hoc modifications to the model in either study [34-36].


Context matters: measuring implementation climate among individuals and groups.

Jacobs SR, Weiner BJ, Bunger AC - Implement Sci (2014)

Example of second order CFA model for group-referenced items.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4012549&req=5

Figure 3: Example of second order CFA model for group-referenced items.
Mentions: To determine if implementation climate can be measured as global construct consisting of expectations, rewards and support, we ran a series of CFAs to fit a second-order factor structure to the data in each study, with separate analyses for the individually and group-referenced items (Figures 2 and 3). In both studies, the second-order CFA model for the individually referenced items converged and demonstrated strong fit (Table 2). Based on model fit and the resulting modifications indices (i.e., the minimum that the chi-square statistic is expected to decrease if the corresponding parameter is no longer assumed to be fixed at zero), we did not need to make any post-hoc modifications to the model in either study [34-36].

Bottom Line: It has been noted that implementation climate is positively associated with implementation effectiveness.The results are context-specific.In addition, more opportunities are needed to validate this measure and understand how well it predicts and explains implementation effectiveness.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Health Policy and Management, Gillings School of Global Public Health, 1101 McGavran-Greenberg Hall, CB #7411, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7411, USA. jacobssr@live.unc.edu.

ABSTRACT

Background: It has been noted that implementation climate is positively associated with implementation effectiveness. However, issues surrounding the measurement of implementation climate, or the extent to which organizational members perceive that innovation use is expected, supported and rewarded by their organization remain. Specifically, it is unclear whether implementation climate can be measured as a global construct, whether individual or group-referenced items should be used, and whether implementation climate can be assessed at the group or organizational level.

Methods: This research includes two cross-sectional studies with data collected via surveys at the individual level. The first study assessed the implementation climate perceptions of physicians participating in the National Cancer Institute's (NCI) Community Clinical Oncology Program (CCOP), and the second study assessed the perceptions of children's behavioral health clinicians implementing a treatment innovation. To address if implementation climate is a global construct, we used confirmatory factor analysis. To address how implementation climate should be measured and at what level, we followed a five-step framework outlined by van Mierlo and colleagues. This framework includes exploratory factor analysis and correlations to assess differences between individual and group-referenced items and intraclass correlations, interrater agreements, and exploratory factor analysis to determine if implementation climate can be assessed at the organizational level.

Results: The confirmatory factor analysis demonstrated that implementation climate is a global construct consisting of items related to expectations, support and rewards. There are mixed results, however, as to whether implementation climate should be measured using individual or group-referenced items. In our first study, where physicians were geographically dispersed and practice independently, there were no differences based on the type of items used, and implementation climate was an individual level construct. However, in the second study, in which clinicians practice in a central location and interact more frequently, group-referenced items may be appropriate. In addition, implementation climate could be considered an organizational level construct.

Conclusions: The results are context-specific. Researchers should carefully consider the study setting when measuring implementation climate. In addition, more opportunities are needed to validate this measure and understand how well it predicts and explains implementation effectiveness.

Show MeSH
Related in: MedlinePlus