Raters Unaffiliated with the Tested Treatment May Have Most Objective Fidelity Feedback


HAQThis ancillary investigation of data from National Drug Abuse Treatment Clinical Trials Network protocol CTN-0031 (“Stimulant Abuser Groups to Engage in 12-Step (STAGE-12)“), by Pacific Northwest Node members Michelle Peavy and Sharon Garrett, as well as Joseph Guydish, Jennifer Knapp Manuel, and colleagues, investigated the correspondence among four groups of raters on adherence to STAGE-12, a manualized 12-step facilitation (TSF) group and individual treatment targeting stimulant abuse.

The four rater groups compared included the study therapists, their supervisors, study-related (“TSF expert”) raters, and non-project-related (“external”) raters.

Results indicated that external raters rated most critically Mean Adherence — the mean of all the adherence items — and global performance. External raters also demonstrated the highest degree of reliability with the designated expert. Therapists rated their own adherence lower, on average, than did supervisors and TSF expert raters, but therapist ratings also had the poorest reliability.

Conclusions:  Findings highlight the challenge in developing practical, but effective methods of fidelity monitoring for evidence-based practice in clinical settings. While funding and licensing agencies increasingly call for use of evidence-based treatments, community-based organizations implementing them will seek the simplest, most reliable and cost-effective ways of monitoring their delivery.

These results suggest that there may be a role for on-site therapists or supervisors rating adherence, and that raters unaffiliated with the treatment being tested may provide the most objective ratings.

Citation: Peavy KM, Guydish JR, Manuel JK,et al. Treatment Adherence and Competency Ratings Among Therapists, Supervisors, Study-Related Raters and External Raters in a Clinical Trial of 12-Step Facilitation for Stimulant Users. Journal of Substance Abuse Treatment 2014 (in press).

Find it in the CTN Dissemination Library!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: