Yesterday was the third & last morning of this year's BABCP summer conference in Warwick.  I have already written about the second day in "Warwick BABCP conference: 2nd day - behavioural activation, Kyrios OCD, 'mind the gap', & DeRubeis on personalization (4th post)".  Overall, there were two particular presentations I was especially looking forward to coming to this conference and they have both delivered in spades.  One was Pim Cuijpers' keynote on "Four decades of outcome research on psychotherapies for adult depression: what next?" - see the blog post "Warwick BABCP conference: first morning - trauma memories & a master presentation on four decades of outcome research (2nd post)".  The other was this morning's symposium "The singer and not the song? Evidencing therapist effects across the IAPT stepped care model".  There were three great papers given by Nick Firth, Dave Saxon and Jo-Ann Periera, followed by a fourth paper delivering what was for me a bit of rearguard unreconstructed working alliance bashing from Glen Waller.  The new exciting data however very clearly came with the first three talks.

The first of these was Nick Firth on "Therapist effects and moderators of effectiveness and efficiency in psychological wellbeing practitioners: a multilevel modelling analysis".  Nick clarified that "A therapist effect means that the patients of some therapists have significantly better outcomes than the patients of other therapists" and he underlined that this is "Really important for providing effective psyschological services".   He noted that a growing body of research highlights that therapist effects typically contribute about 5-10% of variance in patient outcome.  To put this 5-10% figure into context, I quote from Baldwin & Imel's chapter on "Therapist effects: findings & methods" in the truly excellent 2013 6th edition of the "Handbook of psychotherapy and behavior change."  They write (p.277) "Most meta-analyses comparing an active treatment to a wait-list control have posttest effect sizes at or below d=1.0 ... which corresponds to about 20% of the variance in outcome. Thus, psychotherapy as a whole and all the constructs it entails - specific factors, common factors, therapist factors, alliance, adherence, and so on - accounts for only 20% of the variance in outcomes. As we discuss below, the therapeutic alliance appears to account for about 5% of the variance in outcomes and therapist adherence to the treatment protocol 0% of the variance in outcome.  Relatively speaking, 5% of the variance is not so bad."  Mm ... as Pim Cuijpers talk highlighted, psychotherapies for depression don't typically achieve a d of 1.0 - once one has allowed for allegiance effects, problems with wait-list as a control condition, and other methodological issues - so maybe 5% of the variance is even more impressive as a component of the results.  For more background to therapist effects see, for example, my earlier blog posts "Five recent research studies on the worrying variability both in psychotherapist effectiveness and also in willingness to change" and "What shall we do about the fact that there are supershrinks and pseudoshrinks?". 

But back to this morning's fascinating talks - Nick pointed out that not much attention has been paid to therapist effects in low-intensity services and interventions (e.g. IAPT and PWPs).  In his study he looked at the routine clinical outcome data from a single IAPT service (PHQ-9, GAD-7, WSAS) involving 6,111 patients receiving one-to-one treatment from 56 PWP's over three years. Multilevel modelling determined the size of the therapist effect and examined outcome moderators. The headline finding was, even in this very protocol-driven therapeutic situation, PWP therapist effects accounted for 6-7% of outcome variance (6.4% depression, 6.1% anxiety, and 7.% impairment).  Around 15% of PWPs had above average outcomes, around 70% had average outcomes, and around 15% had below average outcomes.  PWPs with above average outcomes achieved almost double the change per treatment session compared with PWPs with below average outcomes.  Three factors moderated the therapist effect - initial severity, treatment duration, and treatment non-completion.  There was a greater difference between PWPs when working with more severely depressed and functionally impaired patients.  There was no change in therapist effect size as severity of anxiety increased.  There were greater differences between PWPs' outcomes as the number of treatment sessions increased.  Wow!  Interesting to note replication of the finding from previous therapist variability research that differences between therapist outcomes emerge more clearly as the challenge presented by the client increases (with depression severity & functional impairment).  No big surprise here, but worth underlining.  The point about variability becoming clearer as number of treatment sessions increases should be seen in the context that we're talking about short therapies here with single figure total number of sessions.  The finding that increasing anxiety severity isn't associated with increasing therapist variability I note, but I would like to see this finding replicated in other patient populations before giving it much credence.

Next up was Dave Saxon on "Variability in practice: therapist effects in an IAPT service delivering CBT and counselling".  This documents therapist effects at step 3 of the IAPT stepped care model.  In IAPT, most patients receive a low intensity intervention delivered by a Psychological Wellbeing Practitioner (PWP) - see Nick's talk above.  Dave Saxon was looking at the minority of patients who get referred up the stepped care ladder to treatment with well-trained counsellors or CBT practitioners.  In his paper he analysed in detail outcomes achieved by 64 therapists (32 counsellors & 32 CBT practitioners) treating 4,837 patients who attended at least 2 treatment sessions and provided first & last PHQ-9 scores.  He also looked at dropout rates for this same therapist group.  He considered two key questions "Is the therapy type (CBT or counselling) a significant predictor of outcome?" and "Does the variability between individual therapists matter more than the variability between therapy types?"  So what were his findings?  I wait to hear a pin drop ...

Intake severity scores on the PHQ-9 for CBT and for counselling were 15.5 and 14.9 respectively.  Outcome scores were identical at 9.8.  Effect sizes were 0.91 and 0.82 respectively and recovery rates 43.6% and 39.6%. These differences were not significant statistically.  Therapy type (CBT or counselling) was not a significant predictor of outcome.  This finding ... of no difference between outcomes achieved by different well-delivered therapies for depression ... is of course just a small scale example of the very large scale meta-analytic conclusion delivered by Pim Cuijpers in his excellent keynote talk at this conference a couple of days ago.  Note that Pim said that meta-analysis didn't always show quite such good outcomes for counselling treatment of depression when compared with CBT, IPT or BA interventions, and that he was not making these equivalence claims when it comes to treating anxiety disorders.  Back to Dave's talk today ... what about variability between individual therapists?  For those who had both recovery and drop-out data, mean recovery rate for CBT was 41.3% but this varied within the 32 therapists involved from 12.3 to 61.9%.  Similarly mean drop-out rate was 29.3%, but the variation between therapists was 7.3 to 65.4%.  The 32 counsellors presented the same picture with a mean recovery rate of 40.3% but therapist variation between 12.5 & 77.8%.   For dropout the mean rate for counselling was 29.9% with variation between 3.2 and 51.7%. There was "Huge variability between therapists on both outcomes (recovery & dropout), in both therapy types".  Interestingly these figures translated to therapist effects accounting for 5.4% of the overall outcome variance (less than the 6.4% found for the PHQ-9 in Nick's PWP study above).  With Dave's study on CBT & counselling, 18.8% of therapists produced significantly better than average outcomes, 67.2% were not significantly different than average, and 14.0% were significantly worse than average.  The differences in mean recovery rates between these three groups (each containing a mix of CBT practitioners & counsellors) were startling - 56.9% for the above average group, 39.7% for the average, and 24.3% for the below average.  The above average group was over twice as effective as the below average group (and over 40% more effective than the average group, who were in turn over 60% more effective than the below average group).  If these differences had been demonstrated between two different types of therapy, we would see banner headlines all around the world! 

For a description of the third of the symposium's talks and a more general discussion see "Warwick BABCP conference: 3rd day - what personal qualities distinguish more & less effective therapists? (6th post)".