I find the recent paper by Kraus & colleagues a bit scary - "Therapist effectiveness: Implications for accountability and patient care" - with its abstract reading "Significant therapist variability has been demonstrated in both psychotherapy outcomes and process (e.g., the working alliance). In an attempt to provide prevalence estimates of "effective" and "harmful" therapists, the outcomes of 6960 patients seen by 696 therapists in the context of naturalistic treatment were analyzed across multiple symptom and functioning domains. Therapists were defined based on whether their average client reliably improved, worsened, or neither improved nor worsened. Results varied by domain with the widespread pervasiveness of unclassifiable/ineffective and harmful therapists ranging from 33 to 65%. Harmful therapists demonstrated large, negative treatment effect sizes (d = - 0.91 to - 1.49) while effective therapists demonstrated large, positive treatment effect sizes (d = 1.00 to 1.52). Therapist domain-specific effectiveness correlated poorly across domains, suggesting that therapist competencies may be domain or disorder specific, rather than reflecting a core attribute or underlying therapeutic skill construct. Public policy and clinical implications of these findings are discussed, including the importance of integrating benchmarked outcome measurement into both routine care and training."  Gosh ... 33 to 65% of therapists rated as unclassifiable/ineffective and/or harmful, while Michael Lambert found that "90% of therapists interviewed thought they were in the top 25% of effective therapists when compared with their peers ... and no therapist rated themselves as below average in effectiveness."  We sure as hell need ways for mental health professionals to get a clearer sense of just how effective or ineffective they are being in helping their clients.  Incidentally this is almost certainly also true for other health professionals as well. 

Happily this need is now increasingly recognized.  The American Psychological Association's publication "Commission on accreditation: implementing regulations" provides guidelines for psychologist training programmes. On pages 52/53 they state "Training in empirically supported procedures/treatments should focus on assisting students and interns to acquire knowledge, skills and attitudes that promote the integration of science and practice ... Through this training, students and interns should ... be provided with supervised experience in collecting quantitative outcome data on the psychological services they provide."  And just this July the APA launched a new outcome measurement database.  PracticeOUTCOMES: Measures for Psychologists began as a pilot project of the 2010 APA Presidential Task Force for Advancing Practice.  The task force was created to develop a strategy for APA members to proactively monitor outcomes in their practices ... The task force ultimately determined that ... psychologists needed to easily identify, review and select measures suitable for their practice.  To that end, the task force conceptualized the development of a comprehensive, searchable database of measures that psychologists can use to monitor patient progress in treatment. The resulting product is PracticeOUTCOMES: Measures for Psychologists ... This practical tool contains information on measure domains, features, costs and reliability, among other indicators. The database also allows members to compare measures side by side in order to help psychologists choose the right outcome measure for their practice."

The post "What shall we do about the fact that there are supershrinks and pseudoshrinks?" explores this territory further, as too do "On becoming a better therapist", "A clear and sensible way to make psychotherapy more helpful" and "Client-directed, outcome-informed therapy".   One option amongst these ways of monitoring outcome are the simple, quick and effective Outcome Rating & Session Rating Scales (ORS & SRS).  The Outcome Rating Scale (ORS), in my opinion, provides a helpful succinct overview of how someone is doing.  I think it complements rather than replaces more specific outcome measures ... see, for example, "Increasing access to psychological therapies (IAPT) outcomes toolkit".  A particular strength of the ORS, if used to monitor progress in a new client, is the availability of expected improvement trajectories so that one can see whether the client is doing worse, as well, or better than typically predicted.  This encourages a rapid response if the client isn't making the gains that one would want.  The Session Rating Scale (SRS) monitors therapeutic alliance and I'll write more on this soon. 

For much more detail about monitoring clinical practice (including links to a variety of well-validated approaches), see the excellent recent book by Barkham & colleagues "Developing & delivering practice-based evidence".