In a previous essay I noted that the reason we do not know much about the power of relationships in CYC work is that the relationships themselves are not measured in evaluation, in practice, or in research. An additional problem I address here is that we rarely study the impact of a professional; by that I mean we do not separate out the effect of the type of practice intervention, the effect of the type of client, and the effect of different types and styles of professionals. These are three separate, measurable influences.
For example, in child welfare we have reasonable studies comparing foster care to group care to at-home models, but these lump everything and everyone together, and they also do not control for good and bad foster care or good and bad group care. In community settings there are many studies of the impact of practices but little study of the impact of individual practitioners apart from the program and controlling for the varieties of youth or client.
We are not alone; allied fields have not done much either; there is one interesting and important exception. Researchers who study therapy have made important progress because of the increased sophistication of a) software programs, b) measurement tools, c) research designs, and c) conceptual development. Sounds dry, but bear with me—we can learn from them, and here I am relying on Barkham, Lutz, Lambert, and Saxon (2017).
Let’s start with variability. There are simple concepts about variability that we intuitively know from practice experience but that are not usually included in analysis of practice effectiveness. The first is that the severity of client problems varies widely, and so do client strengths and capacities. Second, practitioners’ competence also varies from low to high and from inexperienced to experienced. It seems obvious to us that these things are true.
Yet if we do not measure these things, they are included in studies as the “error” term: Unmeasured things we cannot control. Good and poor practitioners are lumped together. All types of clients are also lumped together. All practitioners get the credit whether the results are outstanding or terrible. This has been standard practice in research about practice effectiveness. This is why most studies find “no effect.”
If we do include them, we can begin to think about the differences between groups of practitioners—practitioner effectiveness. If we can manage to study multiple CYC practitioners who have multiple clients, we can distinguish between the best practitioners and average practitioners, and we might also be able to distinguish between the worst practitioners and others. We can come close to being able to say that the best practitioners cause change in clients.
Here is why measuring these things is important to the therapy world, as described by Barkham, et al (2017) and why it might be important to us (I paraphrase them here):
Only recently have we studied the top 15%-20% of therapists, and in CYC we have very little data about the top 15% to 20% of practitioners. We know they exist, but in too many organizations they are not recognized, much less studied, and too often we have vague ideas about what makes them outstanding. We can fix this for the sake of our professional development, for the morale of practitioners, and for the benefit of those we serve.
Reference
Barkham, M., Lutz, W., Lambert, M. J., & Saxon, D. (2017). Therapist effects, effective therapists, and the law of variability. In L. G. Castonguay & C. E. Hill (Eds.), How and Why are Some Therapists Better than Others: Understanding Therapists Effects. American Psychological Association.