Listen to thisEvaluation defined
Professional competence evaluation is made in a series of formal and
informal measurements that result in a judgment that an “individual is
fit to practice a profession autonomously" (McGaghie, 1991). Summative
evaluation describes “how effective or ineffective, how adequate or
inadequate, how good or bad, how valuable or invaluable, and how
appropriate or inappropriate" the supervisee is “in terms of the
perceptions of the individual who makes use of the information provided
by the evaluator" (Isaac 8 Mitchell, 1981, p. 2). Counsellor supervisors
are responsible for summative evaluations and assessments of supervisee
competence to university departments, state licensing boards, and agency
administrators. Summative evaluation is described by Bernard and
Goodyear (1992) as “the moment of truth when the supervisor steps back,
takes stock, and decides how the trainee measures up" (p. 105).
Effective summative evaluation requires clearly stated performance objectives that can be assessed in both quantitative and qualitative terms and that have been made explicit to the supervisee or trainee during initial supervision contacts. The heart of counsellor evaluation, however, is an on-going formative process which uses feedback and leads to trainee skills improvement and positive client outcome. In this case the trainee is the person using the information. Bernard and Goodyear (1992) refer to this kind of evaluation as “a constant variable in supervision." As a result, every supervision session will contain either an overt or covert formative evaluation component.
Evaluation practices and procedures
When supervisors measure behavioural therapeutic skills they find
several difficult areas. First, they find that measurement and
subsequent evaluation of therapeutic skill is a complex process in a
field where many skills inventories and behavioural checklists abound,
and research findings suggest that these may lack adequate reliability
and validity. Second, in teaching settings supervisors recognize the
tension between providing a supportive facilitative environment within
which counsellors-in-training can feel free to stretch and learn
counselling skills and the anxiety that results from academic grades.
Third, lacking a theory of supervision, supervisors are unable to
articulate desired outcomes for their supervisees and may revert to the
evaluation of administrative detail and case management.
As a result of these difficulties, numerous areas of competency may be neglected, anxiety may persist, and supervisors may resort to summative evaluation practices in global and poorly measured terms.
There are resources which outline required skills and knowledge for effective evaluation practices. The Curriculum Guide for Training Counsellor Supervisors (Borders et al., 1991) provides specific learning objectives for supervisors in training. Other current publications (Bernard & Goodyear, 1992; Borders & Leddick, 1987; McGaghie, 1991; Stoltenberg & Delworth, 1987) further develop the Guide’s “three curriculum threads" (p.60) of self awareness, theoretical and conceptual knowledge, and skills and techniques.
Guidelines
The guidelines and suggestions from these resources are summarized in
the following list of effective evaluation practices:
1. Clearly communicate evaluation criteria to supervisees and develop a mutually agreed upon written contract reflecting these criteria.
2. Identify and communicate supervisee strengths and weaknesses. The Ethical Guidelines for Counsellor Supervisors (ACES, 1993) recommend that supervisors “provide supervisees with ongoing feedback on their performance." This performance feedback establishes for supervisees a clear sense of what they do well and which skills need to be developed. Supervisee strengths and weaknesses can be evaluated in terms of process, conceptual, personal, and professional skills (Bernard R Goodyear, 1992, p. 42).
3. Use constructive feedback techniques during evaluations. Supervisees are more likely to “hear" corrective feedback messages when these are preceded by positive feedback, focussed on observable behaviours, and are delayed until a positive relationship has been established.
4. Use specific, behavioural, observable feedback dealing with counselling skills and techniques; avoid terms such as “understanding," “knowing" and “appreciating," and “being aware of." Successful evaluation practices should include behaviourally-based learning objectives (Gage & Berliner, 1984).
5. Use Interpersonal Process Recall (IPR) to raise supervisees” awareness about their personal developmental issues. The unobtrusive and nonthreatening nature of IPR is particularly helpful as supervisees retrospectively explore their thoughts, feelings, and a variety of client stimuli during counselling sessions. This process can assist supervisees in contributing to, and benefiting from, formative evaluation.
6. Employ multiple measures of supervisee counselling skills. These can include a variety of standardized rating scales including measures completed by both supervisor and supervisee, client ratings, and behavioural scales (Stoltenberg & Delworth, 1987). Additional measures such as work samples from audio/videos, critiques of counselling sessions, and conceptual case studies (both brief and detailed) can provide a comprehensive picture of a supervisee’s competency, expectations, needs and professional development, as well as an understanding of the context within which both the counselling and the supervision take place.
7. Maintain a series of work samples in a portfolio for summative evaluation. Since the evaluation of only one session provides an inadequate assessment of supervisee competency, and the selective nature of work samples may prove to he an overly negative reflection of current competency level, the portfolio provides both the supervisor and the supervisee with a more comprehensive and useful basis for a summative evaluation.
8. Use a developmental approach which emphasizes both progressive growth toward desired goals and the learning readiness of the trainee (Nance, 1990). The Nance model emphasizes a learning readiness based on the supervisee’s ability, confidence, and willingness “the assessment of which directs the roles and practices of the supervisor. As a result, supervisors can “match" their supervisee’s level and “move" them toward independent functioning one step at a time. Although Nance does not specify evaluation practices, he clearly describes effective supervisory styles, interventions, role, contracts, and agendas for each developmental stage. These variables can guide the evaluation process indirectly by enabling the supervisor to understand the characteristics and appropriate expectations for supervisees at each developmental level.
Summary
A structured approach to supervisee assessment and evaluation produces
several beneficial outcomes.
First, supervisors can reduce their own, as well as their supervisee–s, anxiety about the process. The meanings associated with assessment can be altered to suggest a positive experience from which both partners can grow and learn.
Second, supervisors who articulate their adopted “supervision theory" to their supervisees will also clarify their evaluation criteria as well as their supervision practices.
Third, when evaluation is viewed as a process of formative and summative “assessment" of the skills, techniques, and developmental stage of the supervisee, both supervisees and their clients benefit.
Fourth, as supervisors deal successfully with the process of supervisee evaluation, they also bring similar skills to the evaluation of their training programs, an area in search of an appropriate evaluation paradigm.
Finally, just as training is most successful when multiple methods (didactic, modelling, and experiential) of skills acquisition are employed, so too the use of multiple methods for evaluation contributes to the supervisee’s sense of self-worth and success.
References
Association for Counsellor Education and Supervision. Ethical Guidelines for Counselling Supervisors (1993). ACES Spectrum, 53(4), 5-8.
Bernard, J. M., 8c Goodyear, R. K. (1992). Fundamentals of clinical supervision. Boston: Allyn & Bacon.
Borders, L. D., Bernard, J. M., Dye, H. A., Fong, M. L., Henderson, P., 8 Nance, D. (1991). Curriculum guide for training counsellor supervisors: Rationale, development, and implementation. Counsellor Education and Supervision, 31, 58-77.
Borders, L. D., & Leddick, G. R. (1987). Handbook of counselling supervision. Alexandria, VA: Association for Counsellor Education and Supervision.
Gage, N. L., & Berliner, D. C. (1984). Educational psychology (3rd ed.). Boston: Houghton Mifflin.
Isaacs, S., & Michael, W. B. (1981). Handbook in research and evaluation (2nd ed.). San Diego: EdITS.
McGaghie, W. C. (1991). Professional competence evaluation. Educational Researcher, 20, 3-9.
Nance, D. W. (1990). ACES Workshop on Counsellor Supervision. Workshop presented at the annual convention of American Association for Counselling and Development, Cincinnati, Okl.
Stoltenberg, C. D., & Delworth, U. (1987). Supervising counsellors and therapists: A developmental approach. San Francisco: Jossey-Bass.
Acknowledgements: ERIC Digest
