Listen to thisI hear and read a lot about evidence-based practice. Recently, I have read that states are beginning to pass legislation calling for reliance on evidence-based practice in education.
I believe that evidence-based practice is a good thing when it informs what we do with children. I become concerned when I see evidence-based practice limiting what professionals do.
So what is evidence-based practice? How good is it? Just how much should we rely on it?
Evidence-based practice is nothing more than practice that is based on evidence. But what evidence? The evidence is most often some measure of behavior. Behavior is, after all, the only thing we can readily measure. We can observe and quantify behavior. And where does the evidence come from? Most often, it comes from scientific studies published in professional literature.
Scientific studies require someone who has the time and the resources, including access to a sufficient number of subjects, to conduct a study. There is a target behavior and an intervention they believe to have merit. There is some measure of behavior before the intervention, the intervention, and then a measure of behavior after the intervention. When the behavior after the intervention shows improvement that is statistically significant, the intervention is considered to have had success.
Studies often involve an experimental group that participates in the intervention and a control group that does not. When the behavior of the experimental group shows improvement when compared to the behavior of the control group, the intervention is considered to have some success (provided that the change is statistically significant).
So. How good is evidence-based practice? The studies tell us something about this. It seems there are always those who benefit significantly from the intervention, others who benefit somewhat, and yet others who benefit little or not at all.
In other words, evidence-based practice is not always successful to the extent that people, clients and professionals, might wish.
But the studies rarely tell us about side effects. They measure the target behavior, but they rarely if ever look at other things – feelings, emotions, beliefs, values, self-esteem, or behaviors other than the target behavior. So we do not know whether or not the evidence-based intervention may be contributing to problems other than the one with which the study is concerned.
Although feelings, emotions, beliefs, values, self-image, self-esteem, and self-confidence are difficult to observe and measure, they contribute quite a bit to behavior. Feelings, emotions, beliefs, and values motivate behavior and determine how consequences affect behavior. For example, money is not a motivator or reinforcement for the behavior of young children who have not yet learned to value money.
Meanwhile, self-image, self-esteem, and self-confidence have a significant effect on children's willingness to attempt new and different behaviors. Typical behavioral interventions – coercive reward and punishment strategies – can affect children's self-image and self-esteem. Rewards, and especially punishments, can be demeaning at times, as when children perceive that adults think they are too lazy or too stupid to do the right thing without a reward (bribe) or the threat of punishment. And when adults think children deserve punishment, it is especially damaging to children's self-image and self-esteem. Long restrictions can be especially damaging to developing children, who are punished, and therefore deserving of punishment, for the duration of the restriction.
What makes different children respond differently to evidence-based interventions? First, different children are...different. They are physically different. They have different strengths and limitations. They have different needs, interests, and goals. And so many other differences.
Second, different children exist in different social circumstances. Even when the environment of children would seem to be similar, as when siblings have the same parents, their relationships with their parents and with each other, are different for each of them. When children live in the same unit in a residential setting, their relationships with staff and peers are different. For some children, relationships with adults are more important; for others, relationships with peers are more important.
What about children who benefit little from our evidence-based intervention? Do we conclude that they are not trying hard enough and pronounce them uncooperative or unmotivated? Do we look for strategies to coerce their cooperation? Or strategies to motivate them? The ever popular reward strategies? Or the even more popular punishment strategies? Do we look for diagnoses to explain why they do not respond to our proven intervention? Do we look for medications to help? Do we discharge them as “untreatable in this environment?”
Or do we take all that we know about human development and behavior in the social environment and the strengths and challenges and social realities of individual children and devise other interventions?
If professionals rely too heavily on evidence-based practice, they might not be providing the best possible service for the children and families with whom they work. Indeed, it is possible that they may be contributing to some other problems. Some children might need something in addition to an evidence-based intervention, or something entirely different.
Consider children who have serious problems with temper control. There are numerous cognitive behavioral interventions that have been shown to have merit. Should one of these interventions eliminate the problem, all is well. Should such an intervention reduce outbursts from several times a week to only a few times a month, what then? Obviously, the intervention has had a beneficial effect. Do we stop here? Are we finished? Have we done all that can be expected? All that is necessary? Are two or three tantrums a month acceptable?
Do we have a child who so misunderstands his social environment and the motivations of others that he perceives a serious threat where none exists, becoming so enraged so quickly at times that he is unable to employ self-control techniques that he seemed to have mastered? Do we have a child who has learned that it is wrong for her to become angry and then feels that she deserves to be punished for allowing herself to become angry?
Or do we have a child who is chronically angry? A child for whom anger is a mood? A child who successfully suppresses a pervasive anger most of the time, until for some reason defenses fail, releasing pent up rage that is out of all proportion to what is happening in the moment.
Do we need to provide some education and experiences in social situations in addition to cognitive behavioral interventions? Do we need to provide specific therapies? Do we need to provide something else? A safe environment? Relationships with caring, understanding people? A little hope? Or something else entirely?
Before qualifying as evidence-based practice, interventions are no more than someone’s informed decision about what might work. Many such interventions do not make it into the literature, no matter how effective. Some professionals simply do not have the time or the resources to conduct a study, write it up, and submit it for publication.
Evidence-based practice may be indicated, perhaps even necessary at times. It may not always be sufficient. Professionals should never allow evidence-based practice to limit their goals, objectives, creativity, or innovation. Rather, they should continue searching and innovating based on their knowledge and understanding not only of human development and behavior in the social environment but also of the children with whom they work and the environments in which those children live, to achieve the outcomes they both seek. Individual children and the social environments in which they live are too complex to rely on one-size fits all prescriptions. We should not rest until we have achieved success.
The evidence that matters is not the evidence from some study. The evidence that matters is the evidence from the children with whom we work. It doesn’t matter how well the intervention did with someone else’s kids in some other setting. The important evidence is how the intervention works for the kid or kids with whom we are working.
But we must be careful, even with this evidence. We should not conclude that we have found the answer just because the evidence shows that our intervention contributed to some incremental change that proves to be statistically significant using some statistical analysis. Reducing temper tantrums from several per week to two or three per month is indeed significant. It is not sufficient. We need to look further.
So, should we rely on evidence-based practice? I think we should always consider using evidence-based practice. I don’t think we should rely on it too heavily. We can and should do more. Our children and families deserve our best.
