Listen to this
Summary
Over the past decade researchers have identified intervention strategies
and program models that reduce delinquency and promote pro-social
development. Preventing delinquency, says Peter Greenwood, not only
saves young lives from being wasted, but also prevents the onset of
adult criminal careers and thus reduces the burden of crime on its
victims and on society. It costs states billions of dollars a year to
arrest, prosecute, incarcerate, and treat juvenile offenders. Investing
in successful delinquency-prevention programs can save taxpayers seven
to ten dollars for every dollar invested, primarily in the form of
reduced spending on prisons.
According to Greenwood, researchers have identified a dozen “proven” delinquency-prevention programs. Another twenty to thirty “promising” programs are still being tested. In his article, Greenwood reviews the methods used to identify the best programs, explains how program success is measured, provides an overview of programs that work, and offers guidance on how jurisdictions can shift toward more evidence-based practices.
The most successful programs are those that prevent youth from engaging in delinquent behaviors in the first place. Greenwood specifically cites home-visiting programs that target pregnant teens and their at-risk infants and preschool education for at-risk children that includes home visits or work with parents. Successful school-based programs can prevent drug use, delinquency, anti-social behavior, and early school drop-out.
Greenwood also discusses community-based programs that can divert first-time offenders from further encounters with the justice system. The most successful community programs emphasize family interactions and provide skills to the adults who supervise and train the child.
Progress in implementing effective programs, says Greenwood, is slow. Although more than ten years of solid evidence is now available on evidence-based programs, only about 5 percent of youth who should be eligible participate in these programs. A few states such as Florida, Pennsylvania, and Washington have begun implementing evidence-based programs. The challenge is to push these reforms into the mainstream of juvenile justice.
Introduction
There are many reasons to prevent juveniles from becoming delinquents or
from continuing to engage in delinquent behavior. The most obvious
reason is that delinquency puts a youth at risk for drug use and
dependency, school drop-out, incarceration, injury, early pregnancy, and
adult criminality. Saving youth from delinquency saves them from wasted
lives.1 But there are other reasons as well.
Most adult criminals begin their criminal careers as juveniles. Preventing delinquency prevents the onset of adult criminal careers and thus reduces the burden of crime on its victims and on society. Delinquents and adult offenders take a heavy toll, both financially and emotionally, on victims and on taxpayers, who must share the costs. And the cost of arresting, prosecuting, incarcerating, and treating offenders, the fastest growing part of most state budgets over the past decade, now runs into the billions of dollars a year. Yet recent analyses have shown that investments in appropriate delinquency-prevention programs can save taxpayers seven to ten dollars for every dollar invested, primarily in the form of reduced spending on prisons.2
The prospect of reaping such savings by preventing delinquency is a new one. During the early 1990s, when crime rates had soared to historic levels, it was unclear how to go about preventing or stopping delinquency. Many of the most popular delinquency-prevention programs of that time, such as DARE, Scared Straight, Boot Camps, or transferring juveniles to adult courts, were ineffective at best. Some even increased the risks of future delinquency.3
Only during the past fifteen years have researchers begun clearly identifying both the risk factors that produce delinquency and the interventions that consistently reduce the likelihood that it will occur. Some of the identified risk factors for delinquency are genetic or biological and cannot easily be changed. Others are dynamic, involving the quality of parenting, school involvement, peer group associations, or skill deficits, and are more readily altered. Ongoing analyses that carefully monitor the social development of cohorts of at-risk youth beginning in infancy and early childhood continue to refine how these risk factors develop and interact over time.4
Fairly strong evidence now demonstrates the effectiveness of a dozen or so “proven” delinquency-prevention program models and generalized strategies.5 Somewhat weaker evidence supports the effectiveness of another twenty to thirty “promising” programs that are still being tested. Public agencies and private providers who have implemented proven programs for more than five years can now share their experiences, some of which have been closely monitored by independent evaluators.6 For the first time, it is now possible to follow evidence-based practices to prevent and treat delinquency.
In this article, I discuss the nature of evidence-based practice, its benefits, and the challenges it may pose for those who adopt it. I begin by reviewing the methods now being used to identify the best programs and the standards they must meet. I follow with a comprehensive overview of programs that work, with some information about programs that are proven failures. I conclude by providing guidance on how jurisdictions can implement best practices and overcome potential barriers to successful implementation of evidence-based programs.
Determining what works
Measuring the effects of delinquency-prevention programs is challenging
because the behavior the programs attempt to change is often covert and
the full benefits extend over long periods of time. In this section, I
review the difficulties of evaluating these programs and describe the
evaluation standards that are now generally accepted within this field.
Evaluation methods and challenges
For more than a century, efforts to prevent delinquency have been guided
more by the prevailing theories about the causes of delinquent behavior
than by whether the efforts achieved the desired effects. At various
times over the years, the primary causes of delinquency were thought to
be the juvenile’s home, or neighborhood, or lack of socializing
experiences, or lack of job opportunities, or the labeling effects of
the juvenile justice system.7 The preventive strategies promoted by
these theories included: removal of urban children to more rural
settings, residential training schools, industrial schools, summer
camps, job programs, and diversion from the juvenile justice system.
None turned out to be consistently helpful. In 1994 a systematic review,
by a special panel of the National Research Council, of rigorous
evaluations of these strategies concluded that none could be described
as effective.8
Estimating the effects of interventions to prevent delinquency – as with any developmental problem – can be difficult because it can take years for their effects to become apparent, making it hard to observe or measure these effects. The passage of time cuts both ways. On the one hand, interventions in childhood may have effects on delinquency that are not evident until adolescence. Likewise, interventions during adolescence may reap benefits in labor force participation only in young adulthood. On the other hand, an intervention may initially lessen problem behavior in children only to have those effects diminish over time.
In addition to these complications, two other problems make it difficult to identify proven or promising delinquency-prevention programs. The first is design flaws in the strategies used by researchers to evaluate the programs. The second is inconsistency in the evaluations, which makes comparison nearly impossible.
The first problem limiting progress in identifying successful program strategies is the weak designs found in most program evaluations. Only rarely do juvenile intervention programs themselves measure their outcomes, and the few evaluations that are carried out do not usually produce reliable findings.9
The “gold standard” for evaluations in the social sciences – experiments that compare the effects on youths who have been assigned randomly to alternative interventions – are seldom used in criminal justice settings.10 Although such rigorous designs, along with long-term follow-up, are required to assess accurately the lasting effect of an intervention, they are far too expensive for most local agencies or even most state governments to conduct. Such evaluations are thus fairly rare and not always applied to the most promising programs.
Instead, researchers typically evaluate delinquency-prevention programs using a quasi-experimental design that compares outcomes for the experimental treatment group with outcomes for some nonrandom comparison group, which is claimed to be similar in characteristics to the experimental group. According to a recent analysis of many evaluations, research design itself has a systematic effect on findings in criminal justice studies. The weaker the design, the more likely the evaluation is to report that an intervention has positive effects and the less likely it is to report negative effects. This finding holds even when the comparison is limited to randomized studies and those with strong quasi-experimental designs.11
The second problem in identifying successful programs is that a lack of consistency in how analysts review the research base makes it hard to compare programs. Different reviewers often come to very different conclusions about what does and does not work. They produce different lists of “proven” and “promising” programs because they focus on different outcomes or because they apply different criteria in screening programs. Some reviews simply summarize the information contained in selected studies, grouping evaluations together to arrive at conclusions about particular strategies or approaches that they have defined. Such reviews are highly subjective, with no standard rules for choosing which evaluations to include or how their results are to be interpreted. More rigorous reviews use meta-analysis, a statistical method of combining results across studies, to develop specific estimates of effects for alternative intervention strategies. Finally, some “rating or certification systems” use expert panels or some other screening process to assess the integrity of individual evaluations, as well as specific criteria to identify proven, promising, or exemplary programs. These reviews also differ from each other in the particular outcomes they emphasize (for example, delinquency, drug use, mental health, or school-related behaviors), their criteria for selection, and the rigor with which the evidence is screened and reviewed. Cost-effectiveness and cost-benefit studies make it possible to compare the efficiency of programs that produce similar results, allowing policymakers to achieve the largest possible crime-prevention effect for a given level of funding.
Evolving standards for measuring effectiveness
Researchers have used a variety of methods to help resolve the issues of
weak design and lack of consistency. The most promising approach to date
is Blueprints for Violence Prevention, an intensive research effort
developed by the Center for the Study and Prevention of Violence at the
University of Colorado to identify and promote proven programs. For
Blueprints to certify a program as proven, the program must demonstrate
its effects on problem behaviors with a rigorous experimental design,
show that its effects persist after youth leave the program, and be
successfully replicated in another site.12 The current Blueprints
website (www.colorado.edu/cspv/blueprints/) lists eleven “model” programs and twenty “promising” programs. The design, research evidence,
and implementation requirements for each model are available on the
site.
Other professional groups and private agencies have developed similar processes for producing their own list of promising programs.13 The programs identified on these lists vary somewhat because of differences in the outcomes on which they focus and in the criteria they use for screening, though the lists have a good deal of consistency as well. But these certified lists do not always reveal how often they are updated and do not report how a program fares in subsequent replications after it has achieved its place on the list.
Another effective way to compare programs is through a statistical meta-analysis of program evaluations. In theory, a meta-analysis should be the best way to determine what to expect in the way of effectiveness, particularly if it tests for any effect of timing, thus giving more weight to more recent evaluations. Once the developers of a program have demonstrated that they can achieve significant effects in one evaluation and a replication, the next test is whether others can achieve similar results. The best estimate of the effect size that a new adopter of the model can expect to achieve is some average of that achieved by others in recent replications. Meta-analysis is the best method for sorting this out.
The first meta-analysis that focused specifically on juvenile justice was published by Mark Lipsey in 1992.14 Lipsey’s analysis did not identify specific programs but did begin to identify specific strategies and methods that were more likely to be effective than others. Lipsey continued to expand and refine this work to include additional studies and many additional characteristics of each study.15
Meta-analysis is also the primary tool used by academics and researchers who participate in the Campbell Collaboration (C2), an offshoot of the Cochran Collaboration, which was established to conduct reviews of “what works” in the medical literature. The goal of C2, with its potentially large cadre of voluntary reviewers, is to become the ultimate clearinghouse of program effectiveness in all areas of social science, including juvenile justice. Progress, however, has been slow so far.
The C2 Criminal Justice Coordinating Group has concluded that it is unrealistic to restrict systematic reviews in their field to randomized experimental studies, however superior they may be, because so few exist.16 A Research Design Policy Brief prepared for the C2 Steering Committee by William Shadish and David Myers proposes, however, that systematic reviews be undertaken only when randomized experiments are available to be included in the review and that estimates of effects for randomized and nonrandomized evaluations be presented separately in all important analyses when both types of studies are included.17
Cost-benefit analysis
Yet another way to identify promising programs is to use cost-benefit
analysis to evaluate the relative efficiency of alternative approaches
in addressing a particular problem. In 1996 a team at RAND published a
study showing that parenting programs and the Ford Foundation-sponsored
Quantum Opportunities Program reduce crime much more cost-effectively
than long prison sentences do.18 Implementing any program, of course,
has some costs, which can be measured against its benefits. If a program
reduces future crimes, it also reduces the cost of any investigations,
arrest and court processing, and corrections associated with the crimes.
Systematic cost-benefit studies of alternative delinquency-prevention
and correctional intervention programs conducted by the Washington State
Institute for Public Policy (WSIPP), the legislative analysis group
serving the state legislature, show that many proven programs pay for
themselves many times over.19
Comparing the cost-effectiveness of alternative crime-prevention strategies requires decisions about which benefits or savings to consider. All programs must be compared on an equal footing. Some analyses consider only savings within the criminal justice system. Others view this issue more broadly: costs must be covered and savings are savings no matter where in government they arise.20 This broader approach requires collecting data reflecting the effect of an intervention on all government spending. David Olds' Nurse Home Visiting Program, for example, is not cost-effective as a delinquency-prevention program alone, but when crime-reduction benefits for both the mother and child are combined with reduced welfare and schooling costs, benefits exceed costs by several orders of magnitude.21
A final financial issue is whether to include the benefits of reduced crime to potential victims, their families, and friends. The criminal justice system has lagged behind fields such as engineering, medicine, public health, and environmental protection in efforts to monetize benefits. Victim surveys provide fairly good estimates of direct out-of-pocket costs such as the value of lost or damaged property, medical costs, and lost wages. These direct costs, however, are only a small share of the total costs to victims imposed by crimes against persons.22 The question is how to estimate the indirect costs of pain and suffering, security expenses, and restricted lifestyle, which can be quite large for some more serious crimes.
Economists argue about the proper way to place a monetary value on “quality-of-life” items like clean air or a safer environment. One way is to find markets that reflect these values, such as the wage premium demanded by workers engaged in dangerous professions. Beyond the police, however, few professions carry a substantial risk of being robbed, raped, or shot. Furthermore, the wages for those who engage in such professions (drug dealers, smugglers, prostitutes) are difficult to assess.
Economists have also used jury awards to get estimates of the total costs to victims for various crimes as shown in table 1.23 In the WSIPP’s cost-effectiveness analysis of proven and existing programs, estimates of savings to victims are based on such analysis and are several orders of magnitude larger than estimates of the savings to taxpayers alone.24
If the only question is how much to spend on delinquency prevention, then critics of the indirect-costs-to-victims studies are right to point out the heroic assumptions required to extrapolate their data from jury verdicts to ordinary crime victims, as well as the huge margins of error involved. But if the purpose is to decide which of several proposed interventions is most cost-effective, then victim costs need not be considered. Instead the preferred metric becomes the number of crimes prevented for a specific amount invested in either program, assuming that both programs prevent the same relative mix of crime types. Victim costs would, however, become relevant if one approach prevented mainly property crimes while the other prevented mainly crimes against persons.
At this fledgling stage in developing appropriate victim cost data for use in cost-effectiveness studies, there are not many choices. Analysts can ignore victim costs altogether and seriously undervalue interventions that prevent crimes, particularly violent crimes. They can use the victim cost data as analysts at the WSIPP have. Or they can use the even higher estimates for the social costs of crime produced by more recent contingent valuation studies.25 Until researchers reach a consensus on this issue, the prudent approach is to use all three methods, reporting separately the results from each. In that way the readers of the study will be free to come to their own conclusions regarding the appropriate method.
For any outcome of interest, it is necessary to decide how to identify and describe effects, how long a follow-up period to use, and how widely to search for effects. The ideal would be to identify all effects of an intervention, no matter where or how far in the future they occur. Many evaluations, however, report outcomes only until participating youth exit from the program.
Reporting effects of an intervention
The simplest way to report effects of an intervention is the
straightforward binary method of statistical significance. Are the
effects significant as measured by standardized statistical tests, using
a sufficiently rigorous research design? This outcome measure,
traditionally used by academic reviewers of research,26 was used both by
David Hawkins and Richard Catalano27 in compiling the list of promising
programs for their Communities That Care program and by Lawrence Sherman
and his colleagues in their evaluation of prevention programs for the
U.S. Department of Justice.28
Reporting only the binary outcomes provided by significance tests, however, fails to capture large differences in the size of effects known to exist between interventions. The standard measure adopted by many reviewers is the effect size, typically defined as the difference between the treatment and control group means, on the selected recidivism measure, standardized (divided) by their pooled standard deviation. This standardized mean difference effect size is commonly used to represent the findings of experimental comparisons in meta-analyses and other quantitative studies.29
In the delinquency field, where the environment and situational factors appear to play a critical role in shaping behavior, some programs have been shown to produce significant effects while youth are participating in them, but no effects after they leave the program. This phenomenon of transient behavioral change has led the Blueprints Project to require evidence that effects persist after a youth leaves the program before it can appear on its list of proven models.30
The issue of how far into the future to measure effects depends on what the future is expected to hold. If many of the benefits of a program are not expected to be evident for many years, then observations will be required until their presence is verified. If current trends and tendencies can be assumed to continue uninterrupted, then shorter follow-up periods will do. The Washington State Institute for Public Policy uses ten years for the cut-off point in their estimates of program benefits.31 But some programs, such as the Perry Preschool, are cost-effective only when benefits such as reduced crime and enhanced income are considered more than a decade after youth leave the program.32
In summary, defining successful programs is challenging, both because of design flaws in many research studies and because comparing inconsistent findings is difficult. But some metric must be designed to allow jurisdictions to begin to implement programs that have been proven effective. Blueprints is the most promising of these techniques, though others such as meta-analysis hold promise.
What works and what doesn’t
For anyone in a position to decide which programs should be continued or
enhanced, which should be scrapped, and which new programs should be
adopted, the ultimate question is “what works” and “how well” does it
work? The answers to these questions now come in two distinctive
categories. One is “generic,” including a number of generalized
strategies and methods that have been tried by various investigators in
different settings. Parent training, preschool, behavior modification,
and group therapy all fall into this category. The other category
includes the “brand name” programs such as Functional Family Therapy and
Multisystemic Therapy. These are programs that have been developed by a
single investigator or team over a number of years and proven through
careful replications, supported by millions of dollars in federal
grants. The generic methods are identified by meta-analysis and
represent the efforts of independent investigators, each testing
particular versions of the method. The brand name programs have met the
criteria established by various review groups for identifying proven
programs.
These two methods overlap in an interesting way. Most of the Blueprints model programs represent an outstanding performer within a larger category. The four experimental trials of Functional Family Therapy (FFT), shown in darker gray in figure 1, represent about 10 percent of all family therapy program evaluations. Figure 1 contains a histogram, plotted by Mark Lipsey, showing the number of family therapy evaluations demonstrating various effect sizes.33 Although a number of evaluations found negative effects, the average for all is well above zero. None of the four FFT evaluations found negative effects, and three are well toward the upper end of the distribution. Figure 1 indicates that family therapy works as a generalized approach and that FFT works even better, when done correctly. Similarly, the other Blueprints models, on average, produce larger effect sizes than the average for the generic category of which they are a part.
The most recent reviews, meta-analyses, certified lists, and cost-benefit analyses provide a variety of perspectives and wealth of information regarding what does and does not work in preventing delinquency. At the very top of the promising program pyramid is the small group of rigorously evaluated programs that have consistently demonstrated significant positive effects and developed effective strategies for helping others to replicate their model and achieve similar results. At the bottom are the vast numbers of programs that have never been evaluated. In the middle are those for which there is some evidence to support their claims of effectiveness in at least one site. Most of the interventions that have been shown to prevent the onset of or continued involvement in delinquency were first developed by researchers or academics outside of the juvenile justice field to deal with other problem behaviors such as child abuse, misbehavior in school, school failure, drug or alcohol abuse, or failure in foster-care placement. However, because all these targeted behaviors are closely related, and often antecedent to delinquency,34 programs developed to prevent them have also turned out to prevent delinquency.
The research is strongest and most promising for school- and community-based interventions that can be used before the demands of public safety require a residential placement. In this area a number of well-specified, proven, cost-effective programs have emerged. For youth in custodial settings there is less research to draw on and what there is suffers from serious methodological problems. Still, some findings appear to hold up across various settings. In this section I review the evidence regarding “what works” in delinquency prevention and intervention.
To categorize the strength of the evidence in support of a particular program or strategy, I have created a descending seven-level scale. The various levels incorporate information about evaluation design, effect size, number of replications, and cost-effectiveness. The first level, preferred programs and strategies, includes models that are proven effective according to the Blueprints standards or rigorous meta-analysis and are found to return significantly more in taxpayer savings than they cost. On the second level, proven programs and strategies meet the three Blueprints qualifications for model programs or are found to be effective by rigorous meta-analysis. On the third level, provisional programs are supported by one evaluation with a strong research design showing evidence of a crime-prevention effect. On the fourth level, promising programs do not meet provisional standards, but the balance of their evidence points toward positive effects. Programs on the remaining three levels are increasingly less strong. Potentially promising programs are those without evidence of effectiveness but whose design incorporates promising practices. Ineffective programs are those shown to have no effect or negative effects. And, finally, unproven programs include all the rest.
Table 2 classifies each listed program or strategy according to this scale in the column headed evidence-based status.
Table 2 can be viewed HERE
I begin this review by focusing on efforts to prevent youth from engaging in delinquent behaviors in the first place, then discuss community-based programs that can divert first-time offenders from further encounters with the justice system, as a condition of probation or parole, or facilitate reentry for youth after an institutional placement, and conclude with programs for youth in custodial settings.
All of the programs described below are listed in table 2 along with the source of their rating, their effect on crime outcomes, the number of evaluations on which their effect size is based, the cost per youth, the estimated government savings and victim benefits per youth treated, and the ratio of their estimated savings divided by their costs.
Prevention programs
Primary prevention programs target the general population of youth and
include efforts to prevent smoking, drug use, and teen pregnancy.
Secondary prevention programs target youth at elevated risk for a
particular outcome, such as delinquency or violence, a group that might
include those in disadvantaged neighborhoods, those struggling in
school, or those exposed to violence at home.
The first opportunity for prevention is with pregnant teens or at-risk children in early childhood. The preeminent program in this category is David Olds' Nurse Home Visitation Program, which trains and supervises registered nurses as the home visitors. This program is found on just about every list of promising strategies based on the strength of evidence regarding its significant long-term effects and portability. It attempts to identify young, poor, first-time mothers early in their pregnancy. The sequence of approximately twenty home visits begins during the prenatal period and continues over the first two years of the child's life, with declining frequency. In addition to providing transportation and linkage to other services, the nurse home visitors follow a detailed protocol that provides child-care training and social skills development for the mother.
A fifteen-year follow-up of the Prenatal/ Early Infancy Project in Elmira, New York, showed that the nurse home visits significantly reduced child abuse and neglect in participating families, as well as arrest rates for the children and mothers.35 The women who received the program also spent much less time on welfare; those who were poor and unmarried had significantly fewer subsequent births.
Many less costly and less structured home visiting models have been tested, using social workers or other professionals, rather than nurses, but none has achieved the same success or consistency as the Olds program with nurses.36 The Olds model, now called the Nurse Family Partnership, has been successfully evaluated in several sites and is now replicated in more than 200 counties and many countries.
For slightly older children, preschool education for at-risk three- and four-year-olds is an effective prevention strategy, particularly when the program includes home visits or work with parents in some other way. The Perry Preschool in Ypsilanti, Michigan, is the most well-evaluated model.
Numerous school- or classroom-based programs have proven effective in preventing drug use, delinquency, anti-social behavior, and early school drop-out, all behaviors that can lead to criminal behavior.37 The programs vary widely in their goals, although they share some common themes: collaborative planning and problem-solving involving teachers, parents, students, community members, and administrators; grouping of students into small self-contained clusters; career education; integrated curriculum; and student involvement in rule-setting and enforcement, and various strategies to reduce drop-out.
For example, the Bullying Prevention Program was developed with elementary and junior high school students in Bergen, Norway. The program involves teachers and parents in setting and enforcing clear rules against bullying. Two years after the intervention, bullying problems had declined 50 percent in treated schools. Furthermore, other forms of delinquency declined as well, and school climate improved.38 The Bullying Prevention Program is one of the eleven Blueprints model programs and is listed as promising by the Surgeon General.
Multiple evaluations of Life Skills Training, a classroom-based approach to substance abuse prevention, have shown it to reduce the use of alcohol, cigarettes, and marijuana among participants. The reductions in alcohol and cigarette use are sustained through the end of high school.39 Life Skills Training is listed as a model program by both Blueprints and the Surgeon General and by most other lists of proven programs.40 The program has been widely disseminated throughout the United States over the past decade with funding from government agencies and private foundations.
Project STATUS is another school-based program designed to improve junior and senior high school climate and reduce delinquency and drop-out. The two primary strategies used are collaborative efforts to improve school climate and a year-long English and social studies class focused on key social institutions. An evaluation of Project STATUS found less total delinquency, drug use, and negative peer pressure and greater academic success and social bonding.41 Project STATUS is rated promising by Blueprints.
The School Transitional Environmental Program (STEP) aims to reduce the complexity of school environments, increase peer and teacher support, and decrease student vulnerability to academic and emotional difficulties by reducing school disorganization and restructuring the role of the homeroom teacher. It specifically targets students at greatest risk for behavioral problems. STEP students are grouped in homerooms where the teachers take on the additional role of guidance counselor. All project students are assigned to the same core classes. Evaluations have demonstrated decreased absenteeism and drop-out, increased academic success, and more positive feelings about school.42 Both Blueprints and the Surgeon General consider STEP a promising program.
Community-based interventions
Delinquency-prevention programs in community settings can be created for
various purposes such as diverting youth out of the juvenile justice
system, serving youth placed on informal or formal probation, or serving
youth on parole who are returning to the community after a residential
placement. Settings can range from individual homes, to schools, to teen
centers, to parks, to the special facilities of private providers. They
can involve anything from a one-hour monthly meeting to intensive family
therapy and services.
The most successful programs are those that emphasize family interactions, probably because they focus on providing skills to the adults who are in the best position to supervise and train the child.43 More traditional interventions that punish or attempt to frighten the youths are the least successful. For example, for youth on probation, two effective programs are family-based interventions designated as proven by Blueprints and the Surgeon General: Functional Family Therapy and Multisystemic Therapy. Functional Family Therapy (FFT) targets youth aged eleven to eighteen facing problems with delinquency, substance abuse, or violence. The program focuses on altering interactions between family members and seeks to improve the functioning of the family unit by increasing family problem-solving skills, enhancing emotional connections, and strengthening parents' ability to provide appropriate structure, guidance, and limits for their children.44 It is a relatively short-term program that is delivered by individual therapists, usually in the home setting. Each team of four to eight therapists works under the direct supervision and monitoring of several more experienced therapist/trainers. The effectiveness of the program has been demonstrated for a wide range of problem youth in numerous trials over the past twenty-five years, using different types of therapists, ranging from paraprofessionals to trainees, in a variety of social work and counseling professions. The program is well documented and readily transportable.
Multisystemic Therapy (MST), also a family-based program, is designed to help parents deal effectively with their youth’s behavior problems, including engaging with deviant peers and poor school performance. To accomplish family empowerment, MST also addresses barriers to effective parenting and helps family members build an indigenous social support network. To increase family collaboration and generalize treatment, MST is typically provided in the home, school, and other community locations. Master-level counselors provide fifty hours of face-to-face contact and 24/7 crisis intervention over four months.
MST works with an individual family for as long a period as FFT does, but it is more intensive and more expensive. In addition to working with parents, MST will locate and attempt to involve other family members, teachers, school administrators, and other adults in supervising the youth. Unlike FFT therapists, MST therapists are also on call for emergency services. Evaluations demonstrate that MST is effective in reducing re-arrest rates and out-of-home placements for a wide variety of problem youth involved in both the juvenile justice and social service systems.45
The third program in this category, Intensive Protective Supervision (IPS), targets non-serious status offenders. Offenders assigned to IPS are closely monitored by counselors who carry reduced caseloads and interact more extensively with the youth and their families than traditional parole officers. The counselors make frequent home visits, provide support for parents, develop individualized service plans, and arrange for professional or therapeutic services as needed. An evaluation of the program found that youth assigned to IPS were less likely to be referred to juvenile court during supervision or during a one-year follow-up period and were more likely to have successfully completed treatment than youth assigned to regular protective supervision.46 IPS is listed as promising by both Blueprints and the Surgeon General. Other effective strategies for youth on probation include cognitive-behavioral therapy, family counseling, mentoring, tutoring, drug and alcohol therapy, interpersonal skills training, and parent training.
Community-based programs that focus on the individual offender rather than on the family are much less successful. Intensive supervision, surveillance, extra services, and early release programs, for example, have not been found effective. Ineffective probation programs and strategies include intensive supervision, early release, vocational training, bringing younger offenders together for programming, and deterrence approaches such as Scared Straight.
Institutional settings
Juvenile courts, like criminal courts, function as a screening agent for
the purpose of sanctions and services. Juvenile offenders' needs for
treatment must be balanced against the demands of accountability
(punishment) and community safety. Only a fraction of the cases reaching
any one stage of the system are passed on to the next stage. Out of all
the juveniles arrested in 1999, only 26 percent were adjudicated
delinquent and only 6.3 percent were placed out of their homes.47 Even
among those arrested for one of the more serious Crime Index offenses,
only 35 percent were adjudicated delinquent and only 9.2 percent were
placed out of their homes. This pattern of case dispositions reflects
the juvenile court’s preference for informal rather than formal
dispositions and the understanding that most programs work better in
community, rather than institutional, settings.
Nevertheless juvenile courts will place youth in more secure custodial placements if the home setting is inappropriate and a more suitable community placement is unavailable or if the youth poses a public safety risk. In these two instances placement in a group setting is more likely.
Youths who are placed out of their homes are referred to a wide variety of group homes, camps, and other residential or correctional institutions. Three generalized program strategies improve institutional program effectiveness. One is focusing on dynamic or changeable risk factors – low skills, substance abuse, defiant behavior, relationships with delinquent peers. The second is individually tailoring programs to clients – needs using evidence-based methods.48 The third is focusing interventions on higher-risk youth, where the opportunity for improvement and consequences of failure are both the largest. These three characteristics provide the basis for the Correctional Program Inventory (CPI), a program assessment instrument now being used by Ed Latessa and several colleagues at the University of Cincinnati to rate the quality of programming in individual correctional facilities.49
Finally, certain program characteristics that are independent of the specific interventions used have been shown to improve outcomes. The integrity with which the program is implemented and maintained is one such characteristic, as is longer duration of treatment. Well-established programs are more effective than newer programs. Programs that support mental health issues are more successful than those that focus on punishment, so treatment programs administered by mental health professionals are more effective than similar programs administered by regular correctional staff.
Generally, programs that focus on specific skills issues such as behavior management, interpersonal skills training, family counseling, group counseling, or individual counseling have all demonstrated positive effects in institutional settings.
Among the specific program models that work well with institutionalized youth are cognitive-behavioral therapy, aggression replacement training, and family integrated transition. Cognitive-behavioral therapy (CBT) is a time-limited approach to psychotherapy that uses skill building – instruction and homework assignments – to achieve its goals. It is based on the premise that it is people’s thoughts about what happens to them that cause particular feelings, rather than the events themselves, and its goal is to change thinking processes. It uses various techniques to learn what goals clients have for their lives and to improve skills that can help them achieve those goals.50
Aggression replacement training also emphasizes focusing on risk factors that can be changed. It is a cognitive-behavioral intervention with three components. The first is “anger control,” which teaches participants what triggers their anger and how to control their reactions. The second is “behavioral skills,” which teach a series of pro-social skills through modeling, role playing, and performance feedback. The third is “moral reasoning,” in which participants work through cognitive conflict in dilemma discussion groups.51
Family integrated transitions (FIT) also focuses on tackling dynamic risk factors – substance abuse, mental health issues, and community reentry from residential placement. Developed for the State of Washington, the program uses dialectical behavioral therapy (another form of cognitive-behavioral therapy), MST, relapse prevention, and motivational enhancement therapy. It was designed to help youth with mental health or chemical dependency issues who are returning to the community following a residential placement. The only evaluation of the program to date showed positive results.52
For youth who have traditionally been placed in group homes – homes that are usually licensed to care for six or more youths who need to be removed from their home for an extended period, but do not pose a serious risk to themselves or others – the preferred alternative is Multidimensional Treatment Foster Care (MTFC). In MTFC, community families are recruited and trained to take one youth at a time into their home. MTFC parents are paid a much higher rate than regular foster parents but have additional responsibilities. One parent, for example, must be at home whenever the child is. Parent training emphasizes behavior management methods to provide youth with a structure and therapeutic living environment. After completing a pre-service training, MTFC parents attend a weekly group meeting run by a case manager for ongoing supervision. Supervision and support are also provided to MTFC parents during daily telephone calls. Family therapy is also provided for biological families. Random assignment evaluations find that arrest rates fall more among participants in the MFTC model than among youth in traditional group homes.53 Although it costs approximately $7,000 more per youth to support MFTC than a group home, the Washington State Institute for Public Policy estimates that MFTC produces $33,000 in criminal justice system savings and $52,000 in benefits to potential crime victims.
Implementing best practices
With more than ten years of solid evidence now available regarding what
does and does not work in preventing juvenile delinquency and reducing
recidivism, jurisdictions should be adopting an evidence-based approach
to implementing new programs. Taking this approach will prevent wasted
lives, save taxpayer dollars, and protect communities from unnecessary
crime victimization.
Cost-benefit studies conducted by the Washington State Institute for Public Policy (WSIPP), summarized in the far right-hand column of table 2, indicate that many evidence- based programs can produce savings on the order of five to ten times their cost. In one case, the Washington State legislature, confronting a projected requirement to build two additional prisons, asked WSIPP to estimate how a substantial increase in spending on evidence-based programs would affect projected prison bed requirements. The analysis, published in 2006, showed that doubling current investments in high-quality programs could eliminate the need for additional prison capacity.54
Before a jurisdiction begins identifying successful programs, it must first determine whether there are any gaps in the service and quality of its existing programs. A service gap indicates a lack of suitable treatment options for a particular type of youth; a quality gap indicates a lack of sufficient evidence-based programming.
After completing the audit, a jurisdiction can follow one of two basic strategies to identify successful programs. It can follow the Blueprints recommendations and replace existing programs with the Blueprints proven models. Or it can use meta-analysis findings as a guide to improve existing programs. The steps involved and financing required for these two approaches are quite different, with the Blueprints approach being the costlier and more intense of the two.
If a jurisdiction opts to implement the Blueprints approach to fill service gaps, it should begin by selecting the program model that best fits both the clients to be served and the capabilities of the agency and staff that will provide the service. In addition to carefully reviewing the Blueprints publication describing the model, the jurisdiction may need to speak with the model developer and other agencies that have adopted it.
The second step is to arrange for training. Most developers of the Blueprints model programs have established organizations to provide training, technical assistance, oversight, and certification to sites desiring to adopt their model. Most require applicants to meet a number of qualifying conditions before being considered for implementation. Initial training fees can range from $20,000 to more than $50,000, and annual licensing fees can cost more than $100,000 a year. Some developers offer training on a regular schedule in one or two locations. Others will send their trainers to the applicant’s site if a sufficient number of staff need to be trained. The waiting period for training may be as long as six to nine months. Once training has been scheduled, the third step is to designate or hire appropriate staff. Many agencies make the mistake of selecting and training staff who are not comfortable with the requirements of the program and do not last long in the job. Some programs require only one type of staff, such as a family therapist, while others require several different types, such as case manager, skills trainer, and family therapist. The fourth step is to “sell” the program to potential customers and agency personnel. Without a strong champion within the host agency, a demanding new program has little chance of ever getting off the ground. The fifth step is to heed the recommendation of most model developers and arrange for ongoing monitoring and feedback, usually by having weekly phone conferences to discuss cases or by reviewing videotapes of project staff in action. The final step, implementing a quality assurance mechanism, usually involves questionnaires or observational rating sheets to assess the fidelity of the program to the original model.
If a jurisdiction opts for the meta-analytic (or Lipsey) approach to improve the effectiveness of its programs, the first step is to identify the programs to be assessed. The second is to identify key elements of each program and compare them with the “best practice” standards identified by meta- analysis. The third step is to determine the average effect size the combination of elements for each program has produced in previous evaluations.55 If the expected effects of a program are small, because it lacks evidence-based elements, an agency can consider adding elements from table 2 that would raise the anticipated effectiveness. For instance, a residential program containing no evidence-based elements can be made more effective by adding cognitive-behavioral therapy or aggression replacement training. Likewise, a community supervision program with no evidence-based elements can be made more effective by adding a family therapy or parent training component.
After selecting an evidence-based program, an agency must adopt and implement a validated risk assessment instrument that can provide a basis for assigning youth to specific programs, for comparing the effectiveness of alternative programs in treating similar youth, and for measuring the progress of individual youth. These instruments are readily available from a number of vendors, some of whom offer training in using the instrument as well as in online data entry and analysis.56
The next step in developing an evidence-based practice is to develop a way to assign youths to the most appropriate program, taking into account all the relative costs and differences in effectiveness of each program. Whenever uncertainty exists about which program particular types of youth should be assigned to, an evaluation should be conducted to determine which of the competing alternatives is best.
Finally, once programs have been implemented, they must be monitored to ensure that they follow the program model as intended. Vendors of many proven programs have developed their own fidelity measurement instruments. Locally developed programs will require local development of such instruments.
The juvenile court is in an excellent position to identify quality and service gaps in the current program mix and to identify programs that are not performing up to their true potential, because it sees other agency’s failures. The records of individual cases that come before the court provide informative case studies of how well the system is performing and where screening, assessment, or programming gaps exists. The court is in the best position to identify where particular types of youth are slipping through the cracks or particular parts of the system need to improve their performance.
Challenges and obstacles to implementing evidence-based
practice
Despite more than ten years of research on the nature and benefits of
evidence-based programs, such programming is the exception rather than
the rule. Only about 5 percent of youth who should be eligible for
evidence-based programs participate in one.57
One reason for the slow progress is the general lack of accountability for performance within the juvenile justice system, or even any ability to measure outcomes. Only rarely does a jurisdiction take delinquency prevention and intervention seriously enough to measure the outcome of its efforts. Rather, it tends to evaluate agencies on how well they meet standards for protecting the health and safety of their charges and preventing runaways or incidents requiring restraints. Without the availability of such data as re-arrests or high school graduation rates, there is little pressure on agency officials to improve their performance.
A second challenge is a lack of funding. Implementing evidence-based programs, especially the Blueprints models, is expensive. Training a single team of therapists and their supervisor can cost more than $25,000. The agency may have to hire new staff that meet higher credentialing standards before start-up, without any revenue to cover their costs. State and local agencies have a hard time finding that kind of funding even in good economic times. Today it is difficult indeed. Even after youth begin being referred to the program, it may still take time for the flow of cases to fully occupy all the staff charged to the program.
To fund start-up activities, some states have set up grant mechanisms, for which local communities compete. Some jurisdictions seek grants from state or federal agencies. Even after an evidence-based program is implemented, it may be hard to find funds to continue its operation. Most of the savings from effective programs accrue to the state in the form of lower corrections costs. If some of these anticipated savings are not passed down to the local entities that must fund the programs, they may have trouble competing for scarce local funding against better-established programs. Some sites have solved this problem by working with state licensing officials to ensure adequate funding and reimbursement rates from Medicaid, Mental Health, or other federally subsidized funding streams.
Another problem faced by agencies that have invested in appropriate evidence-based training for their staff are competitors who claim to be offering the same programming benefits without all of the up-front costs associated with evidence-based programs. The guy who is trying to sell a program that is “a lot like MST” at half what MST Inc. charges is a lot like the guy selling fake Rolex watches on the street corner for a fraction of the usual price. Counterfeits all!
Many of the established evidence-based programs have tried to solve the problem of counterfeits by certifying those that have paid for the training and meet their performance standards. But if local funders are not aware of this legitimacy issue, the better qualified program may lose out in the bidding process to an uncertified cut-rate competitor.
Yet another difficulty for funding agencies or providers who wish to select the most effective evidence-based program is the lack of any standardized system for rating programs. Many other entities besides Blueprints claim to be reliable sources of information on program effectiveness. Some are government agencies; others are housed at universities or within professional organizations. The ratings assigned to programs by some of these organizations sometimes reflect low standards for rating the rigor of evaluation methods or can be biased in favor of programs that the rating organization helped develop or identify.
Another big problem is resistance from staff. It is one thing to sell the director of an agency on the value of evidence-based programs. It is quite another to sell the staff who must adopt the new behaviors, because they have spent their whole career developing their own intuitive approach. When they begin the training they are reluctant to admit that someone at some distant university has come up with a better approach than they have. As in all cognitive-behavioral therapy, there is a certain amount of cognitive dissonance when they start applying the new methods. It just does not feel right. Some staff never overcome this initial resistance and must be shifted to other programs.
A different question is whether an agency has the competence or capacity to take on a Blueprints program. Some of these programs are very demanding in terms of staff qualifications, supervision, information systems, and quality assurance. Often program developers find that an applicant agency needs a year or two to develop the capacity even to begin the first steps of implementing their model.
Conclusions
Over the past decade researchers from a variety of disciplines have
identified or developed an array of intervention strategies and specific
program models demonstrated to be effective in reducing delinquency and
promoting more pro-social development. They have developed a variety of
training methods and other technical assistance to help others replicate
these successful methods. They have accumulated evidence that many of
these programs are cost-effective, returning more than five times their
cost in future taxpayer savings. Evidence also confirms that the general
public overwhelmingly prefers treatment and rehabilitation over
confinement and punishment for juvenile offenders.
Still, only about 5 percent of the youth who could benefit from these improved programs now have the opportunity to do so. Juvenile justice options in many communities remain mired in the same old tired options of custodial care and community supervision. It is as if the major research accomplishments of the past decade had never happened.
In the long run, the authority of science may win out, and the necessary changes will occur. But the authority of science is undermined on a daily basis by those who refuse to distinguish the difference between fact and opinion. Every year of delay in implementing evidence-based reforms consigns another cohort of juvenile offenders to a 50 percent higher than necessary recidivism rate.
Enough states and local communities have begun to take action on this issue that it is now possible to see the pattern of changes and reform that must occur. The evidence-based approach has to be adopted agencywide. It cannot take root and flourish within just one part of the organization, while other units continue on as usual. Either the reform movement will continue to gain converts and momentum, eventually spreading throughout the organization, or the rest of the organization will find a way to kill it.
The concept of evidence-based reform is easiest to sell at the CEO level, where it is just another new concept to grapple with “something that CEOs do every day. The reforms get harder to sell the further you go down in the organization chart. Down on the front lines, underpaid staff, working with difficult youth on a daily basis, develop their own personal styles and methods of dealing with these youth and their issues. Most evidence-based models require staff to make significant change in both style and methods when working with youth and provide quality assurance processes (usually involving surveys of clients) to make sure their performance is up to standards.
The political and institutional changes needed to bring about evidence-based practice require champions in every organization to make them happen. Those in positions of authority for juvenile justice policy must be informed about the evidence-based programs now available to them and about how those programs can help them reduce delinquency rates, ensure safer communities, and reduce government spending.
Policymakers will have to be assisted by experts in evidence-based practices in designing and implementing the reforms required. States will have to create financial incentives for local communities to invest in effective prevention programs, most likely by returning some share of the savings in future corrections costs to counties or local communities. Requests for proposals will have to require evidence-based programming and services, and those buying the services must be able to distinguish evidence-based proposals and programs from other proposals and programs. Providers will eventually be held accountable for the results they achieve.
Practitioners who are going to work with juvenile offenders and at-risk youth will have to be trained and monitored to ensure that they are delivering services in the most appropriate and prescribed manner.
Achieving the consistency and fidelity that effective programs appear to require will necessitate new ways of supervising and managing those who have direct contact with youth and their families. Shifting from a management focus on preventing abuse or infractions to one that empowers employees to provide effective services to their clients is going to be a major struggle.
Those who wish to develop or promote new methods of intervention will
have to learn how to play by the new set of rules and protocols that
have made possible the programming advances of the past decade. Programs
can no longer be promoted for wide-scale dissemination until they have
been proven effective by a rigorous evaluation.
None of these challenges is impossible. Efforts to expand the use of
Blueprints programs in Florida, Pennsylvania, and Washington have been
under way for several years now, with considerable success. Both North
Carolina and Arizona have undertaken efforts in collaboration with Mark
Lipsey to evaluate all their programs.58 Hundreds of communities have
adopted and implemented proven program models and are reaping the
benefits of reduced delinquency and lower system costs.59 The challenge
now is to move beyond these still relatively few early adopters and push
these reforms into the mainstream of juvenile justice.
Notes
1. David P. Farrington and Brandon C. Welsh, Saving Children from a Life of Crime: Early Risk Factors and Effective Interventions (Oxford University Press, 2007).
2. Steve Aos, Marna Miller, and Elizabeth Drake, Evidence-Based Public Policy Options to Reduce Future Prison Construction, Criminal Justice Costs, and Crime Rates (Olympia, Wash.: Washington State Institute for Public Policy, 2006).
3. Lawrence W. Sherman and others, Preventing Crime: What Works, What Doesn–t, What’s Promising (Washington: U.S. Department of Justice, Office of Justice Programs, 1997).
4. Mark W. Lipsey and James H. Derzon, “Predictors of Violent and Serious Delinquency in Adolescence and Early Adulthood,” in Serious and Violent Juvenile Offenders, edited by Rolf Loeber and David P. Farrington (Thousand Oaks, Calif.: Sage, 1998).
5. Delbert Elliot, Blueprints for Violence Prevention (Boulder, Col.: Center for the Study and Prevention of Violence, University of Colorado, 1997).
6. S. A. Mihalick and others, Blueprints for Violence Prevention Replications: Factors for Implementation Success (Boulder, Col.: University of Colorado, 2002).
7. Margaret K. Rosenheim and others, eds., A Century of Juvenile Justice (University of Chicago Press, 2002).
8. Albert J. Reiss and Jeffrey A. Roth, Understanding and Preventing Violence (Washington: National Academy Press, 1993).
9. Sherman and others, Preventing Crime (see note 3).
10. William R. Shadish, Tomas D. Cook, and Donald T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (Boston: Houghton-Mifflin Co., 2002); Sherman and others, Preventing Crime (see note 3).
11. David Weisburd, Cynthia M. Lum, and Anthony Petrosino, “Does Research Design Affect Study Outcomes in Criminal Justice?” Annals of the American Academy of Political and Social Science, 578 (November 2001): 50–70.
12. Elliot, Blueprints for Violence Prevention (see note 5).
13. U.S. Department of Health and Human Services, Youth Violence: A Report of the Surgeon General (Rockville, Md.: U.S. Department of Health and Human Services, 2001); Mihalick and others, Blueprints for Violence Prevention Replications (see note 6).
14. Mark W. Lipsey, “Juvenile Delinquency Treatment: A Meta-Analytic Inquiry into the Variability of Effects,” in Meta-Analysis for Explanation: A Casebook, edited by T. D. Cook and others (New York: Russell Sage Foundation, 1992), pp. 83–127.
15. Mark W. Lipsey and David Wilson, “Effective Intervention for Serious Juvenile Offenders: A Synthesis of Research,” in Serious and Violent Juvenile Offenders, edited by Rolph Loeber and David Farrington (Thousand Oaks, Calif.: Sage, 1998); Mark W. Lipsey, “The Effects of Community-Based Group Treatment for Delinquency: A Meta-Analytic Search for Cross-Study Generalizations,” in K. A. Dodge, T.J.Dishion, and J. E. Lansford, Deviant Peer Influences in Programs for Youth (New York: The Guilford Press, 2006).
16. D. P. Farrington and A. Petrosino, “The Campbell Collaboration Crime and Justice Group,” Annals of the American Academy of Political and Social Science, 578 (2000): 35–49.
17. William Shadish and David Myers, Policy Brief, Campbell
Collaboration Research Design, 2002.
18. P. W. Greenwood and others, Diverting Children from a Life of Crime:
Measuring Costs and Benefits (Santa Monica, Calif.: RAND, 1996).
19. S. Aos and others, The Comparative Costs and Benefits of Programs to Reduce Crime (Olympia, Wash.: Washington State Institute for Public Policy, 2001).
20. M. R. Gold and others, eds., Cost-Effectiveness in Health and Medicine (Oxford University Press, 1996).
21. Lynn Karoly and others, Investing in Our Children (Santa Monica, Calif.: RAND, 1998).
22. P. J. Cook and J. Ludwig, Gun Violence: The Real Costs (Oxford University Press, 2000).
23. Ted Miller, Mark Cohen, and Brian Wiersema, Victims, Costs, and Consequences: A New Look, National Institute of Justice Report, NCJ “155282 (Washington: U.S. Department of Justice, 1996).
24. Aos and others, The Comparative Costs and Benefits of Programs to Reduce Crime (see note 19).
25. Cook and Ludwig, Gun Violence: The Real Costs (see note 22).
26. David Hawkins, Richard Catalano, and David Brewer, “Preventing Serious, Violent and Chronic Offending: Effective Strategies from Conception to Age 6,” in Sourcebook on Serious, Violent and Chronic Juvenile Offenders, edited by J. C. Howell and others (Thousand Oaks, Calif.: Sage, 1995); P. Tolan and N.G.Guerra, What Works in Reducing Adolescent Violence (Boulder: University of Colorado, 1995).
27. D. Hawkins and R. Catalano, Communities That Care (San Francisco: Jossey-Bass, 1992).
28. Sherman and others, Preventing Crime (see note 3).
29. H. Cooper and L. V. Hedges, eds., The Handbook of Research Synthesis (New York: Russell Sage, 1994); Lipsey and Wilson, “Effective Intervention for Serious Juvenile Offenders” (see note 15).
30. Elliot, Blueprints for Violence Prevention (see note 5).
31. Steve Aos and others, Benefits and Costs of Prevention and Early Intervention Programs for Youth (Olympia, Wash.: Washington State Institute for Public Policy, 2004).
32. Karoly and others, Investing in Our Children (see note 21).
33. Presentation at meeting of the Association for the Advancement of Evidence-Based Practice in Cambridge, Maryland, November 2007.
34. Delbert S. Elliot, David Huizinga, and Scott Menard, Multiple Problem Youth: Delinquency, Substance Use and Mental Health Problems (New York: Springer-Verlag, 1989).
35. David Olds and others, Prenatal and Infancy Home Visitation by Nurses, in D. S. Elliot (series ed.), Blueprints for Violence Prevention (Boulder, Colo.: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado“Boulder, 1998).
36. David L. Olds and others, “Effects of Home Visits by Paraprofessionals and by Nurses: Age-Four Follow-up of a Randomized Trial,” Pediatrics, 114, no. 6 (2004): 560–68.
37. Denise C. Gottfredson, David B. Wilson, and Stacy Skroban Najaka, “School-Based Crime Prevention,” in Evidence-Based Crime Prevention, edited by Lawrence W. Sherman (London: Routledge, 2002).
38. D. Limber Olweus and Sharon Mihalic, Bullying Prevention Program, in D. S. Elliot (series ed.), Blueprints for Violence Prevention (Boulder, Colo.: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado“Boulder, 1998).
39. Gilbert L. Botvin and others, “Long-term Follow-up Results of a Randomized Drug Abuse Prevention Trial in a White Middle-Class Population,” Journal of the American Medical Association, 273 (1995): 1106–12; Gilbert L. Botvin and others, “Preventing Adolescent Drug Abuse through a Multimodal Cognitive-Behavioral Approach: Results of a Three-Year Study,” Journal of Consulting and Clinical Psychology, 58 (1990): 437–46.
40. Mihalick and others, Blueprints for Violence Prevention Replications (see note 6).
41. Denise C. Gottfredson, Changing School Structures to Benefit High-Risk Youths. Troubled and Troubling Youth: Multidisciplinary Perspectives (Newbury Park, Calif.: Sage, 1990).
42. Olga Reyes and Leonard A. Jason, “An Evaluation of a High School Dropout Prevention Program,” Journal of Community Psychology, 10 (1991): 277–90.
43. Peter W. Greenwood, “Cost-Effective Violence Prevention through Targeted Family Interventions,” Annals of the New York Academy of Sciences, 1036 (Dec. 2004): 201–14, special issue on Youth Violence: Scientific Approaches to Prevention.
44. J. Alexander and others, Blueprints for Violence Prevention: Functional Family Therapy (Boulder, Colo.: University of Colorado, 1998).
45. S. W. Henggler and others, Multisystemic Therapy, in D. S. Elliot (series. ed.), Blueprints for Violence Prevention (Boulder, Colo.: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado, 1998).
46. H. Sontheimer and L. Goodstein, “Evaluation of Juvenile Intensive Aftercare,” Justice Quarterly, 10 (1993): 197–227.
47. C. Puzzanchera and others, Juvenile Court Statistics 1999 (Pittsburgh: National Center for Juvenile Justice, 2003).
48. D. A. Andrews and others, “Does Correctional Treatment Work?” Criminology, 28 (1990): 369–404.
49. Edward J. Latessa, S. J. Listwan, and D. J. Hubbard, Correctional Interventions: Changing Offender Behavior (Boston: Roxbury Publishing, 2005).
50. Mark W. Lipsey, Nana A. Landenberger, and Sandra J. Wilson, Effects of Cognitive-Behavioral Programs for Criminal Offenders (Center for Evaluation Research and Methodology, Vanderbilt Institute for Public Policy Studies, Campbell Collaboration, August 2007).
51. Barry Glick, “Aggression Replacement Training in Children and Adolescents,” The Hatherleigh Guide to Child and Adolescent Therapy, 5 (1996):191–226.
52. Steve Aos, Washington State’s Family Integrated Transitions Program for Juvenile Offenders: Outcome Evaluation and Benefit-Cost Analysis (Olympia, Wash.: Washington State Institute for Public Policy, 2004).
53. P. Chamberlain, Multidimensional Treatment Foster Care, in D. S. Elliot (series ed.), Blueprints for Violence Prevention (Boulder, Colo.: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado, 1998).
54. Aos, Miller, and Drake, Evidence-Based Public Policy Options (see note 2).
55. James C. Howell and Mark W. Lipsey, “A Practical Approach to Evaluating and Improving Juvenile Justice Programs,” Juvenile and Family Court Journal, 55, no. 1 (2004): 35–48.
56. See the article by Edward P. Mulvey and Anne-Marie R. Iselin in this issue.
57. K. Henigan and others, Phase 1: Survey of Interventions and Programs; A Continuum of Graduated Responses for Juvenile Justice in California (Juvenile Justice Data Project, Center for Research on Crime, University of Southern California, 2007).
58. Howell and Lipsey, “A Practical Approach” (see note 55).
59. Report of the Blueprint Commission (Tallahassee, Fla.: Florida Department of Juvenile Justice, 2007).
This feature: From The Future of Children, a collaboration of the Woodrow Wilson School of Public and International Affairs at Princeton University and the Brookings Institution. Juvenile Justice, 18, 2. Fall 2008
