Methodology for Estimating the Costs and Benefits of Youth Development Programs

Nia Harrison, Nikola Juris, Dori Stern, and Steven Stern

Department of Economics, University of Virginia

March 2008

 

 

Roadmap

 

            There are six steps involved in performing the cost-benefit analyses: 1) choose programs to analyze; 2) measure/estimate the effects of the program on behavior; 3) estimate the annual benefits of behavior change; 4) compute long-term benefits of behavior change; 5) perform sensitivity analyses; and 6) report results.  Each of these is described below.

 

Choosing Programs to Evaluate

 

We surveyed the literature looking for youth development programs that had been evaluated using a control group and showing some concern for selection issues.  While we identified many such studies, we do not claim to have done a comprehensive search of the literature.  Based on the perceived quality of the analyses, we chose to proceed further with evaluations of  the programs listed in Table 1.

 

Identified Programs

Programs

Author of Evaluation

Focus of Program

Resolution

All-Stars Programs

McNeal et al. (2004)

drug prevention

not kept

Big Brothers/ Big Sisters of America

Tierney, Grossman, & Resch (1995)

mentoring

kept

Boys and Girls Clubs

Schinke, Cole, & Poulin (2000)

after school

not kept

Career Academies

Kemple, J. and J. Scott-Clayton (2004)

high school graduation

kept

Children at Risk Programs

Harrell et al (1999)

multiple goals

kept

Drug Education Classes

Hansen & Graham (1991)

drug prevention

kept

Job Training Partnership Act Programs

Bloom et al. (1997)

job training

kept

Life Skills Training Programs

Griffin, Botvin, and  Nichols (2006)

drug prevention

kept

Service Learning Programs

Uggen & Janikula (1999)

service learning

kept

Sponsor a Scholar Programs

Johnson (1999)

college preparation

kept

Teen Outreach Programs

Allen et al. (1997)

service learning

kept

Twenty-first Century Community Learning Programs

Dynarski et al. (2003)

after school

not kept

 

 

Upon closer evaluation of the studies, we chose not to proceed with a few programs.  For Boys and Girls Club, we chose not continue because the analysis showed that children participating in Boys and Girls Club were more likely to participate in a set of unhealthy behaviors than those who did not participate.  It looked like this was the result of a selection problem; the unobserved part of the value of participating was positively correlated with the unobserved part of the value of engaging in the unhealthy behaviors.  Since Schinke, Cole, & Poulin (2000) made no attempt to control for such correlation, the program seemed to encourage unhealthy behavior.[1]  We looked for another evaluation of the Boys and Girls Club and were not successful in finding one, and we were not successful in receiving one from the local chapter either. 

 

We also chose not to proceed with the All-Stars program and the Twenty-first Century Community Learning programs because, in both cases, the analyses suggested that the programs were not effective.  We have no reason to think these are analyses are wrong, and we have not evaluated the studies with enough care to determine if they have significant problems.  In either case, there was no point in proceeding with a cost-benefit analysis given the results.

 

Measuring Program Effects on Behavior

 

            Each of the analyses provides direct estimates of the effects of the relevant program on different aspects of behavior.  We take these estimates as given.  In some cases, the respective researcher did not measure changes in a particular type of behavior, and we made a “reasonable guess” of the effect. A good example of this type is a program that measures changes in drug abuse but no changes in high school completion.  In such cases, using another source,[2]  we estimate the effect of drug usage on high school completion and then multiply the two effects to estimate the effect of the program on high school completion.  In all such cases, we also do sensitivity analyses to measure the effect of different “reasonable guesses” on our bottom-line estimates.  In other cases, either when we think it is too difficult to make a “reasonable guess” or where a reasonable conservative guess would be no effect, we assume that there is no effect.  A good example of this is not including an effect on criminal activity for Teen Outreach because it was not considered by Allen et al. (1997).[3]

 

            One also needs to be aware of potential differences in estimation methodology in the original analyses affecting results.  The best example of this is the effect of service learning programs on teen pregnancy.  Allen et al. (1997) estimate that the program they analyzed, Teen Outreach, reduced teen pregnancy by 1.7%, while Uggen and Janikula (1999) estimate that the program they analyzed, Service Learning, reduced teen pregnancy by 5.6%, more than three times as much.  The critical question is whether this difference represents a difference in estimation methodology or a difference in some critical characteristic of the program.  If it is the former, then the resulting difference in long-term benefits is not meaningful.  If it is the latter, then it would be worthwhile to understand the differences in the program.  Another similar example is the very large reported effects of drug education classes in Hansen and Graham (1991).  There are other, more recent, empirical evaluations of such programs that are much more pessimistic about their effectiveness.[4]  Is it that the estimation methodology changed, that kids have changed, or that the program has changed?  Disentangling these types of effects is beyond the scope of this project.

 

            One way to deal with uncertainty about program effects is to use reported standard errors in the empirical analysis in a sensitivity analysis.  This is discussed in more detail below.

 

            An important point to keep in mind here is that we are conditioning on the estimates provided in the empirical analyses.  To the degree that these effects are estimated with error (as is true of all estimates), our results may be incorrect.  While there are other sources of randomness in the remainder of the analysis, the randomness inherent in the estimated program effects is probably the largest source of error.  In doing a critical analysis of the reported long-term benefits here, one should be careful to specify with which estimates and/or assumptions one is concerned.

 

Estimating the Annual Benefits of Behavior Change

 

            Conditional on estimates of program effects on behavior, we must next translate those estimated effects into annual (monetary) benefits.  We do this on a case-by-case basis.  Note that some of the monetary benefits discussed below accrue to the program participant (e.g., increased future wages) while some accrue to society in general (e.g., savings on incarceration costs).  Throughout the analysis, we do not distinguish between private and societal costs and/or benefits.  They matter only to the degree that a policy analyst might worry about who should pay for a particular program; potential program participants will be willing to pay only for those services that result in large private gains.

 

1)  High School Completion: The major implication of completing high school is that one’s wages/earnings increase significantly.  Card (1999) reports that, holding other characteristics constant, having a high school diploma increases one’s wage by 46%.  In all of our base cases, we assume that a male (female) high school graduate earns $15.00 ($13.50) per hour[5] and work 2000 hours per year.  This implies annual earnings of $30,000 ($27,000) for males (females).  Given the estimate in Card (1999), the annual earnings benefit of finishing high school is $13,800 ($12,420) for males (females).  The annual benefit associated with an increased probability of graduating high school is the annual benefit of having a high school diploma multiplied by the increase in the probability of graduating high school.  For example, Bloom et al. (1997) estimate that JTPA increases graduation rates by 0.5% (7.7%) for males (females).  Therefore, the annual benefit associated with participating in JTPA is 0.5%*$13,800 =$690 (7.7%*$12,420 = $956) for males (females).  Relying on similar estimates in Card (1999) and using a similar calculation, we can compute the benefit of any program’s effect on the probability of getting higher levels of education.  Note that all of this benefit is enjoyed by the program participant.  We ignore any value that the general community might place on having a more educated populace.  This is not to say that no such benefit exists; rather it is too hard to measure.

 

For some programs, we impute the program’s effect on the probability of graduating from high school.  For Children at Risk, we assumed that the ratio of the effect of Service Learning on the probability of finishing high school divided by the effect of Service Learning on the probability of being arrested is equal to the ratio of the effect of Children at Risk on the probability of finishing high school divided by the effect of Children at Risk on committing crimes.   Also, we assume that the effect of Life Skills Training on the probability of graduating from high school is the same as for Children at Risk.

 

2)  College Entry:  As with high school completion, the major implication of college entry is that one’s wages/earnings increase significantly.  We use Card (1999) and Hinton et al. (2008) to estimate that, holding other characteristics constant, entering college increases one’s wage by 26.9%.[6]  We make the same assumptions about average wages for males (females) and work hours.  The rest of the methodology is the same as for high school completion.

 

2)  Crime Reduction:  There are three major financial implications of crime reduction: a) savings associated with incarceration costs; b) increased wages associated with not having a criminal record; and c) avoided costs to crime victims.  Note that (a) and (c) are public gains, while (b) is a private gain.

 

For (a) savings associated with incarceration costs, Rory Carpenter (from the Commission on Children and Families) provided us with information on the annual cost of incarceration: $32,560.  We assume (obviously incorrectly) that all arrests lead to jail time and calculate the annual benefit of the reduced arrest probability as $32,560 multiplied by the change in arrest probability.   The other critical issue here is how long people stay in jail and the recidivism rate.  These are discussed in more detail below.

 

For (b) increased wages associated with not having a criminal record, [citation needed][7] reports that, holding other characteristics constant, spending time in jail reduces one’s wage by 28.4%.  We multiply 28.4% by high school earnings and by the change in the probability of being arrested to estimate the annual private benefit of crime reduction.  Note that we ignore the facts that, while one is in jail, one can not earn money and that being in jail is not fun.  Without that much trouble, we could account for the lost wages while in jail.  Accounting for the disutility of being in jail would be significantly more difficult though probably larger in magnitude.

 

For (c) avoided costs to crime victims, using numbers in [citation needed], we estimated that the average cost to a victim of crime is $3300.  The other problem with estimating the benefit to reduced criminal activity is that most of our reported results are in terms of reductions in the probability of being arrested, and being arrested is not equivalent to committing a crime.  [citation needed] report that the probability of being arrested, conditional on committing a crime is 8.8% nationally.  Given this number, we need to divide our reported changes in arrest probabilities by 8.8% to compute changes in criminal activity.  For example, Bloom et al. (1997) find that arrest probabilities for boys decrease by 7.7%, implying that crimes decrease by 7.7%/8.8% = 0.875.

 

3)  Reductions in Substance Abuse:  Different authors use different measures of substance abuse reported in Table 2.  We ignore the short-term use of gateway drugs in Harrell et al. (1999) because we think the longer term effect is more important, and we ignore selling drugs because we do not know how to quantify its cost.  Next, we treat using drugs in Tierney, Grossman, and Resch (1995), long-term use of using drugs in Harrell et al (1999), using marijuana in Hansen and Graham (1991), and using drugs in Griffin, Botvin, and  Nichols (2006) as the same behavior.[8]  Also, we treat using alcohol in Tierney, Grossman, and Resch (1995) and using alcohol in Hansen and Graham (1991) as the same but distinguish both from getting drunk in Hansen and Graham (1991). 

 

To compute the benefit associated with reduced drug abuse, we adjusted estimates from French et al. (2002) to fit a one-year time period and calculated that the annual benefit of not using drugs is $5,103.  We assumed that the annual benefit of not getting drunk was the same as the annual benefit of not using drugs and that the annual benefit of not using alcohol was 1/3 of the annual benefit of not using drugs or getting drunk.  Given these assumptions, the annual benefits of the programs’ effects on substance abuse are the effects on behavior multiplied by the benefit associated with not participating in the behavior.  For example, given the reduction in long-term use of drugs of 7.0% caused by Children at Risk as reported in Hansen and Graham (1991) and the annual benefit of not using drugs of $5,103, the annual benefit of the reduction in drug use caused by Children at Risk is estimated to be $5,103*7% = $357.

 

 

Table 2

Program

Author

Measures

Big Brothers/ Big Sisters

Tierney, Grossman, and Resch (1995)

using drugs, using alcohol

Children at Risk

Harrell et al (1999)

short-term using gateway drugs, long-term using drugs, long-term selling drugs

Drug Education Classes

Hansen and Graham (1991)

using alcohol, getting drunk, using marijuana, using cigarettes

Life Skills Training

Griffin, Botvin, and  Nichols (2006)

using drugs

 

 

4)  Reductions in Teen Pregnancy, STDs:  As was true for substance abuse, different analyses measure different components of the problem here as reported in Table 3.  We treat the measures for Service Learning in Uggen and Janikula (1999) and for Teen Outreach in Allen et al. (1997) as measuring the same thing.  We use the estimate of the annual cost to a mother of being a pregnant teen of $3,700 from Strategic Planning Group (1999) which itself relies heavily on estimates in Maynard (1996).  Maynard finds that the effects of pregnancy for boys are statistically insignificant, and we conservatively follow her lead.  We assume that the probability of getting an STD conditional on having multiple sex partners is 12.5% and the cost of getting an STD is $2000, implying an annual benefit of not having multiple sex partners of $2000*12.5% = $250.  Also, we assumed that the probability of getting pregnant conditional on having sex when drunk or high was 2%.

           

Table 3

Program

Author

Measures

Life Skills Training

Griffin, Botvin, and  Nichols (2006)

having multiple sex partners, having sex when drunk or high

Service Learning

Uggen & Janikula (1999)

pregnancy before age 18

Teen Outreach

Allen et al. (1997)

getting pregnant

 

Computing Long-Term Benefits of Behavior Change

 

The basic idea in computing long-term benefits of behavior change is to take a stream of annual benefits, discount each element of the stream appropriately, and add up.  First, we discuss some of the details of computing present values, and then we discuss details particular to this project. 

 

Consider comparing the value of receiving a dollar today relative to receiving a dollar one year from today.  The dollar received today is more valuable than that received one year from now because there is value in having it sooner.  At a minimum, one could take a dollar today, invest it in an asset providing an annual rate of return r, and have (1+r) one year from now.  Also, it might be that individuals discount future rewards basically because they are impatient.  This may be especially true for youth.  We define the discount rate as the rate at which one discounts the future.  For example, if the source of discounting is the opportunity to earn interest in an asset at rate r, then r is the discount rate.  We define the discount factor as the value of $1 one year from now.  For example, if the source of discounting is the opportunity to earn interest at rate r, then the discount factor is β = 1/(1+r).  We define the present value of some flow X, received one year from today, as the value of the flow today: βX.   Note that, if we received βX today, we could invest it at rate r and have (1+r)βX =  (1+r)X/(1+r) = X one year from today.

 

Next, consider comparing the value of receiving X today relative to receiving X in n years from today.  If one invests X today for n years, then, at the end of n years, one will have (1+r)n X.  If one receives βnX = X/(1+r)n.  today and invests it for n years, then, at the end of n years, one will have (1+r)n X/(1+r)n = X.  Thus, the present value of X received n years in the future is βnX.

 

Finally, consider an investment opportunity where one pays C today and then receives x once a year starting in one year from today for T years.  Our goal is to compute the net present value of the investment.  The cost C is paid today, so there is no discounting.  The amount received in year t for some 1≤t≤T is βtx.  If we add up the discounted costs and benefits, we get

 

NB = -C + βx + β2x + … + βTx.

 

This simplifies to[9]

 

NB = -C + [β (1-βT+1)x/(1-β)]. 

 

Note that this converges to

 

NB = -C + [β x/(1-β)]

 

as T → ∞ (because, since 0 < β < 1, βT. → 0  as T → ∞).  Also note that the last few terms of a sum with a large terminal point T are very small; this implies that, for investments with long payoff periods, we can approximate the net present value by assuming that the payoffs continue indefinitely.[10]   Sometimes 1/(1-β) is called the multiplier.  For a more comprehensive discussion on net present value, one can refer to almost any elementary economics or finance text book (for example, see Frank and Bernanke, 2007, 232 – 235)

 

 For this project, we can think of participation in each of the programs as an investment.  There is a per capita program cost paid at the time of participation, and then there are future benefit flows.  For each program either the author of the program analysis reported the per capita cost or we estimated its cost.  The future benefit flows are the annual benefits (prior to discounting) discussed in the previous section.  For the majority of cases in this project we approximate the net discounted value as –C + [X/(1-β)] where X is the annual benefit associated with some estimated change in behavior caused by the program.

 

There are some special cases specific to this project that are worth discussing: 

 

1)  Many of the programs were reported to reduce arrest probabilities.  We need to make an assumption about how long people stay in jail and how likely they are to return to jail.  Let K be the average number of years spent in jail and let H be the hazard rate for returning to jail.  Then the proportion of time one spends in jail over a lifetime is P = KH/[(K-1)H+1].  As K becomes large, P converges to 1, and, as H converges to one (complete recidivism), P converges to 1.  For most of the analysis we set P = 1.  This may be too extreme an assumption.  However, it is balanced by some of the costs of crime we ignore. 

 

2)  Throughout the analysis we assume that the cost to victims of crime is a one-shot cost.  Thus, we do not multiply it by any multiplier.  This is consistent with assuming that someone commits a crime never commits another crime.  This is also somewhat unreasonable but is balanced against the strong assumptions in (1) above.

 

            It is worth considering how the assumptions made with respect to crime affect the analysis.  Throughout the analysis, for the base case, we use an annual cost of jail of $32,560, a cost to victims of $3,300, and cost of reduced future wages of $8,520.  In Table 4, we assume a discount factor of 0.9 and consider the benefits of a program that reduces arrest probabilities by 5%.  In the base case, with the proportion of time one spends in jail P equal to 1, no crimes committed after arrest, and no lost wages due to time spent in jail, the present value of incarceration (jail) is $16,280, the present value of victim loss is $1,875, and the present value of lost earnings (due to a lower wage rate) is $4,260, leading to total benefits of $22,415.  If, instead we assume that, conditional on being arrested, one spends half of his discounted life[11] in prison (P = .5) while holding all other assumptions constant, the benefits associated with reduced jail time declines to $8,140 leading to an overall decline to $14,275.  If we assume instead that P = 0.2, the benefit of reduced jail time becomes $3,256, and the benefit or reduced arrest probabilities falls to $9,391. 

 

Table 4

 

Jail

Victim

Lost Earnings

Total

Annual Cost

$32,560

$3,300

$8,520

 

Annual Cost * ΔPr(Arrest)

$1,628

$1,875

$426

 

P = 1

$16,280

$1,875

$4,260

$22,415

P = .5

$8,140

$1,875

$4,260

$14,275

P = .2

$3,256

$1,875

$4,260

$9,391

# Crimes Proportional to Time in Community, P = 1

$16,280

$1,875

$4,260

$22,415

# Crimes Proportional to Time in Community, P = .5

$16,280

$9,375

$4,260

$29,915

No Earnings While in Jail, No HS, P = 1

$16,280

$1,875

$162,000

$180,155

No Earnings While in Jail, No HS, P = .5

$8,140

$1,875

$104,004

$114,019

 

We may want to measure the sensitivity of results with respect to the assumption that one stops committing crimes after being arrested.  In fact, this is inconsistent with assuming a high recidivism rate.  If instead we assume that one commits crimes at the same rate after arrest (and release from prison) as before arrest, then the results depend on P.  If P = 1, then committing crimes does not increase because one never leaves prison.  If P falls to 0.5, then the benefits to crime victims of the program increases to $9,375. 

 

We also may want to measure the benefit associated with being able to work more because arrest probabilities decline.  This again depends critically on P, and also it depends on the wage rate one assumes.  With respect to the latter assumption, we assume that the relevant person would have earned 56% of a high school graduate wage (the effect of not graduating from high school) had they not been in jail.  With respect to the former assumption, if P = 1, then their amount of time lost for working is large and the benefit associated with reduced arrest probabilities is $162,000.  If P = 0.5, then program benefit is $104,004.  These are obviously large costs. 

 

A subtler point is that the benefit of the programs’ effect on arrest probabilities depends on the effectiveness of programs to reduce recidivism.  If OAR[12] becomes more effective and P declines, then the value of keeping youth out of criminal activity declines.

 

3)  There is some concern that many of the estimated program effects reported in the studies listed in Table 1 are temporary effects.  For example, it might be that Life Skills Training reduces the probability of using drugs by 4.7%, as reported in Griffin, Botvin, and  Nichols (2006), but only for a short time after participating in the program.  If so, then we are drastically overestimating the benefits of the program.[13]  To deal with this possibility, in many of the sensitivity analyses discussed below, we consider the possibility that the effects of the program decline by 80% each year until one reaches age 18.  This is equivalent to changing the discount factor for the years in between program participation and age 18 from 0.9 to 0.9*0.2 = 0.18.  This tends to change multipliers from 10 to numbers in the range of 2 to 3, depending upon the age of program participants.

 

            However, an alternative way for programs to deal with deteriorating effects is to provide services to the participating youth for a longer period of time.  We implement this suggestion in our sensitivity analyses by considering the alternative scenario where youth are assumed to participate in the program every year until they reach age 18 and pay the same annual cost each year.  This brings the gross benefits (prior to subtracting cost) back to where they were before assuming program effect deterioration, and it increases the cost from C to C + βC + … + βnC = C(1-βn+1)/(1-β) where n is the number of years one is assumed to participate in the program.

 

Performing Sensitivity Analyses

 

            Throughout the analysis, we need to make assumptions about unobserved parameters.  To the degree allowed by availability of previous research and time constraints, we have tried to base those assumptions on reliable information.  In some cases, we are left to make wild guesses.  In either case, it is important for us to measure how sensitive our results are to the various assumptions that are made.  Such an analysis allows interested parties to a) focus criticism, discussion, and further research on the issues that are most critical, and b) provides a measure of how confident policymakers should be concerning expectations of a proposed program.

 

            For each of the programs analyzed in Table 1, we perform a set of policy analyses.   In general, we measure the effect of changing the assumed discount rate, either because we think people are less patient or because of deteriorating program effects.  We also measure the sensitivity of results to changes in assumed program effects not directly reported in the program evaluation.  Finally, we measure the effect of changing the cost of the program. 

 

            In each case, we make one change in an assumption and hold everything else constant.  While this allows us to measure the effect of each assumption by itself, it misses interaction effects.  Most of the time, these interaction effects will be small.  However, we saw above an example where relaxing two assumptions, the proportion of time one spends in jail after arrest and the propensity to commit crime after being released have significant interaction effects.

 

            All of the sensitivity analyses we do concern assumptions that we made.  However, the other, possibly large source of uncertainty about results is the randomness associated with the estimates of the program effects.  For example, the standard error of effect of Service Learning on arrest probabilities is approximately 1/3 of the effect.  Thus, a 90% confidence interval for the program effect for boys is 8% ± 2*2.33% = (3.67%, 12.67%), and a 90% confidence interval for the present value of this program change (excluding benefits to crime victims) is $32,864 ± 2 * $10955 = ($10,954, $54,774).  While the effect of randomness of the reported program effects is obviously large, we did not perform sensitivity analyses to measure the sensitivity of results to them because of time constraints and because many of the analyses listed in Table 1 report p-value ranges (e.g., p<0.01) rather than standard errors.

 

Reporting Results

 

            For each program, we report the important program effects from the program analysis.  Next, we report annual benefits associated with changing the behaviors affected by the program.  We then translate them into long-term benefits using appropriate discounting multipliers and report net long-term benefits.  In the summary report, we also report benefit/cost ratios.

 

Works Referenced

 

Allen, J., S. Philliber, S. Herrling, and G. Kuperminc (1997). “Preventing Teen Pregnancy and Academic Failure: Experimental Evaluation of a Developmentally Based Approach.” Child Development. 64(4): 729-742.

 

Baldwin, Marjorie (1999). “The Effects of Impairments on Employment and Wages: Estimates from the 1984 and 1990 SIPP.” Behavioral Sciences and the Law. 17: 7-27.

 

Bloom, H., L. Orr, S. Bell, G. Cave, F. Doolittle, W. Lin, and J. Bos (1997). “The Benefits and Costs of JTPA Title II-A Programs: Key Findings from the National Job Training Partnership Act Study.” Journal of Human Resources. 32 (3): 549-576.

 

Card, D. (1999). “The Causal Effect of Education on Earnings.” In O. Ashenfelter and D. Card (Eds.). Handbook of Labor Economics, 3, pp. 1801-1863. Amsterdam: North-Holland.

 

Dynarski, Mark, Mary Moore, John Mullens, Philip Gleason, Susanne James-Burdumy, Linda Rosenberg, Wendy Mansfield, Sheila Heaviside, Daniel Levy, Carol Pistorino, Tim Silva, and John Deke (2003). “When Schools Stay Open Late: The National Evaluation of the 21st-Century Community Learning Centers Program. First Year Findings.” ED Pubs, Education Publications Center, U.S. Department of Education.  http://www.ed.gov/pubs/21cent/firstyear.

 

Frank, Robert and Ben Bernanke (2007). Principles of Economics.  Volume 3. New York: McGraw-Hill.

 

 French, M., H. Salomé, J. Sindelar, and A. McLellan (2002). Benefit-Cost Analysis of Addiction Treatment: Methodological Guidelines and Empirical Application Using the DATCAP and ASI. Health Services Research. 37(2): 433-455.

 

Griffin, K., G. Botvin, and T. Nichols (2006). “Effects of a School-Based Drug Abuse Prevention Program for Adolescents on HIV Risk Behavior in Young Adulthood.” National Institutes of Health. http://www.wellnessandpreventionoffice.org/LST_other/Effects_Behavior.pdf

 

Harrell, A., S. Cavanagh, and S. Sridharan (1999). “Evaluation of the Children at Risk Program: Results 1 Year After the End of the Program.” National Institute of Justice. http://www.ncjrs.gov/pdffiles1/nij/178914.pdf

 

Hansen, W. and J. Graham (1991). “Preventing Alcohol, Marijuana, and Cigarette Use Among Adolescents: Peer Pressure Resistance Training versus Establishing Conservative Norms.” Preventative Medicine. 20: 414-430.

 

Hinton, Ivora, Jessica Howell, Elizabeth Merwin, Steven Stern, Sarah Turner, Ishan Williams, and Melvin Wilson (2007). “The Educational Pipeline for Health Care Professionals:  Understanding the Source of Racial Differences.” Unpublished manuscript.

 

Johnson, A.W. (1997). “Mentoring At-Risk Youth: A Research Review and Evaluation of the Impacts of the Sponsor-A-Scholar Program on Student Performance.” Ph.D. dissertation, University of Pennsylvania, United StatesPennsylvania.

 

 Kemple, J. and J. Scott-Clayton (2004). “Career Academies Impact on Labor Market Outcomes and Educational Attainment.” MDRC http://www.mdrc.org/publications/366/full.pdf

 

Lynskey, M., C. Coffey, L. Degenhardt, J. Carlin, and G. Patton (2003). “A Longitudinal Study of the Effects of Adolescent Cannabis Use on High School Completion.” Addiction. 98(5): 685-692.

 

Maynard, R. A. (1996).  “The Costs of Adolescent Childbearing,” In R. A. Maynard (eds). Kids Having Kids: A Robin Hood Foundation Special Report on the Costs of Adolescent Childbearing.  Urban Institute, Washington, D.C.

 

McNeal, Ralph, William Hansen, Nancy Harrington, and Steven Giles (2004). “How all Stars Works: An Examination of Program Effects on Mediating Variables.”  Health Education and Behavior. 31(2): 165-178.

 Schinke, S., K. Cole, and S. Poulin (2000). “Enhancing the Educational Achievement of At-Risk Youth.” Prevention Science. 1(1): 51-60.

 

Stern, Steven and Amelia McKeithen (2007). “The Costs and Benefits of Mental Health Care.” http://www.people.virginia.edu/~sns5r/resint/psychstf/mentalhealthbenefits.pdf

 

Strategic Planning Work Group of the Task Force on Teen Pregnancy Prevention, Charlottesville and Albemarle County, Virginia (1999). A Community Strategic Plan for Preventing Teen Pregnancies and Sexually Transmitted Diseases. http://people.virginia.edu/~sns5r/teenpregstf/strpln.pdf.

 

Tierney, J.P., J. Grossman, and N. Resch (1995). Making a Difference: An Impact Study of Big Brothers/ Big Sisters. Philadelphia: Public/Private Ventures. http://www.ppv.org/ppv/publications/assets/111_publication.pdf

 

 Uggen, C. and J. Janikula (1999). “Volunteerism and Arrest in the Transition to Adulthood.” http://www.socsci.umn.edu/~uggen/Uggen_Janikula_SF_99.pdf.



[1] It is worth noting that the JTPA analysis implied that the program reduced wage rates; this may also suffer from selection bias especially because one would have to wonder why participants would agree to join a training program with negative returns.  Another way to say the same thing is that maximum likelihood estimates with a restriction that the value of participating must be positive would never result in an estimated negative return to the program.  We kept JTPA in the analysis anyway because of the observed large effects on crime reduction.

[2] See Lynskey et al. (2003).

[3] This presents an extra problem, however, because another examined service learning program examined by Uggen and Janikula (1999) does directly estimate the effect of the program on criminal activity, and this has very large effects on the long-term benefit of the program.  One should be hesitant to compare Teen Outreach and Service Learning without taking this difference in analysis into account.

[4] Though we know of the existence of such empirical analyses, we were not able to find them given time constraints.

[5] Based on information in Baldwin (1999) and used in another cost-benefit analysis for the Commission on Children and Families for mental health in Stern and McKeithen (2007).

[6] Card (1999) reports that wages for college graduates are higher than for high school graduates by 46%, and Hinton et al. (2008) find that the probability of finishing college conditional on starting college is 58.4%.

[7] We are still searching through our files to find the three missing citations on this page.

[8] One could reasonably argue that using marijuana in Hansen and Graham (1991) is really more like using gateway drugs in Harrell et al (1999) than long-term using drugs in  Harrell et al (1999).  However, since there is no other measure of drug use in Hansen and Graham (1991), we do not have much better alternatives.

[9] Define S = βx + β2x + … + βTx, implying that β S = β2x + β3x … + βT+1x.  Subtracting  β S from S results in (1- β)S = 1 -  βT+1x, implying that S  = (1 -  βT+1x) /  (1- β).

[10] For example, for r = 0.1 implying β  ≈ 0.9, the present value of receiving $1 a year for 40 years is (1-β41) /(1-β) = $9.867, and the present value of receiving $1 a year forever is $10. 

[11] Discounted life weighs earlier years more than later years.

[12] OAR is a local program to help offenders stay out of jail.

[13] This point was made to us convincingly by Karen Walker.