4 May 2010

April Research Round-Up

This is the second in our series
examining the most interesting, most discussed academic research on education
published each month. We’re publishing it a little early since May 1st
was a Saturday this month. As always, let us know if we forgot anything – or
if you have any researchers that you’d like us to add to our list!

Study: Evaluation of the Comprehensive School Reform Program
Implementation and Outcomes: Fifth-Year Report
Source: Daniel K.
Aladjem, Beatrice F. Birman, Martin Orland, Jennifer J. Harr-Robins, Alberto
Heredia, Thomas B. Parrish, Stephen J. Ruffini, WestEd Research Report

Results: The
Comprehensive School Reform program was designed to improve high poverty,
low-achieving schools by instituting scientifically-proven comprehensive
reforms. A new report by WestEd researchers finds that of the 7,000 schools
that received funding, only a third "selected reform models identified as
having a scientific research basis". Overall, the program was "not
associated with widespread achievement gains". Indeed, out of the 1,037
CSR elementary schools that were low performing, there were only 47 that showed
“dramatic and sustained” achievement gains.

What’s Interesting: Of
the schools that did make achievement gains, there was no one strategy that all
of the schools had in common that seemed to lead to achievement gains.

Caveat: There was very
little cohort data available before 2004-2005, so it was impossible to track
cohorts of students. Differences in state standards for testing made it difficult
to compare students nationally.

Study: Financial Incentives and Student
Achievement: Evidence from Randomized Trials
Source: Roland G. Fryer, National Bureau of Economics Research Working Paper

Results: Roland Fryer’s experiment
to incentive students via monetary rewards did not lead to measurable increase
in student achievement in New York City or Chicago, where students were paid
for good test scores or grades in core courses. More effective were two smaller
experiments that rewarded students for so-called "inputs" (as opposed
to "outcomes" like test scores). These "input" experiments,
which compensated students for the number of books read and for attendance and
homework, had statistically significant effects on student achievement.

What’s Interesting: The
Dallas program—which paid second graders $3 for each book read—had the
strongest effect on increasing reading comprehension and vocabulary and was the
least expensive program. The effect of this intervention did not lead to
negative results once the incentive was taken away—rather, there were
continued, albeit smaller, gains in achievement after the program ended.

Caveat: White and Asian
students did not experience gains, although the sample size was small, and
students not eligible for free lunch improved more than students who were eligible.
The gains measured once the program ended only looked at one year of data.

Study: Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement Source: Jesse
Rothsein, The Quarterly Journal of Economics

Results: This paper looks
at some of the assumptions behind so-called “Value-Added” models of teacher
quality ("VAMs") and finds that "the assumptions underlying
common VAMs are substantially incorrect, at least in North Carolina". Classroom assignments are not sufficiently
random to allow for a causal conclusion in common models.

What’s Interesting: Rothstein
makes two main points: (1)Accountability policies that rely on measures of
short-term value added would do a poor job of rewarding the teachers who are
best for students’ longer-run outcomes; and (2) Models that rely on incorrect
assumptions are likely to yield misleading estimates, and policies that use
these estimates may reward and punish teachers for the students they are
assigned as much as for their actual effectiveness in the classroom.

Caveat: Only looked at
data from students in North Carolina.

Study: A Closer Look at
Charter Schools and Segregation
Source: Gary Ritter,
Nathan Jensen, Brian Kisida, Joshua McGee, EducationNext

Results: A report issued by Education Next takes issue with the January 2010 report by the Civil Rights Project which found high levels of racial segregation in charter schools as compared to traditional public schools. The Education Next report argues that by comparing the demographic composition of all charter schools to that of all traditional public schools, the CRP’s report ignored significant
differences in neighborhood demographics within school districts. In order to
correct for this, Education Next compares the levels of segregation for
students in charter schools to that of their TPS counterparts in the central
city districts of the 8 largest metropolitan areas in the CRP report. The
authors found that the segregation gap narrows from the 20% reported in the CRP
report to 10%.

What’s Interesting: Authors
demonstrate that the majority of students in central cities, in both charters
and TPS, attend highly segregated minority schools.

Caveats: Authors caution
that their methodology could be improved one step further, by comparing the
demographics of charter schools with the traditional public schools that
charter students would have attended had they gone to their local zoned school.

One Comment

  1. Gideon says:

    There is a critique of the Rothstein paper that finds looking at value added over time virtually eliminates the sorting bias. http://economics.missouri.edu/working-papers/2009/wp0902_koedel.pdf

Comment