Music in the Urban Context

Considering Ethics…

It seems that we ought to consider ethical issues in all areas and educational enterprises.  The issue of ethics is also significant in urban education, particularly in the newly found interest and investment in charter schools.  This continues to play a serious role in the mid-way liberalization of schooling, with possibilities in terms of administration and issues, such as the ones regarding the weakening of Unions.  Consider the issues in the following article excerpt:

Headline-Grabbing Charter School Study Doesn’t Hold Up To Scrutiny
November 12, 2009

BOULDER, Colo. and TEMPE, Ariz. (November 12, 2009) — A recent report on New York City charter schools found achievement results at the charters to be better than comparison traditional schools. But that report relies on a flawed statistical analysis, according to a new review.

The report is How New York City’s Charter Schools Affect Achievement and was written by Caroline Hoxby, Sonali Murarka, and Jenny Kang. When it was released in late September, it was enthusiastically and uncritically embraced by charter advocates as well as media outlets. The Washington Post offered an editorial titled, “Charter Success. Poor children learn. Teachers unions are not pleased.” The editorial’s first paragraph reads:

“Opponents of charter schools are going to have to come up with a new excuse: They can’t claim any longer that these non-traditional public schools don’t succeed. A rigorous new study of charter schools in New York City demolishes the argument that charter schools outperform traditional public schools only because they get the ‘best students.’ This evidence should spur states to change policies that inhibit charter-school growth. It also should cause traditional schools to emulate practices that produce these remarkable results.”

The editorial argues throughout that the study provides unquestionable evidence that charters result in improved student achievement. It ends, “Now the facts are in.”

The New York Daily News was no less effusive: “It’s official. From this day forward, those who battle New York’s charter school movement stand conclusively on notice that they are fighting to block thousands of children from getting superior educations.”

Because of the declared importance of the new report, we asked Professor Sean Reardon to carefully examine the report’s strengths and weaknesses for the Think Tank Review Project and write a review that would help others use the study in a sensible way. Reardon, like the report’s lead author Hoxby, is a professor at Stanford University. He is an expert on research methodology.

The Hoxby report estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school. A key finding, repeated in press reports throughout the U.S., compares the cumulative effect of attending a New York City charter school for nine years (from kindergarten through eighth grade) to the magnitude of average test score differences between students in Harlem and the wealthy New York community of Scarsdale. The report estimates this cumulative effect at roughly 66% of the “Scarsdale-Harlem gap” in English and roughly 86% of the gap in math.

In his review, Reardon observes that the report “has the potential to add usefully to the growing body of evidence regarding the effectiveness of charter schools.” New York charter schools’ use of randomized lotteries to admit students to charter schools offers the possibility that the study of those schools can roughly approximate laboratory conditions.

But Reardon points out that the report’s key findings are grounded in an unsound analysis — an inappropriate set of statistical models — and that the report’s authors never provide crucial information that would allow readers to more thoroughly evaluate “its methods, results, or generalizability.”

Reardon’s review notes these shortcomings in the report:

o    In measuring the effects of charter schooling on students in grades 4 through 12, the study relies on statistical models that include test scores from the previous year, measured after the admission lotteries take place. Yet because of that timing, those scores could be affected by whether students attend a charter school. As a consequence, the statistical models “destroy the benefits of the randomization” that is a strength of the study’s design. (The use of a different model makes the results for students in grades K-3 more credible, he notes.)
o    The report’s claims regarding the cumulative effects of attending a New York City charter school from kindergarten through eighth grade are based on an inappropriate extrapolation.
o    It uses a weaker criterion for statistical significance than is conventionally used in social science research (0.05), referring to p-values of roughly 0.15 as “marginally statistically significant”.
o    The report describes the variation in charter school effects across schools in a way that may distort the true distribution of effects by omitting many ineffective charter schools from the distribution.

– see the remaining info at: http://www.epicpolicy.org/