When Science Becomes Advocacy:
Childhood obesity in Somerville, Massachusetts
10 May 2007 in Regulatory Policy, Peer Review
Wall Street Journal health columnist Tara Parker Pope has a Page One article about a recent program intended to reduce the incidence of childhood obesity in Somerville, Massachusetts. The occasion for the story is publication of a scientific review of the program.
Pope's article, and the journal article on which it is based, raise troubling questions about scientists and journalists behaving as advocates. It also exposes problems inherent to scientific peer review.
Pope's article (temporarily available without subscription) documents the extensive efforts made by Tufts University assistant professor Christina Economos to reduce the incidence of childhood obesity in Somerville, the hometown of Tufts University. She obtained a $1.5 million grant from the federal Centers for Disease Control, plus other grant monies from privater sources, to change a host of environmental, dietary and other behavioral factors through a program called Shape Up Somerville. These changes included such things as altering school lunch menus, persuading local restaurants to reduce portion sizes and substitute low-fat foods, and repainting crosswalks to motivate children to walk to school. (Presumably they had been driven to school by parents.)
Pope says the program worked:
Pope tacitly acknowledges that Economos, the driving force behind the project, is also the principal author of the study purporting to show that it was effective. Nowhere in Pope's article does she reveal the potential for bias arising from Economos' dual roles as policy advocate and program evaluator. Moreover, the article is not yet publicly available from the journal Obesity. That means Pope had privileged access to the manuscript, its authors, the research methods and the data.
It's certainly possible that Economos and her colleagues conducted a fair and impartial review of their program despite their role as program advocates. Normally, program evaluation is performed by independent scientists without a professional interest in demonstrating success or failure. That's not the case here.
It's also possible that Pope performed a careful review of the manuscript and critically questioned the authors about it. But her story does not read that way. Most notably, she makes an elementary but huge error by calling the study a "controlled experiment." (The authors used comparison communities as statistical "controls" -- a very different and much less rigorous research methodology.)
Instead, it appears that Pope largely, if not completely, accepted the version provided her by the Tufts researchers and conducted no serious independent inquiry. There's no evidence from her story suggesting otherwise. In other words, Pope assumed that the manuscript provided an unbiased evaluation of the program. If there are weaknesses in their methods or problems with their data or unjustified inferences by the authors, we don't learn about them from Pope.
In cases where project advocates are also the project evaluators, scholarly journals have an especially acute need for rigorous peer review. It's not clear that the peer review performed by Obesity actually met this higher standard. There is reason for concern because the journal's purpose has both scholarly and advocacy elements. (Its editors are almost certainly not agnostic concerning whether obesity is good or bad.)
There is an additional peer review problem in this case: one of the authors is also an associate editor of the journal in which it will be published.
Economos CD, Hyatt RR, Goldberg JP, Must A, Naumova EN, Collins JJ, Nelson ME. "A Community Intervention Reduces BMI z-Score in Children: Shape Up Somerville First Year Results." Obesity. (May) 2007;15(5).
See also the Tufts University press release.
