Archive | Getting Approved by Journal Editors RSS feed for this section

Evidence Regarding Subjectivity in Approval Process

3 Jul

I’m going to deviate from my previous mentioned plan for today’s post.

I came across the article below in a weekly e-newsletter from AAAS / Science:
Editor’s Choice: Highlights of the Recent Literature, April 30, 2010

It asserts a finding that there is a 10% drop in publications and funding when a “superstar” researcher is subtracted abruptly from a team due to death.

“The authors’ analyses of these consequences favor a causal explanation in which the critical factor in these downward trends was being deprived of the intellectual input from these superstars, as opposed to a loss of collective experimental expertise or of privileged channels of communication to funding panels and journal editors.”

I urge you to consider, however, if this is not evidence that the selection of research for peer-reviewed publication is subjective, and driven by politics and personality over the substance of the ideas.

The Invisible College of Ideas

Gilbert Chin

It is no longer uncommon to see multi-authored original research papers, and in many instances, these studies represent the fruits of collaborations between multiple laboratories, especially in the biomedical sciences. How important are the lead researchers in these social and scientific networks? Answering this question empirically appears at first glance to be intractable, but Azoulay et al. have compiled a data set that enables them to take advantage of natural events—when still-active superstar researchers are subtracted from collaborations via death. Of the roughly 230,000 U.S. medical school faculty members, 10,000 were classified as elite according to seven objective professional criteria; during the last two decades of the 20th century, 112 of these scientists died suddenly. The effect on the productivity of the surviving faculty-level collaborators in these superstar-coauthor dyads was unambiguous and persistent: They suffered decrements of almost 10% in publications and funding. The authors’ analyses of these consequences favor a causal explanation in which the critical factor in these downward trends was being deprived of the intellectual input from these superstars, as opposed to a loss of collective experimental expertise or of privileged channels of communication to funding panels and journal editors.

Q. J. Econ. 125, 549 (2010).  IMAGE: Leena Peltonen-Palotie, 1952–2010. Image Credit: PASI HYTTI

_________________________________________________

Are we surprised that there are 10,000 out of 230,000 medical school faculty members that meet the criteria to be “elite” superstars? I wonder what the numbers would look like if the study focused on the top 1000 esteemed medical school faculty members.

A 10% drop in publications and funding is not that bad. It could be that the publishers and funders were willing to publish or fund boring or minor research because of the star, but when the star was gone, the dynamic changed. This would still be a case of the system not working as the public expects, namely that professional consideration is based on standards with objectivity, weighing the substance of the ideas.

Why don’t I buy the conclusion of the authors’ analyses? A team of researchers do most of the big decisions at the beginning of undertaking their project: the subject of the investigation, the method of investigation, key assumptions, etc. The research team and their investigation has a lot of inertia. The superstar on the team leaving abruptly might have changed aspects of presentation, or might recommend some adjustments to the actions of the team, but I find it hard to believe that an abrupt exit due to the death of the superstar researcher would change the main thrust or substance of the investigation. The substance of the investigation includes the inquiry into trying to answer some question about how or why the world works the way it does. This inquiry is what we expect to be the primary interest of the funding panel or journal editor.