Tag Archives: subjectivity in selection of research ideas

Evidence Regarding Subjectivity in Approval Process

3 Jul

I’m going to deviate from my previous mentioned plan for today’s post.

I came across the article below in a weekly e-newsletter from AAAS / Science:
Editor’s Choice: Highlights of the Recent Literature, April 30, 2010

It asserts a finding that there is a 10% drop in publications and funding when a “superstar” researcher is subtracted abruptly from a team due to death.

“The authors’ analyses of these consequences favor a causal explanation in which the critical factor in these downward trends was being deprived of the intellectual input from these superstars, as opposed to a loss of collective experimental expertise or of privileged channels of communication to funding panels and journal editors.”

I urge you to consider, however, if this is not evidence that the selection of research for peer-reviewed publication is subjective, and driven by politics and personality over the substance of the ideas. Following is the article from the weekly e-newsletter.

By Gilbert Chin

It is no longer uncommon to see multi-authored original research papers, and in many instances, these studies represent the fruits of collaborations between multiple laboratories, especially in the biomedical sciences. How important are the lead researchers in these social and scientific networks? Answering this question empirically appears at first glance to be intractable, but Azoulay et al. have compiled a data set that enables them to take advantage of natural events—when still-active superstar researchers are subtracted from collaborations via death. Of the roughly 230,000 U.S. medical school faculty members, 10,000 were classified as elite according to seven objective professional criteria; during the last two decades of the 20th century, 112 of these scientists died suddenly. The effect on the productivity of the surviving faculty-level collaborators in these superstar-coauthor dyads was unambiguous and persistent: They suffered decrements of almost 10% in publications and funding. The authors’ analyses of these consequences favora causal explanation in which the critical factor in these downward trends was being deprived of the intellectual input from these superstars, as opposed to a loss of collective experimental expertise or of privileged channels of communication to funding panels and journal editors.

Q. J. Econ. 125, 549 (2010).  IMAGE: Leena Peltonen-Palotie, 1952–2010. Image Credit: PASI HYTTI

_________________________________________________

An alternate possibility is that the publishers and funders were willing to publish or fund boring or minor research because of the star, but when the star was gone, the dynamic changed. This has not been ruled out.

Why don’t I buy the conclusion of the analyses of Azuoulay et al? A team of researchers do most of the big decisions at the beginning of undertaking their project: the subject of the investigation, the method of investigation, key assumptions, etc. Following this, the research team and their investigation has a lot of inertia (resistance to change). The superstar on the team leaving abruptly might have changed aspects of presentation, or might recommend some adjustments to the actions of the team, but I find it hard to believe that an abrupt exit due to the death of the superstar researcher would change the main thrust or substance of the investigation. The substance of the investigation includes the inquiry into trying to answer some question about how or why the world works the way it does. This inquiry is what we expect to be the primary interest of the funding panel or journal editor. Do the other participants become any less professional? The dependence of research publication and funding on superstar researchers points to human factors independent from the research itself. Does the politics of research – as it is currently done – help us achieve the next level of discovery and knowledge?

In the end, the pursuit of knowledge is an activity involving people working together. And this will means that there is no escaping the human aspects of conflict, collaboration, prestige and intuitive judgements.