I jog some days. When I do, generally I have something playing in my ears. This morning, among other things, I listened to one of my regular podcasts. It’s called Nature Podcast and is published by the scientific journal called Nature. This particular episode is titled Audio long read: So you got a null result. Will anyone publish it? The article was written by Max Kozlov and read by the podcast host Benjamin Thompson. What an interesting concept. Most journals, including Nature, are looking for scientific studies with some sort of positive correlation outcome. Something that has a new or novel conclusion. However, experience teaches that most scientific efforts result in a negative correlation, or null outcome, just like any other human endeavor. I’m reminded of how many failures the Edison company had coming up with a working filament for the lightbulb before finally getting one that worked. We now tell ourselves this famous story, but until there was a working light bulb, nobody had any interest in telling the story of all the other versions that didn’t work. Had they never gotten it write would there be any interest in the failure and likely death of the company? Even with success, are there articles that review each failure and speak to why they failed? I believe not. It seems like we just lump all the filament failures into a number to show how many times the experimenters in Edison’s employ failed as a tail about persistence paying off.
Kozlov points out, among other things, how journals are in the business of attracting readers. The article notes how there are some sites in cyber space where these negative results are published, but with few submissions and little readership. As I listened to the long read, I was reminded of several other famous arguments made in the past. For example, Robert Merton famously wrote on scientific norms in an attempt to explain what motivates scientists. Many have since argued that Mertonian norms depict an optimistic list and suggest counter norms that seem more realistic to the authors. One could make the argument that any set of norms ascribed are a function of the subjective preferences of the list constructor.
Another argument that came to my mind was that of the 'Matthew effect', an idea also coined by Robert Merton. This assertion touts that those who get published gain some sort of credibility which then makes their future findings more likely to be published. The idea is from the biblical book of Matthew when the statement in one of the parables asserts to him who have shall be given more, and to him who has not shall be taken away even that which he has. Now, of course, these New Testament ideas were not about scientific credibility, nonetheless Merton makes hay from the idea. Taking it a step further, Margaret Rossiter added a feminist perspective asserting that women in this predicament are even less likely to get published or recognized for their scientific research, dubbing her argument the 'Matilda effect'.
The Nature article shares several unintended consequences of this propensity to only publish positive findings and ignore null outcomes. In the environment of ‘publish or perish’ people who have null research outcomes are likely to drop efforts and not document them. There is a reinforcing relationship between readership levels and publication levels of negative findings. Also, there are likely missed opportunities in that some researchers might find inspiration in null findings to move the research in a different direction. These avenues could be missed if they don’t hear about the research. When such findings are not shared, others may also waste time by conducting the same failed research unawares. As with Nature, it seems to me there ought to be some way to encourage publication and consumption of null-finding research.
Kozlov points out, among other things, how journals are in the business of attracting readers. The article notes how there are some sites in cyber space where these negative results are published, but with few submissions and little readership. As I listened to the long read, I was reminded of several other famous arguments made in the past. For example, Robert Merton famously wrote on scientific norms in an attempt to explain what motivates scientists. Many have since argued that Mertonian norms depict an optimistic list and suggest counter norms that seem more realistic to the authors. One could make the argument that any set of norms ascribed are a function of the subjective preferences of the list constructor.
Another argument that came to my mind was that of the 'Matthew effect', an idea also coined by Robert Merton. This assertion touts that those who get published gain some sort of credibility which then makes their future findings more likely to be published. The idea is from the biblical book of Matthew when the statement in one of the parables asserts to him who have shall be given more, and to him who has not shall be taken away even that which he has. Now, of course, these New Testament ideas were not about scientific credibility, nonetheless Merton makes hay from the idea. Taking it a step further, Margaret Rossiter added a feminist perspective asserting that women in this predicament are even less likely to get published or recognized for their scientific research, dubbing her argument the 'Matilda effect'.
The Nature article shares several unintended consequences of this propensity to only publish positive findings and ignore null outcomes. In the environment of ‘publish or perish’ people who have null research outcomes are likely to drop efforts and not document them. There is a reinforcing relationship between readership levels and publication levels of negative findings. Also, there are likely missed opportunities in that some researchers might find inspiration in null findings to move the research in a different direction. These avenues could be missed if they don’t hear about the research. When such findings are not shared, others may also waste time by conducting the same failed research unawares. As with Nature, it seems to me there ought to be some way to encourage publication and consumption of null-finding research.
RSS Feed