{"id":41160,"date":"2025-11-27T07:15:00","date_gmt":"2025-11-27T15:15:00","guid":{"rendered":"https:\/\/www.lifeandnews.com\/articles\/?p=41160"},"modified":"2025-11-28T23:28:16","modified_gmt":"2025-11-29T07:28:16","slug":"absence-of-evidence-is-not-evidence-of-absence-and-that-affects-what-scientific-journals-choose-to-publish","status":"publish","type":"post","link":"https:\/\/www.lifeandnews.com\/articles\/absence-of-evidence-is-not-evidence-of-absence-and-that-affects-what-scientific-journals-choose-to-publish\/","title":{"rendered":"Absence of evidence is not evidence of absence \u2013 and that affects what scientific journals choose to publish"},"content":{"rendered":"\n

Mark Louie Ramos<\/a>, Penn State<\/a><\/em><\/p>\n\n\n\n

Should you believe the findings of scientific studies? Amid current concerns about the public\u2019s trust in science, old arguments<\/a> are resurfacing<\/a> that can sow confusion.<\/p>\n\n\n\n

As a statistician involved in research<\/a> for many years, I know the care that goes into designing a good study capable of coming up with meaningful results. Understanding what the results of a particular study are and are not saying can help you sift through what you see in the news or on social media.<\/p>\n\n\n\n

Let me walk you through the scientific process, from investigation to publication. The research results you hear about crucially depend on the way scientists formulate the questions they\u2019re investigating.<\/p>\n\n\n\n

The scientific method and the null hypothesis<\/h2>\n\n\n\n

Researchers in all kinds of fields use the scientific method<\/a> to investigate the questions they\u2019re interested in.<\/p>\n\n\n\n

First, a scientist formulates a new claim \u2013 what\u2019s called a hypothesis. For example, is having some genetic mutations in BRCA genes related to a higher risk of breast cancer<\/a>? Then they gather data relevant to the hypothesis and decide, based on the data, whether that initial claim was correct or not.<\/p>\n\n\n\n

It\u2019s intuitive to think that this decision is cleanly dichotomous \u2013 that the researcher decides the hypothesis is either true or false. But of course, just because you decide something doesn\u2019t mean you\u2019re right.<\/p>\n\n\n\n

If the claim is really false but the researcher decides, based on the evidence, it\u2019s true \u2013 a false positive \u2013 they commit what\u2019s called a Type 1 error<\/a>. If the claim is really true but the researcher fails to see that \u2013 a false-negative conclusion \u2013 then they commit a Type 2 error.<\/p>\n\n\n\n

Moreover, in the real world, it gets a little messier. It\u2019s really hard to decide about the truth or falsity of a claim just based on what\u2019s observed.<\/p>\n\n\n\n

For that reason, most scientists employ what is called the null hypothesis significance testing<\/a> framework. Here\u2019s how it works: A researcher first states a \u201cnull hypothesis,\u201d something that\u2019s contrary to what they want to prove. For instance, in our example the null hypothesis is that BRCA genetic mutations are not associated with increased breast cancer occurrence.<\/p>\n\n\n\n

The scientist still gathers data and makes a decision, but the decision is not about whether the null is true. Instead, a researcher decides whether there\u2019s enough evidence to reject the null hypothesis or not.<\/p>\n\n\n\n

\"man
Careful statistical analysis along with a well-formulated null hypothesis lend confidence to a study finding. Jackyenjoyphotography\/Moment via Getty Images<\/a><\/figcaption><\/figure>\n\n\n\n

What rejecting the null does and doesn\u2019t mean<\/h2>\n\n\n\n

Understanding this distinction is crucial. Rejecting the null is equivalent in practice to acting as though it is false \u2013 in the example, rejecting the null means claiming that those with some BRCA gene mutations do have a higher risk of breast cancer. Along with other evidence, such as the size of the increased risk, this outcome can justify recommending early breast cancer screening for people with the identified BRCA mutations<\/a>.<\/p>\n\n\n\n

But failing to reject the null hypothesis doesn\u2019t imply that it\u2019s true \u2013 in this case, it doesn\u2019t mean there is no association between the BRCA mutations and breast cancer. Rather, such a result is inconclusive; there\u2019s not enough evidence to claim there is an association. A negative result \u2013 inadequate evidence to say the null is false \u2013 does not necessarily invite the researcher to believe the null is true.<\/p>\n\n\n\n

This is because null hypothesis significance testing is set up to control for Type 1 error (false positive) at a level defined in advance by the researcher but at the cost of having less control over Type 2 error (false negative).<\/p>\n\n\n\n

A researcher\u2019s chances of correctly rejecting the null if there is increased risk can depend on how much data they have, how complex the design of the study is and, most importantly, how large the effect actually is. It\u2019s much easier to reject the null if BRCA mutations truly increase cancer risk many times than it is if the risk is only slightly elevated. A researcher can end up with a result that is not statistically significant but cannot rule out the possibility of an increased risk that is too small for the study to detect.<\/p>\n\n\n\n

Which results are more often publicized<\/h2>\n\n\n\n

Once they have their result and the researchers want to disseminate their work, they typically do so through peer-reviewed publication<\/a>. Journal publishers consider a researcher\u2019s write-up of their study, send it out for other scientists to review, and then decide whether to publish it.<\/p>\n\n\n\n

In this process, the publishers tend to favor studies that rejected their null hypothesis<\/a> over those that failed to reject it. This is called positive publication bias.<\/p>\n\n\n\n

It is natural for publishers to prefer studies that support new claims since they objectively carry more information than studies that failed to reject their null hypothesis. Journals want to publish something new and noteworthy.<\/p>\n\n\n\n

Many sources flag this phenomenon as \u201cbad science<\/a>,\u201d but is it really? Remember, the framework used to make decisions about scientific claims is intentionally only capable of either rejecting the null hypothesis \u2013 in other words, supporting the claim \u2013 or alternatively declaring inconclusive results.<\/p>\n\n\n\n

The framework isn\u2019t designed to be able to prove the null hypothesis. That said, researchers can reverse the design of a scientific investigation so that a previous claim becomes the null hypothesis in a new study with fresh data.<\/p>\n\n\n\n

For instance, rather than a null hypothesis that there is no association between BRCA mutations and breast cancer, the null hypothesis becomes that the increased breast cancer risk from BRCA mutations is equal to or greater than some value the researcher settles on before gathering fresh data.<\/p>\n\n\n\n

Rejecting the null this time would mean the increased risk is smaller than that set value, thus supporting the claim consistent with what had previously been the null hypothesis on prior data. In the example, rejecting the null means the effect of BRCA genes is small enough to be practically negligible in terms of developing breast cancer.<\/p>\n\n\n\n

It\u2019s critical for a researcher to structure their study so that what they\u2019re interested in proving is aligned with the rejection of the null. Publishers are naturally less inclined to consider studies that failed to reject their null hypothesis, not because they do not want to publish studies that support negative statements but because null hypothesis significance testing does not actually support negative statements. Failure to reject the null just means your results are inconclusive \u2013 and may perhaps seem less newsworthy.<\/p>\n\n\n\n

\"library
Research journals want to publish results that will have an impact. luoman\/iStock via Getty Images Plus<\/a><\/figcaption><\/figure>\n\n\n\n

What positive publication bias does<\/h2>\n\n\n\n

So what does the practice of preferring to publish studies that reject their null hypothesis do?<\/p>\n\n\n\n

While we can\u2019t know for certain, we can see how this plays out under different circumstances. You can explore the scenarios in this app<\/a> I made.<\/p>\n\n\n\n

If scientists are acting in good faith, using null hypothesis significance testing appropriately, it turns out that positive publication bias on the part of scientific journal publishers will increase the proportion of true discoveries in their pages much more than it will increase the proportion of false positives.<\/p>\n\n\n\n

If editors did not exercise any positive publication bias, journals would be almost entirely full of studies with inconclusive results.<\/p>\n\n\n\n

Of course, if scientists are not acting in good faith and are just interested in getting published while ignoring proper use of statistical tests, that can lead to false-positive rates being as high or higher than the rate of true discoveries. But this possibility is true even without positive publication bias.<\/p>\n\n\n\n

Mark Louie Ramos<\/a>, Assistant Research Professor of Health Policy and Administration, Penn State<\/a><\/em><\/p>\n\n\n\n

This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Mark Louie Ramos, Penn State Should you believe the findings of scientific studies? Amid current concerns about the public\u2019s trust in science, old arguments are resurfacing that can sow confusion. As a statistician involved in research for many years, I know the care that goes into designing a good study capable of coming up with […]<\/p>\n","protected":false},"author":56,"featured_media":41161,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[37,292,10,36,3410,38],"tags":[16981,17185,17182,885,891,886,860,5638,17184,390],"_links":{"self":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/41160"}],"collection":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/users\/56"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/comments?post=41160"}],"version-history":[{"count":1,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/41160\/revisions"}],"predecessor-version":[{"id":41162,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/41160\/revisions\/41162"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/media\/41161"}],"wp:attachment":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/media?parent=41160"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/categories?post=41160"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/tags?post=41160"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}