{"id":39933,"date":"2025-07-17T12:45:00","date_gmt":"2025-07-17T12:45:00","guid":{"rendered":"https:\/\/www.lifeandnews.com\/articles\/?p=39933"},"modified":"2025-07-18T05:45:23","modified_gmt":"2025-07-18T05:45:23","slug":"research-replication-can-determine-how-well-science-is-working-but-how-do-scientists-replicate-studies","status":"publish","type":"post","link":"https:\/\/www.lifeandnews.com\/articles\/research-replication-can-determine-how-well-science-is-working-but-how-do-scientists-replicate-studies\/","title":{"rendered":"Research replication can determine how well science is working \u2013 but how do scientists replicate studies?"},"content":{"rendered":"\n

Amanda Kay Montoya<\/a>, University of California, Los Angeles<\/a><\/em><\/p>\n\n\n\n

Back in high school chemistry, I remember waiting with my bench partner for crystals to form on our stick in the cup of blue solution. Other groups around us jumped with joy when their crystals formed, but my group just waited. When the bell rang, everyone left but me. My teacher came over, picked up an unopened bag on the counter and told me, \u201cCrystals can\u2019t grow if the salt is not in the solution.\u201d<\/p>\n\n\n\n

To me, this was how science worked: What you expect to happen is clear and concrete. And if it doesn\u2019t happen, you\u2019ve done something wrong.<\/p>\n\n\n\n

If only it were that simple.<\/p>\n\n\n\n

It took me many years to realize that science is not just some series of activities where you know what will happen at the end. Instead, science is about discovering and generating new knowledge.<\/p>\n\n\n\n

Now, I\u2019m a psychologist<\/a> studying how scientists do science. How do new methods and tools get adopted? How do changes happen in scientific fields, and what hinders changes in the way we do science?<\/p>\n\n\n\n

One practice that has fascinated me for many years is replication research, where a research group tries to redo a previous study. Like with the crystals, getting the same result from different teams doesn\u2019t always happen, and when you\u2019re on the team whose crystals don\u2019t grow, you don\u2019t know if the study didn\u2019t work because the theory is wrong, or whether you forgot to put the salt in the solution.<\/p>\n\n\n\n

The replication crisis<\/h2>\n\n\n\n

A May 2025 executive order<\/a> by President Donald Trump emphasized the \u201creproducibility crisis<\/a>\u201d in science. While replicability and reproducibility may sound similar, they\u2019re distinct.<\/p>\n\n\n\n

Reproducibility<\/a> is the ability to use the same data and methods from a study and reproduce the result. In my editorial role at the journal Psychological Science, I conduct computational reproducibility checks<\/a> where we take the reported data and check that all the results in the paper can be reproduced independently.<\/p>\n\n\n\n

But we\u2019re not running the study over again, or collecting new data. While reproducibility is important, research that is incorrect, fallible and sometimes harmful can still be reproducible.<\/p>\n\n\n\n

By contrast, replication is when an independent team repeats the same process, including collecting new data, to see if they get the same results. When research replicates, the team can be more confident that the results are not a fluke or an error.<\/p>\n\n\n\n

\"A
Reproducibility and replicability are both important, but have key differences. Open Economics Guide<\/a>, CC BY<\/a><\/figcaption><\/figure>\n\n\n\n

The \u201creplication crisis<\/a>,\u201d a term coined in psychology in the early 2010s, has spread to many fields, including biology<\/a>, economics<\/a>, medicine<\/a> and computer science<\/a>. Failures to replicate high-profile studies concern many scientists in these fields.<\/p>\n\n\n\n

Why replicate?<\/h2>\n\n\n\n

Replicability is a core scientific value: Researchers want to be able to find the same result again and again. Many important findings are not published until they are independently replicated.<\/p>\n\n\n\n

In research, chance findings can occur. Imagine if one person flipped a coin 10 times and got two heads, then told the world that \u201ccoins have a 20% chance of coming up heads.\u201d Even though this is an unlikely outcome \u2013 about 4% \u2013 it\u2019s possible.<\/p>\n\n\n\n

Replications can correct these chance outcomes, as well as scientific errors, to ensure science is self-correcting.<\/p>\n\n\n\n

For example, in the search for the Higgs boson, two research centers at CERN, the European Council for Nuclear Research<\/a>, ATLAS and CMS, independently replicated<\/a> the detection of a particle with a large unique mass, leading to the 2013 Nobel Prize in physics<\/a>.<\/p>\n\n\n\n

\"A
The ATLAS experiment at the Large Hadron Collider at CERN is one of two that led to the discovery of the Higgs boson. CERN<\/a>, CC BY<\/a><\/figcaption><\/figure>\n\n\n\n

The initial measurements from the two centers actually estimated the mass of the particle as slightly different. So while the two centers didn\u2019t find identical results, the teams evaluated them and determined they were close enough. This variability is a natural part of the scientific process. Just because results are not identical does not mean they are not reliable.<\/p>\n\n\n\n

Research centers like CERN have replication built into their process, but this is not feasible for all research. For projects that are relatively low cost, the original team will often replicate their work prior to publication \u2013 but doing so does not guarantee that an independent team could get the same results.<\/p>\n\n\n\n

\"A
Because the results on vaccine efficacy were so clear, replication wasn\u2019t necessary and would have slowed the process of getting the vaccine to people. XKCD<\/a>, CC BY-NC<\/a><\/figcaption><\/figure>\n\n\n\n

When projects are costly, urgent or time-specific, independently replicating them prior to disseminating results is often not feasible. Remember when people across the country were waiting for a COVID-19 vaccine?<\/p>\n\n\n\n

The initial Pfizer-BioNTech COVID-19 vaccine took 13 months<\/a> from the start of the trial to authorization from the Food and Drug Administration. The results of the initial study were so clear and convincing that a replication would have unnecessarily delayed getting the vaccine out to the public and slowing the spread of disease.<\/p>\n\n\n\n

Since not every study can be replicated prior to publication, it\u2019s important to conduct replications after studies are published. Replications help scientists understand how well research processes are working, identify errors and self-correct. So what\u2019s the process of conducting a replication?<\/p>\n\n\n\n

The replication process<\/h2>\n\n\n\n

Researchers could independently replicate the work of other teams, like at CERN. And that does happen. But when there are only two studies \u2013 the original and the replication \u2013 it\u2019s hard to know what to do when they disagree. For that reason, large multigroup teams often conduct replications where they are all replicating the same study.<\/p>\n\n\n\n

Alternatively, if the purpose is to estimate the replicability of a body of research \u2013 for example, cancer biology<\/a> \u2013 each team might replicate a different study, and the focus is on the percentage of studies that replicate across many studies.<\/p>\n\n\n\n

These large-scale replication projects have arisen around the world and include ManyLabs<\/a>, ManyBabies<\/a>, Psychological Accelerator<\/a> and others.<\/p>\n\n\n\n

Replicators start by learning as much as possible about how the original study was conducted. They can collect details about the study from reading the published paper, discussing the work with its original authors and consulting online materials.<\/p>\n\n\n\n

The replicators want to know how the participants were recruited, how the data was collected and using what tools, and how the data was analyzed.<\/p>\n\n\n\n

But sometimes, studies may leave out important details, like the questions participants were asked or the brand of equipment used. Replicators have to make these difficult decisions themselves, which can affect the outcome.<\/p>\n\n\n\n

Replicators also often explicitly change details of the study. For example, many replication studies are conducted with larger samples \u2013 more participants \u2013 than the original study, to ensure the results are reliable.<\/p>\n\n\n\n

Registration and publication<\/h2>\n\n\n\n

Sadly, replication research is hard to publish: Only 3%<\/a> of papers in psychology, less than 1%<\/a> in education and 1.2%<\/a> in marketing are replications.<\/p>\n\n\n\n

If the original study replicates, journals may reject the paper because there is no \u201cnew insight.\u201d If it doesn\u2019t replicate, journals may reject the paper because they assume the replicators made a mistake \u2013 remember the salt crystals.<\/p>\n\n\n\n

Because of these issues, replicators often use registration to strengthen their claims. A preregistration<\/a> is a public document describing the plan for the study. It is time-stamped to before the study is conducted.<\/p>\n\n\n\n

This type of document improves transparency<\/a> by making changes in the plan detectable to reviewers. Registered reports take this a step further, where the research plan is subject to peer review<\/a> before conducting the study.<\/p>\n\n\n\n

If the journal approves the registration, they commit to publishing the results of the study regardless of the results. Registered reports are ideal for replication research because the reviewers don\u2019t know the results when the journal commits to publishing the paper, and whether the study replicates or not won\u2019t affect whether it gets published.<\/p>\n\n\n\n

About 58%<\/a> of registered reports in psychology are replication studies.<\/p>\n\n\n\n

Replication research often uses the highest standards of research practice: large samples and registration. While not all replication research is required to use these practices, those that do contribute greatly to our confidence in scientific results.<\/p>\n\n\n\n

Replication research is a useful thermometer to understand if scientific processes are working as intended. Active discussion of the replicability crisis<\/a>, in both scientific and political spaces, suggests to many researchers that there is room for growth. While no field would expect a replication rate of 100%, new processes among scientists aim to improve the rates from those in the past.<\/p>\n\n\n\n

Amanda Kay Montoya<\/a>, Associate Professor of Psychology, University of California, Los Angeles<\/a><\/em><\/p>\n\n\n\n

This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Amanda Kay Montoya, University of California, Los Angeles Back in high school chemistry, I remember waiting with my bench partner for crystals to form on our stick in the cup of blue solution. Other groups around us jumped with joy when their crystals formed, but my group just waited. When the bell rang, everyone left […]<\/p>\n","protected":false},"author":56,"featured_media":39934,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[292,42,10,3410,15533],"tags":[885,891,886,860,16658,228,16657,16659,4036,4042,193],"_links":{"self":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/39933"}],"collection":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/users\/56"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/comments?post=39933"}],"version-history":[{"count":1,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/39933\/revisions"}],"predecessor-version":[{"id":39935,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/39933\/revisions\/39935"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/media\/39934"}],"wp:attachment":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/media?parent=39933"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/categories?post=39933"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/tags?post=39933"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}