Scientific shaky ground
PRINT PAGE
Scientific shaky ground
In 2014 a young Japanese female researcher named Haruko Obokata came into the limelight in the male dominated area of stem cell research. At 30 years old she was already the head of her own laboratory and was a ‘rock star’ of the scientific research community.
She was the complete picture. Young, beautiful, intelligent, and quirky, she was known to wear a kitchen apron in her lab instead of a lab coat and she adorned her lab with cartoons and pink. She was hailed as a national hero when she produced world-changing research by demonstrating how to turn ordinary body cells into a cell similar to stem cells with an incredibly simple laboratory process. These cells were reported to be able to abundantly multiply into any type of cell in the body, an ability called pluripotency, which had major ramifications to medical health science.
However, her meteoric rise to fame and stardom was short-lived when, several days after her papers were published in Nature, disturbing allegations of plagiarism and doctored images used in her research findings emerged.
Several research labs attempted to reproduce Obokata’s findings in order to build on the research, but could not reproduce the original results. An investigation soon took place and found Obokata guilty of scientific misconduct. Obokata’s supervisor Yoshiki Sasai, who mentored young Obokata and validated her research was, in his words ‘overwhelmed with shame’ and after a month in the hospital being treated for depression, took his own life in the stairwell of a research facility.
Obokata joined the ranks of an increasing number of other high profile scientific fraudsters who have all experienced high profile public cases of research misconduct resulting in the retraction of published papers and millions of dollars of research funding.
In 2015, another notorious case involved a research scientist named Dong Pyou-Han who was sentenced to 57 months of jail for fabricating and falsifying data in HIV trials. He was also ordered to pay back $7.2 million in research funding.
Recently in Australia, the Vice Chancellor of Monash University, a Professor of Sociology from the University of South Australia, a skin cancer researcher from the University of New South Wales, and a professor at the Queensland University of Technology have all been exposed and stood down for research misconduct.
This list could go on for days, but it is apparent that research misconduct has become so widespread and common that it is now a research topic in itself. A review of the 2,047 retractions listed in PubMed as of 2012 found a majority (67%) were attributable to misconduct[1]. A 2012 survey by the British Medical Journal found 13% of surveyed researchers admitted knowledge ‘inappropriately adjusting, excluding, altering, or fabricating data’[2].
This raises the question of the very integrity of scientific research as a whole upon which we base many of our decisions as a society.
A recent survey in Nature conducted on over 1500 scientists revealed that more than 70% of researchers tried and failed to reproduce another scientist’s experiments and more than half failed to even reproduce their own experiments[3].
Data on how much scientific literature is actually reproducible has recently been a focus for reputable scientific journals and biomedical research companies alike whose reputation and decision making depend on reliable research results.
Glen Begley, the head of cancer research for the multi-national pharmaceutical company Amgen, recently attempted to replicate 53 landmark cancer research experiments published in the world’s most reputable journals. He and his team only managed to replicate 6 of the 53 studies; a shocking 11%[4].
In a similar study, researchers at Bayer attempted to replicate 67 studies into the therapeutic potential of several cancer drugs, with only a quarter being able to be reproduced[5]. Another study estimated reproducibility to be between 20-40%[6] in the field of psychology. This data is actually giving us quite damning evidence that over three quarters of peer reviewed published scientific research in mainstream journals is actually irreproducible.
What does this mean?
Well, for those of you who did not pay attention in your high school science class, reproducibility is one of the main tenets and cornerstones of scientific method. Being able to independently reproduce a scientific experiment based on what is published is what gives the study ‘reliability’. The idea behind scientific reliability is if an experiment is to be considered scientific proof, it must be more than a one-off finding. It must be able to be reliably repeated and the results reproduced.
So, does this mean that around three quarters of all peer reviewed scientific research is not reliable and therefore invalid and cannot be considered scientific proof?
Well, according to the founding tenets of scientific methodology, yes. This is actually such a big “can of worms” that Nature has now published on their website a special feature page dedicated to editorials, features, and news/analysis on the epidemic of irreproducible research. Entire websites and organizations, are springing up to highlight the problem, such as Retraction Watch [7].
Why don’t scientists attempt to reproduce studies more often to improve reliability? The most obvious reason is that research costs money and funding for replication studies is scarce and unavailable. Furthermore, there is very little glamour or personal incentive in doing something that someone else has already done, as most scientists would rather spend their time and money discovering something new. Some labs are introducing methods to improve documentation and standardisation procedures to introduce reproducibility measures. However, it is estimated that this doubles the monetary cost and time involved in undertaking a study and therefore, only high profile studies may occasionally have this check and balance.
Before we condemn all scientists as being egotistical charlatans, we have to deepen our understanding of why research misconduct happens in the first place. To do this we have to take a closer look at our current system for publishing scientific research. To publish a study in any prestigious scientific journal, your research must undergo what is known as the peer review process.
This is the process by which several other research scientists who are deemed experts in their respective fields, scrutinize the statistics, research methodology and interpretation of data presented in the submitted paper. The peer reviewers can then either give their full stamp of approval to publish, publish with major/minor edits, or reject the paper altogether.
Now, this may seem like a rigorous process, but a few important facts in this process must be highlighted:
-
The reviewers do not get access to the original raw data, and even if they did, it would not be practical for them to re-analyse it;
-
Remember how we said that the founding tenet of what makes a study scientifically valid is that it is reproducible? Well, reviewers do not attempt to replicate the study to test its reliability, but instead make a judgement call based purely on reading the article.
Herein lies the temptation: picture yourself as a scientific researcher who has spent tens of thousands of dollars on your education to get a PhD. Among the 6000 others who also received one of those degrees every year, you are one of the lucky 5% who actually found a paid job. However, your job is not permanent and to keep this job you have to compete with other hungry researchers to obtain funding and the primary way to compete and get funding is the number of papers you publish in peer reviewed journals.
Now imagine that you have made it to the top of this heap and you have spent several years and hundreds of thousands, if not millions of dollars on a research project and you are at the finish line, but when you analyse the data there is a niggling and bothersome bit that is sticking out and not giving you the results you need. All you need to do is push the delete button on your keyboard to get rid of those bits that are sticking out and all is good and no one will be any the wiser…
This is the scenario that is all too commonplace in our scientific communities.** When fraud is expertly undertaken in a modest manner and especially in a subject area that does not promote you to instant science rock star status, then the reviewer is highly unlikely to identify the problem and the dodgy research passes the gates undetected.** After all, the research will never even attempt to be replicated so nobody will ever even detect the fraudulent data.
What we now have to realise is that our scientific communities and indeed, most of our intellectual and knowledge base, holds published peer reviewed research in prestigious journals as the ‘gold standard’ for what is deemed reliable research.
However, if these very journals are starting to call out the fact that an overwhelmingly large proportion of these published studies are unreproducible and thus unreliable, then where does that leave us?
Furthermore, what does this say about organisations such as The Friends of Science, who wield the false and increasingly tarnished peer reviewed and published scientific ‘gold standard’ patina to attack complementary health care, based on the premise that the evidence provided from thousands of case studies is not scientifically reliable?
This is another discussion in itself. However, we must realise that our current systems for promoting scientific research – that derives the majority of its funding from a competitive model that is based on an assessment on the number of research articles published by a laboratory or individual – is not actually producing reliable scientific knowledge.
Instead, it is producing a scientific community that is increasingly pressured to do anything it can, including lying cheating, stealing, and backstabbing to secure funding and retain authority.
Meanwhile, invaluable research in areas such as complementary medicine goes unpublished due to lack of funding or is crushed under the false authority of not being scientifically reliable.
Evidence based research has a significant and important role in science. There is no doubt about that. However, it takes great faith to believe that we should base our entire philosophical understanding of life purely on a systematic way of thinking that is being increasingly shown by the application of its own tenets to be fundamentally flawed.
As informed individuals we have to educate ourselves, be aware, and look at all research with a critical eye, but also an open heart. We cannot give our choice and ability of discernment to organisations that claim to have the authority over what is reliable research and what is not. These very organisations – such as Friends of Science – are using increasingly shaky ground to stand over others in a bid to push their own corporate agendas and personal egos. Eventually, this increasingly shaky ground upon which they stand, will break apart – as evidenced by the prevalence of the rather large cracks already showing in the system. As individuals and communities, we must ask ourselves, how much of our belief systems and ways of living are based upon this shaky ground, and most importantly, where does it leave us when we realise we have built an entire society on these grounds?
References:
Filed under
