Play it again Sam – last night’s leftovers and reproducibility in science, Part 2
PRINT PAGE
Play it again Sam – last night’s leftovers and reproducibility in science, Part 2
In Part 1 we opened up the discussion about reproducibility by comparing it to preparing a roast dinner – and the fact that no matter how hard we try, it never quite turns out the same.
Let’s take the analogy further: it is a meal you are preparing for your very difficult potential in-laws or your impossible-to-impress boss. Now the pressure is really on, especially if you are hanging on their approval.
Perhaps you can now put yourself in the place of a scientist carrying out a study to which you have dedicated years of your life (not days or weeks – most experiments take years to develop, carry out and analyse) and into which many hundreds of thousands, up to millions of dollars, have been poured by very interested investors.
Then imagine having your career and your ongoing position within your research institution and your next grant for funding (the money that is essential to you keeping your job and carrying out your work) absolutely dependent upon your getting a promising, i.e. positive, result.
Yes, there is quite of bit of external pressure on scientists to come up with the goods.
Then pour into the mix your professional standing and your reputation, built on your capacity to develop great studies, yield impressive results and get published in prominent journals ... and perhaps it is easy to see how the determination to get a good result can nudge the most well meaning scientist off an honest course.
Little wonder that scientists comb through their own data with a very detailed and perhaps biased eye, looking for those links, associations and patterns that yield some sort of indication of a positive result to show that all of their work has not been for naught. It has also been reported recently that scientists have been found to 'switch outcomes' in trials[i]. This means that they are being selective about what they report, and failing to disclose what their objectives were at the outset of the study – in essence, misguiding their readers. All of these apparently harmless tricks impact on the reliability of the results and their reproducibility.
We need to understand this as a systemic issue as well as a personal one, because scientists are regular people, just as vulnerable and needing of approval and a steady wage as the rest of us.
The system is not set up to support reproduction:
- At an institutional level – they do not have the funds to allocate to experiments that do not advance science or their reputations and, let’s be honest, no one gets kudos for doing something the second time
- At the level of funding – whether funding is sought from a private investor or government institution – it is very hard to justify giving more money to a project that has already been completed
- At the level of publication – as publication is vital to a career in science. The journals are hungry for exciting new content that pushes the boundaries, not last year’s experiment, re-done. It is as appealing as last night’s leftover lamb ... reheated for too long in the microwave.
The reproduction of a study is a fraught exercise, because failure to reproduce results leads to more than mistrust in the first experiment. Failure to reproduce a study will leave a bad smell around the scientist/s who first published it. This will compromise their careers and, if serious enough to look like flagrant corruption, will bring about their end. In some cases, results cannot be reproduced because there were no actual results in the first place. There are scientists, perhaps desperate to make a name for themselves, who have completely fabricated their results out of thin air[ii].
But let’s take a step back and place reproducibility in a bigger context . . .
An important factor that influences all we do, (although we do not give it due consideration), is the state of being we are in when we do those things. Our level of conscious presence, our emotional state and our need to fulfil an agenda have a marked impact on the results of any activity. We learn how powerful this is when we start to consciously make life a living science. It is just as powerful and impactful in the science laboratory.
We know that when we are feeling great we will generally have a great day in which everything flows very smoothly and things work well. We tackle issues with ease, and on a very good day we will have a free flow of inspiration and understanding. Our bodies are an intimate part of the act, and the science experiment, which is actually a very physical pursuit, will go very smoothly with its very precise activities.
A lousy day will throw up one challenge after another. Equipment will not work ... guaranteed. The essential chemical bottle will not be where it ought to be, interrupting your flow. Clumsiness can kick in, causing errors that throw the experiment out. Lack of presence will cause errors because you just were not paying attention. Keep in mind an incremental error in measurement at the beginning can create disastrous outcomes at the end, and if your mind is on your sick dog and the vet bill that is on its way, focus can get a bit compromised and errors creep in.
All of these are very natural concerns and considerations when we hold a very strict view of how science needs to be, must be, for us to rely on it so completely as we do.
Let’s take a bigger step back, to widen our vista, such that . . .
. . . the scientist and their laboratory, reproducibility and science – are placed in a context that encompasses the world and possibly this universe. We live in a world where nothing, absolutely nothing stays the same. This is a very contentious statement for our consideration. What it means to live in an expanding universe never enters discussion in scientific circles, except perhaps astrophysicists’ tea parties, but do they wonder at its relevance to human life and its implications in science?
It is a consideration that challenges what we are trying to achieve when we seek to make science reproducible, for that wide, encompassing vista acknowledges that we are an inseparable part of a whole planet ... and, as much as we try to ignore it, the inconceivable enormity of the Universe. That wider vista also challenges our understanding of the true nature of science and what it means to human life.
The simple fact is that we live on a planet that is constantly evolving and we are part of a Universe that is in every single moment expanding.
The particles that constitute everything on our planet, including the ones that make us, answer to this immense Universal pull, no matter how much we like to think we can control life, limit it to the immediate confines of our planet, our nation, our university or institution or our laboratory, and thus control it enough to keep everything the same. Thus, are we trying to use our science to pin life down to a single, static point and say that is 'it' ... the one and only answer?
This begs the question as to whether any scientific study can in fact ever be reproduced at all, when the entire world (and indeed the whole universe) changes every moment and has moved on from the moment that the experiment was first conducted?
Are we in fact pulling against evolution when we try to hold on to one moment in time, and reproduce it over and over?
If we really want to restore trust to science, and get Scientist Sam to 'play it again', there is a great deal more to consider than the way bias, error and wishful thinking influence results. It begs for us to explore what reproducibility means when we take into account the evolutionary pull of the whole Universe.
Could it be that reproducibility can only ever be found in the quality and state of the human being doing the experiment, and not ever in the results they yield? This includes their state of being in every aspect of their lives, not just when they are in the laboratory and at work. Would reproducibility then be more responsive to the broader conditions that are influencing us constantly, hence not tied down to plotting the same numbers on a chart?
Even the state of being of a scientist with awareness of this fact will not be a constant. The only reproducible factor for such a man and woman will be their relationship with evolution, and with their deepening connection to inner qualities. Dare we say in an article about science, their connection to their Soul and to Universal Truth?
This is a conversation worth exploring, especially in light of the eroded trust that has blighted science in recent times. It takes it back to the grassroots of the human being in science, the ‘Sam’ we are asking to ‘play it again’. It also asks us to be less reliant on forcing the same results, compartmentalising and reducing life to a set of fixed answers.
This will be one of the ways we will restore science to its broader, truer context, one that is open to the energetic flow of life and the inconceivable mystery that makes everything the way it is.
This calls for us to reawaken our wonderment and awe from their slumber, for they are the greatest antidotes to the ever more demanding needs for control over science through funding, publication and the playing of the same old tune we left behind long ago.
Filed under
Evidence-based, Bullying, Corruption, Evidence, Intelligence
