Tuesday, December 19, 2017

The state of science - hyperbole doesn't help.

It seems like every few weeks these days there is a breathless essay or editorial saying science is broken, or that science as a whole is in the midst of a terrible crisis, or that science is both broken and in the midst of a terrible crisis.  These articles do have a point, and I'm not trying to trivialize anything they say, but come on - get a grip.  Science, and its cousin engineering, have literally reshaped society in the last couple of hundred years.  We live in an age of miracles so ubiquitous we don't notice how miraculous they are.  More people (in absolute numbers and as a percentage of the population) are involved in some flavor of science or engineering than ever before.

That does mean that yes, there will be more problems in absolute numbers than before, too, because the practice of science and engineering is a human endeavor.  Like anything else done by humans, that means there will be a broad spectrum of personalities involved, that not everyone will agree with interpretations or ideas, that some people will make mistakes, and that occasionally some objectionable people will behave unethically.   Decisions will be made and incentives set up that may have unintended consequences (e.g., trying to boost Chinese science by rewarding high impact papers leads to a perverse incentive to cheat.).   This does not imply that the entire practice of science is hopelessly flawed and riddled with rot, any more than a nonzero malpractice rate implies that all of medicine is a disaster.

Why is there such a sense of unease right now about the state of science and the research enterprise?  I'm not a sociologist, but here's my take.

Spreading information, good and bad, can happen more readily than ever before.  People look at sites like pubpeer and come away with the impression that the sky is falling, when in fact we should be happy that there now, for the first time ever, exists a venue for pointing out potential problems.  We are now able to learn about flawed studies and misconduct far more effectively than even twenty years ago, and that changes perceptions.  This seems to be similar to the disconnect between perception of crime rates and actual crime rates.

Science is, in fact, often difficult.  People can be working with complex systems, perhaps more complicated than their models assume.   This means that sometimes there can be good (that is, legitimate) reasons why reproducing someone's results can be difficult.  Correlation doesn't equal causation; biological and social phenomena can be incredibly complex, with many underlying degrees of freedom and often only a few quantifiable parameters.  In the physical sciences we often look askance at those fields and think that we are much better, but laboratory science in physics and chemistry can be genuinely challenging.  (An example from my own career:  We were working with a collaborator whose postdoc was making some very interesting nanoparticles, and we saw exciting results with them, including features that coincided with a known property of the target material.  The postdoc went on to a faculty position and the synthesis got taken over by a senior grad student.  Even following very clear directions, it took over 6 months before the grad student's particles had the target composition and we reproduced the original results, because of some incredibly subtle issue with the synthesis procedure that had changed unintentionally and "shouldn't" have mattered.)

Hyperbolic self-promotion and reporting are bad.   Not everything is a breakthrough of cosmic significance, not every advance is transformative, and that's ok.  Acting otherwise sets scientists and engineers up for a public backlash from years of overpromising and underdelivering.   The public ends up with the perception that scientists and engineers are hucksters.  Just as bad, the public ends up with the idea that "science" is just as valid a way of looking at the world as astrology, despite the fact that science and engineering have actually resulted in technological society.  Even worse, in the US it is becoming very difficult to disentangle science from politics, again despite the fact that one is (at least in principle) a way of looking at the world and trying to determine what the rules are, while the other can be driven entirely by ideology.  This discussion of permissible vocabulary is indicative of a far graver threat to science as a means of learning about the universe than actual structural problems with science itself.  Philosophical definitions aside and practical ones to the fore, facts are real, and have meaning, and science is a way of constraining what those facts are.

We can and should do better.  Better at being rigorous, better at making sure our conclusions are justified and knowing their limits of validity, better at explaining ourselves to each other and the public, better at policing ourselves when people transgress in their scientific ethics or code of conduct.

None of these issues, however, imply that science itself as a whole is hopelessly flawed or broken, and I am concerned that by repeatedly stating that science is broken, we are giving aid and comfort to those who don't understand it and feel threatened by it.


Anonymous said...

Science requires quality control and ethics of giving credit to the right person. Jeffrey Hall in his Nobel Speech has said about PI (Principal Investigator) and AI ( Actual Investigator). He goes to the extent of saying PIs do little work. Surely he let of his pent up emotions on the right occasion. This PI and AI he says towards the end of the speech. Jeffery left active research 10 yrs ago.

His speech below

Stanford has started a dept Meta research foundation


The video from the above web site is interesting

Hope something good comes out of research on research.

Anonymous said...

I agree with your assessments, but I want to add a few perspectives.

Science reporters need something to report, and clicks equal cash in the current setup. Sensationalized reporting and simply wrong science reporting is a real problem. (A fun video: https://www.youtube.com/watch?v=0Rnq1NpHdmw) Part of the 'science is broken' mantra, I think, really starts from the place that lots of people don't know what to make of the information they're given and we're not good at helping.

Also, as you say, science is often complicated. I'd point out that today, by and large, we are off into the weeds of every topic in all the fields. Sure, this is necessary. Sure, there are a lot of big questions to be answered. Sure, there are people working on big topics with huge implications that everyone can really understand, simply. But that certainly isn't the bulk of science today. In today's age we don't see 7-page theses as in days of old (just half a century ago even). Even if something was so simple as to be explained that way, today, advisors would never let that student out on such simplicity, regardless of how big or groundbreaking that topic may be. They'd keep that student around to get the most out of them (especially their next funding round). There's an overdriving notion that it should be hard and rigorous and you have to 'do your time', and with most of the science way out in the weeds, hard and rigorous are true and needed. But then there's that contradiction that most people aren't interested in the minutia of obscure fields or really on reporting the non-sensationalized version. Scientists know the importance of hard and rigorous, but do very little as a whole to help the public understand it, outside of hard, rigorous, and off in the weeds.

This leads me to say that science doesn't have a problem, but rather our treatment of it is. Having 3 generations of PhD physicists in my family, the last 2 are completely dismayed at the hoops grad students have to jump through these days in the name of rigor. (One has a 12 page PhD thesis.) 'It has to be hard and rigorous!' Those that publish sensationalize and mis-report for lack of real understanding, and arguably our ability to make it clear.

As scientists, we are doing something wrong to help ourselves out. Often because we're too busy hacking our way through the weeds, rigorously, on the really hard stuff.