Tuesday, September 04, 2018
Looking back at the Schön scandal
As I mentioned previously, I've realized in recent weeks that many current students out there may never have heard of Jan Hendrik Schön, and that seems wrong, a missed opportunity for a cautionary tale about responsible conduct of research. It's also a story that gives a flavor of the time and touches on other issues still current today - faddishness and competitiveness in top-level science, the allure of glossy publications, etc. It ended up being too long for a blog post, and it seemed inappropriate to drag out over many posts, so here is a link to a pdf. Any errors are mine and are probably the result of middle-aged memory. After all, this story did start twenty years ago. I'm happy to make corrections if appropriate. update 9/9/18 - corrected typos and added a couple of sentences to clarify things. update, 2020: This write-up is now deposited at Rice's scholarship and has a doi: https://doi.org/10.25611/8P39-3K49 .
Subscribe to:
Post Comments (Atom)
23 comments:
Nice summary, Doug. This is a service to the history of this sad episode, and I hope it can be archived somehow.
The only significant omission I noted was that the inspiration for the project came not just from Bertram Batlogg but also the late Bob Laudise. The idea that one could learn about the ultimate limits of organic materials, such as polymers, by studying high-quality crystal of small molecules with van der Waals bonding was a good one, I think, although it proved vulnerable to wishful thinking.
I also think that the spin-off of the silicon and optoelectronics businesses from Lucent was important in another way than you mentioned. It became critically necessary for Bell Labs management to demonstrate results that were simultaneously (1) unrelated to the spun-off businesses and (2) convincingly relevant to technological applications. The organic electronics work fit the bill perfectly, and was featured as 2 of the 3 Bell Labs results in Lucent's annual report in (I think) 2001.
Sorry to say I don't remember being the reviewer on your papers, but I hope my contribution was positive. An important but less formal part of the culture that you allude to is the presentation of exciting results in internal seminars, where people could be honest to the point of being rude. That part of the quality control was made ineffective in this case, because it became clear that skeptical questions were not welcome. That was a huge mistake.
Nice article. I still remember reading in astonishment the incredible achievements of Schön. It was like physics was being completely reinvented. And then it all came crashing down. Apart from Batlogg, I really find it surprising that serious collaborators like Dodabalapur did not think about rechecking the claims. This account deserves wider audience.
I agree with Don that this is a very nice summary --- you give a great description of the atmosphere. I'm not sure how much I agree with Don that sceptical questions were not welcome. I seem to recall a lot of questions --- but they all were more technical like ... "how do you know the zero resistance superconductor you are measuring isn't a short" and things like this. There were certainly plenty of these questions and everyone seemed to have their own conjecture as to what piece was incorrect and why. Few, if any, said out loud (until fairly late) that they thought it was all made up. Particularly with the blessing of Batlogg behind it, that seemed just impossible. The problem is that our community is well designed to find scientific errors --- but not well designed to find fraud.
Thanks all. Don, your comments and feedback did help my papers :-) Steve, were there actually any internal talks at all about this stuff in 1999 or 2000? I don't remember any internal seminars about the work while I was there, just surprise when things appeared in print.
Thanks for the clarification, Steve. My statement was no doubt an oversimplification based on hearsay, and it is worth more critical assessment. I do remember hearing about a seminar where critical questioning was shut down by management. I also remember hearing about other staff members finding their productivity, impact, and spending being compared unfavorably to Hendrik when they asked management for resources. On the other hand, my own questions were taken very seriously when I offered them.
What I think we can agree on was that the internal quality-control process was not effective and a lot of garbage went out the door to pollute the larger scientific community.
Very nice writeup Doug. I was a graduate student at the time, working on high-mobility semiconductors, and remember all the excitement surrounding Hendrik Schon's visit at our university. During his talk, he was cataloguing all the marvelous results, and saying, after each one, "I don't have time to get into the details" (how convenient!). Then, after the talk, a lot of questions from frustrated researchers who had been trying to reproduce the results.
I also remember thinking of his results that, in spite of all the Science and Nature papers, and of the praise that Hendrik was getting for making his devices work "as intended", there wasn't really anything striking in the data. Given the novelty of the materials system, you would expect some anomalies, some unusual dependencies that are not immediately explained. Now we know why...
You ask about the sanity of someone who would fabricate so much data. I think at some point he said something of the sort: "Well, I measured this organic sample, didn't get any good data, but I knew that it *should* behave in such a way [e.g., as an ambipolar FET), so I just drew a curve that showed what the data should look like."... and that's what he published. Logical, isn't it?
Thank you for sharing these reflections, which complement the other discussions of the scandal. Weaving in the contemporaneous events (tech boom and bust, 9/11) was illuminating.
Going back to the new superconductivity paper, I think it makes some sense that the authors don't want to share their samples before the paper gets accepted somewhere. After all, a Nobel prize is at stake here. Nobody wants to get scooped. However, if no respected journal wants to accept this paper after a while, it will be safe to say that the authors' credibility will be tarnished. Therefore, I am surprised that they chose to post such a splash paper to arxiv before it got accepted somewhere. Thus, everybody got alerted rapidly. Therefore, I think that the the authors are either really daft or they are really into some major breakthrough (hence their confidence in posting it to arxiv). We will see soon...
Ericsson, not Erickson?
Thanks for nice "review" lest anyone forgets it, Doug. I was starting my graduate life with Behrang at that time, so I remember the fallout vividly. Since having lived through three U.S. corporate research environments, your noted malaise in governance and associated cognitive biases that favor short-term gains of money or influence (as exemplified by Batlogg and cohorts) seem to be the pervasive norm in the corporate R&D world very much. Their guiding principle seems to be "anything I can get away with is legit", since as you noted - these "management leaders" are short-timers who don't expect to be around to face the eventual fallout of their BS, if not outright fraud. I have observed VPs lose their jobs when they couldn't get out in time, but they mostly do "get away" and sometimes even promoted if they manage to throw the right adversary under the bus for political gains.
Doug and Don: Yes, there were some internal seminars by Schoen that I remember, as well as by his co-authors advertising his results (I certainly recall Bao and Kloc at least). But you are right that many results appeared first in print and were not circulated internally first. I do agree with Don that the internal quality control process was not as rigorous as it could have been. Back in the grand old days of Bell each department head was an expert in everything that was going on in their department, and you could be pretty sure that if a department head signed off on something it had been looked at pretty carefully. I don't know if Schoen's department head was signing off on his papers (It is probably in the Beasley report exactly which papers were signed-off by whom), but with all the reorganization, by that time, department heads were certainly not expert in what was going on in their departments (ex: I was put in charge of everything from biology to material science at some point). In Schoen's case, his department heads certainly were not expert in his field, maybe not even remotely familiar with the field.
I should also say that at some points the management did ask Schoen to put a halt to his rapid publication for a while until things could be looked at internally (I think this was about 6 months before Sohn announced the multiple reuse of the same plots --- actual discovery credit goes to the Lynn Loo's apparently eidetic memory). I think this instruction was mostly ignored. My recollection is that when the story broke, Cherry Murray called a meeting of the department heads to explain what was going on and Dick Slusher said "Didn't we ask him to stop publishing?", and Cherry said, "We did. He didn't."
Was Gershenson's 2006 RMP (https://journals.aps.org/rmp/abstract/10.1103/RevModPhys.78.973) also part of the post-Schoen cleanup?
Doug, thanks for documenting your experiences with this. Finding ways to help today's students who weren't born when Cold Fusion came and went and the Schon saga occurred has enormous value. These two events are exemplars of what it means to do science correctly (by giving such dramatic counterexamples).
With respect to the internal quality-control process at Bell Labs, it is worth noting that most published research is done at universities, which by-and-large have NO PROCESS AT ALL for vetting research publications. This is customarily left to the journals, whose enabling role in this episode has not to my knowledge been adequately discussed in public.
FWIW, the Beasley committee decided not to explore either the roles of either the journals or Bell Labs, for a variety of reasons. We did feel compelled to address the co-author responsibility question to a degree.
Hi Doug, just wanted to thank you for an interesting read. I'm a psychologist, so the finer points of the physics are lost on me, but I've been interested in the Schon affair ever since I read "Plastic Fantastic". Thanks for writing this!
Hello all. Thanks for the kind words.
JeanTate, you are right. I blame autocorrect :-) I'll make the correction when I make a few minor edits.
Steve, interesting. Either those internal seminars were after I left or I was so immersed in my job search that I missed them.
Fox, yes, but I think that was more of a grass-roots effort by researchers in the field to show that they'd really learned some neat stuff, rather than an agency-directed survey.
Don, yes! I tried to make that point, but I could have been more emphatic. When you think about it, from the perspective of university research oversight, it's rather terrifying, and the average non-academic has no idea. They read about some terrible misconduct case in the news and think, how did the department/school/university not catch this, when the answer is, there is enormous trust placed in university investigators.
Another scandal going on
https://retractionwatch.com/2018/09/04/cancer-journals-retract-10-papers-flag-8-more-and-apologize-for-the-delay/
Doug,
This is a fantastic writeup. Following up on a comment Steve made earlier about Schön being questioned internally about his work, one aspect of Schön's "methodology" that really stuck out to me is how he was very adept at listening to people's critiques and questions about his results and then fabricating whatever results were necessary in order to address whatever concerns were raised. This perverse "ask and you shall receive" behavior served as both a defense of his published work and a source for new publications. Quoting from Reich's book:
"At one point an MTS in the lab next door, Harold Hwang, told Schön that it was problematic to claim superconductivity only on the basis of a drop in the resistance of his crystals, because of the possibility that an experimental artifact such as a short circuit might produce the same effect. Schön did not argue, but simply asked Hwang what would be needed [to] convince him. Hwang described a couple of follow-up experiments, and Schön duly came back and claimed to have performed one, leading to another publication in Science. Peter Littlewood, a theoretical physicist at Cambridge University who had previously been the head of the Department of Theoretical Physics Research and spent summers at Murray Hill, described the situation this way: 'You would say something, and then it would happen. You would get caught up in the progress of the subject. For a long time, I didn’t believe it could have been fraud because I didn’t believe one person could make all that up. Then I realized, we all made it up.' Unknowingly."
I'd also like to remark on a tangential point you made in your writeup in which you stated: "One big lesson I learned to appreciate at Bell Labs: Technicians and research scientists can be every bit as intellectually deep as PIs, regardless of the lack of a doctorate or the choice not to be an independent researcher." I studied the quantum Hall effect in GaAs samples that were grown and initially characterized and screened by Loren Pfeiffer, Ken West, and Kirk Baldwin after they moved to Princeton, and working with Kirk and Ken taught me the same lesson. I remember early on asking one of the older grad students about Kirk and Ken's background, and the reply was, "Oh, of course they have PhD's!" Eventually I would come to find out that they did not have doctorates, but nevertheless they had managed to have wonderful careers in physics and established stellar reputations among their colleagues. It was a great lesson for me that titles and degrees don't have to completely define who you are and what you can do as an engineer or scientist.
This is a long read.
https://peerscientist.com/news/here-are-the-7-biggest-problems-science-is-facing-according-to-270-scientists/
I still remember the time I was in the graduate school (~2010), the lecturer in one of my classes showed a documentary movie of this scandal (I forgot the movie name) instead of giving a lecture as usual.
I could not understand why he did like this, and I even told him that I would prefer a normal lecture.
Now, I really appreciate his abnormal lecture.
For all, be it students or profs.
https://physicstoday.scitation.org/doi/10.1063/PT.3.4019
This was an excellent read-- I actually was just discussing this with some students here recently, so it was nice to link to this. Thanks!
Finally got around to reading this after Chad Orzel linked to it on Twitter a few days ago.
Thank you for chronicling this history - I am certainly aware of many characters in this story. I was part of some research in Allen's lab (around 2010) into ionic-liquid gating for novel field effect devices - strange how this saga probably played a role in making that project available for me to work on!
Post a Comment