Yesterday I attended the 40th annual summer research colloquium of the Smalley-Curl Institute at Rice, a fun internal conference that provides a great opportunity for undergrads (including visitors), graduate students, and a few postdocs to present their work. The keynote speaker was our EVPR, Prof. David Sholl, who gave a very informative talk about reproducibility in the chemical engineering/materials literature. We hear a lot these days about crises of reproducibility in scientific research, and Prof. Sholl rightly points out that in some fields the expectation of reproducible results is high - no one would spend $1B on a chemical engineering plant if they weren't very sure that the catalytic processes were going to work as expected at scale. Keys to reproducibility include, unsurprisingly, repeated results and independent replication. One metaresult that was interesting is this paper, looking at the literature on metal-organic frameworks and how often there are published replications of syntheses; not as often as you would think or want!
A truly surprising (to me, anyway) result is this one. The Brunauer–Emmett–Teller (BET) (yes, that Teller) method is a long-established technique that uses gas adsorption measurements to infer the surface area of porous materials. Many research groups were given identical raw adsorption isotherms and asked to calculate the specific surface areas, resulting in a surprisingly large spread of results (Fig 1 of the paper). Clearly not everyone had the same analysis procedures even for a technique developed in the 1930s!
Some take-away lessons from this are encapsulated here, in an article titled "Five easy ways to make your research more reproducible". Good stuff. The talk raised a number of questions relevant to our present era of huge enthusiasm about AI-based materials research and "self-driving" labs. If the AI models are all trained on the literature, and the literature is not representative of complete and reproducible procedures, that's a problem.
One personal anecdote about reproducibility and its challenges in materials synthesis. Twenty years ago (!), I was working with a colleague who had a postdoc who was synthesizing Fe3O4 (magnetite) nanoparticles via wet chemistry methods (see here). We did some fun electronic transport experiments bridging very closely spaced electrodes with such nanoparticles, and we saw some very dramatic hysteretic response kick in as \(T\) was reduced below about 120 K. That's the temperature of the Verwey transition in magnetite, where the material enters a more insulating low temperature phase. Basically all of the devices we made with that batch of nanoparticles showed this phenomenon. Then the postdoc took up a faculty position and a senior grad student came in and took over the synthesis, and for several months, subsequent batches of nanoparticles just didn't seem to show the effect. The key issue is oxygen stoichiometry. Get a little oxygen rich, and you form nanoparticles that include some \(\gamma\)-Fe2O3, which doesn't have the Verwey physics and in nanoparticle form looks really similar in x-ray diffraction to the desired magnetite. Anyway, we started working with a collaborator who could grow epitaxial Fe3O4 films, and in those devices the electronic effect was there all the time. All this led to this publication and subsequent papers, and I still think it's a cool set of result about a nonequilibrium transition in a correlated material. In the end, after several months the chemistry grad student did get back to making nanoparticle batches that showed the transition. It turns out that at some point he had changed the length of a piece of tubing in the gas manifold, and unexpectedly that had altered the reaction kinetics just a little. Changing it back got the synthesis to be reliable again. This is an example of how finicky materials synthesis can be!
No comments:
Post a Comment