I hope to write more about some new papers in the literature soon, as well as providing links to some pretty cool physics-related content. First, though, I want to say something about this preprint, which has people talking. The article says, in the case of an ensemble of social sciences papers, anyway, that AI-based review is better, in nearly every quantifiable way, than human peer review. This question of AI for reviewing papers and proposals has been in the air for a bit now, as in this article in Science, this editorial at Nature Nano, and the fact that the present administration has been using AI tools to "review" grants for content that allegedly goes against executive orders.
I'm an associate editor at a journal, and I see a large volume of papers and reviews. (Note that this post reflects my personal views, not an official take by the journal or its publisher.) It is 100% clear that some reviewers are using AI tools to prepare their reviews, despite clear statements that this is not ok. There is a distinct format that AI reviews take. After a summary paragraph, usually there are five specific bullet points, one or two of which tend to zero in on some particular detail or peculiarity in wording of the paper in a way that humans probably wouldn't. Often the phrasing is somewhat aggressive, with a bit of a "gotcha" tone. Of course, in any given case, I can't prove AI was used, but if you read enough of these, they stand out. To be clear: I'm not talking about people who use AI to clean up grammar. I'm talking about people feeding articles to an AI tool and asking it to spit out a review.
The point of peer review, whether it's for grants or articles, is to get the informed views of the relevant scholarly community about the work. We want to know what actual practitioners think of the results and their presentation. Purely AI reviews just do not serve this purpose.
Moreover, there are serious confidentiality issues at stake. Sure, many scientists use preprint archives these days to put up prepublication work, but plenty still do not, especially for very hot topics where people are worried about being scooped. Running submitted manuscripts or grant proposals through AI tools that can then scrape the content into their training sets is a fundamental violation of the confidentiality of the process. Imagine someone uploads your manuscript about a machine learning search for a new high temperature superconductor to Gemini to generate a review, and then amazingly SandboxAQ later scoops you on this. (Not accusing google of anything here, just a hypothetical example. Feel free to say GPT and Mattergen.) The math community already worries about this stuff.
So: If you are asked to review a paper or a grant proposal, bear in mind (1) the people doing the asking actually want your feedback, and (2) taking confidential documents and allowing them to be scraped by AI companies is bad.
1 comment:
Right. If the editor wanted to put in zero effort, they would just prompt the AI themselves and generate some slop. Also, the paper being read and critiqued by a human makes it more likely that other humans will understand and be able to build on the work.
Post a Comment