AI for "peer" review

I hope to write more about some new papers in the literature soon, as well as providing links to some pretty cool physics-related content.  First, though, I want to say something about this preprint, which has people talking.  The article says, in the case of an ensemble of social sciences papers, anyway, that AI-based review is better, in nearly every quantifiable way, than human peer review.  This question of AI for reviewing papers and proposals has been in the air for a bit now, as in this article in Science, this editorial at Nature Nano, and the fact that the present administration has been using AI tools to "review" grants for content that allegedly goes against executive orders.

I'm an associate editor at a journal, and I see a large volume of papers and reviews.  (Note that this post reflects my personal views, not an official take by the journal or its publisher.)  It is 100% clear that some reviewers are using AI tools to prepare their reviews, despite clear statements that this is not ok.  There is a distinct format that AI reviews take.  After a summary paragraph, usually there are five specific bullet points, one or two of which tend to zero in on some particular detail or peculiarity in wording of the paper in a way that humans probably wouldn't.  Often the phrasing is somewhat aggressive, with a bit of a "gotcha" tone.  Of course, in any given case, I can't prove AI was used, but if you read enough of these, they stand out.  To be clear:  I'm not talking about people who use AI to clean up grammar.  I'm talking about people feeding articles to an AI tool and asking it to spit out a review.

The point of peer review, whether it's for grants or articles, is to get the informed views of the relevant scholarly community about the work.  We want to know what actual practitioners think of the results and their presentation.  Purely AI reviews just do not serve this purpose.

Moreover, there are serious confidentiality issues at stake.  Sure, many scientists use preprint archives these days to put up prepublication work, but plenty still do not, especially for very hot topics where people are worried about being scooped.  Running submitted manuscripts or grant proposals through AI tools that can then scrape the content into their training sets is a fundamental violation of the confidentiality of the process.  Imagine someone uploads your manuscript about a machine learning search for a new high temperature superconductor to Gemini to generate a review, and then amazingly SandboxAQ later scoops you on this.  (Not accusing google of anything here, just a hypothetical example.  Feel free to say GPT and Mattergen.)  The math community already worries about this stuff.

So:  If you are asked to review a paper or a grant proposal, bear in mind (1) the people doing the asking actually want your feedback, and (2) taking confidential documents and allowing them to be scraped by AI companies is bad.

Update:  This paper is highly relevant, showing very interesting consistency issues with LLMs as reviewers.

Materials-related postdoctoral opportunity at Rice, 2026-2027 edition

See here.

The Rice Advanced Materials Research Institute has launched its 2026-2027 competition for prestigious postdoctoral fellowships - see here:  https://rami.rice.edu/rami-postdoctoral-fellowship-program for details.  The actual application link is here.

If you are interested and meet the criteria, I'd be happy to talk.  I have some ideas that lean into the materials for electronics direction, and other possibilities are welcome.  

Science policy/funding updates, early October edition

This post is a bit later than I'd wanted, due to a combination of work and wanting to be in the right frame of mind to write about what's new in science policy (directly and adjacent matters) and funding.

• The NSF did underspend their federal appropriation for FY26 by more than 20%, obligating only $6.37B out of more than $8B budgeted.   In terms of numbers of awards, they made 2410 fewer awards in FY26 than in FY25, and about 5000 fewer awards than typical in FY21-24.  
• This includes the almost complete evisceration of the Social, Behavioral, and Economic Sciences directorate.  As that Science news article points out, SBE has until now funded about 2/3 of all US research in the social sciences (including linguistics and archeology).  The presidential budget request has been calling for SBE's elimination for the last two years.  Congress has rejected that in its budgeting, but seemingly OMB has gotten its way regardless.  
• There are more NSF X-Labs topics on the way, and a webinar next week to talk more about the "AI for Physical Systems" one.  
• NSF has also created a "metascience" office and released a statement about collaborating with the UK on this.  I've given my take on metascience before - the idea of seriously studying which types and mechanisms of research funding and programs give the "best" "results" is good in principle, but extraordinarily difficult in practice.  The idea that sound conclusions can be drawn in just a few years is absurd.  
• On the unambiguously positive side (gotta take the win when there is one), the NSF did announce the surprise support of the National Math Stars effort. 
• NSF also announced the first of their new Grand Challenge programs, rumored to be a possible destination of (some of?) the unspent funding from the top point above.  No real information about them yet, though the Quantum+X one has a webpage where you can sign up for updates.  Given the description ("Whether you're a researcher at a university, a startup, or an established company, this prize challenge is your chance to help define the next era of quantum technology."), it's hard not to wonder if this is another conscious effort to have NSF be a funding vehicle for industry/business rather than longer-term research.
• These announcements happened in part at the "Science: A New Golden Age Summit" (link to White House release about this.).  The National Medal of Science was presented to various tech leaders (not scientists).  If any of my readers attended this event, I'd be very curious about a first-hand account.
• DOE has announced the Genesis Mission Graduate Fellowship program.  The idea is a 4-year PhD with "cocompetencies" between a discipline and AI, with required extended internships at DOE labs/user facilities and industrial partners. The actual call is here, with a LOI due on 9 November and a preproposal deadline of 23 November. 
• For those of you who are PIs in condensed matter physics, the new DOE notice of funding opportunities (pdf) is out.  Pay close attention to the condensed matter physics program, as its focus has shifted.  
• Similarly, the new Air Force Office of Scientific Research BAA is out, and as with DOE, the focus of the condensed matter program has shifted.  Both DOE and AFOSR are now explicitly very interested in quantum geometry.  

On the more general research landscape:

• It's pretty clear that no matter what happens in the upcoming US elections, the next two years at least are likely to be comparatively bleak for federal support of science and engineering research.  Fundamentally, regardless of what OSTP says and despite glossy announcements of new programs, it sure seems like the executive branch (OMB) still wants to curtail federal support for research, and there is not much that even an opposed Congress alone can do in the near term.
• The number of STEM PhDs entering the job market in 5-ish years is going to be 25% down or more, because funding uncertainty has led to two consecutive years of double-digit decreases in admissions.  Remember, only 10% or less of STEM PhDs go into academia, so this demographic drop is going to be largely felt by industry.  I remain surprised by the apparent apathy about this.  If I thought my technical workforce applicant pool for some jobs might decline by double digits, I'd at least have some concern.
• Perhaps industry is collectively assuming that AI will somehow make up for this.  OpenAI released 722 preprints out there this week with results in math and mathematical physics.  This is simultaneously impressive and awful, and much has been written about it.  See here for an article and here for a weekly digest, from Quanta.  Peter Woit is hosting an interesting discussion, and points to a collection of comments about this.
• The US administration also announced a rule change that would impose a $70000 charge for "curricular practical training" (the mechanism that allows international students on F1 visas to have internships during their time as students) and a total of $100000 in fees for "occupational practical training" (the mechanism for international students to be allowed to work in the US for a period after completing their studies).  The actual proposed rule change is here, and people can submit public comments here until November 9.  Personally, I think that this rule change is yet another component of a seeming concerted effort to make the US unattractive as a destination for international scholars, a strategy that is incredibly self-defeating for the country.  (Sure looks like a lot of the "yes, kill OPT" comments are verbatim copy/paste of the same text.  Seems like someone has seen prior public commenting efforts on other controversial rule changes and decided to mobilize.)  Like the prior proposed duration of status changes for F1 and J visas (injunction discussed here), there are likely to be multiple big lawsuits filed over this.  

Whew.  This was a lot.  It's hard to maintain my customary equanimity when the pace of disruptive news is this rapid.  I will hopefully write more about some cool science soon.

The arXiv - where it's been and where it's going

Back in the ancient mists of time, scientists and mathematicians would circulate preprints of their articles among friends and colleagues via the postal service, as a courtesy, to get feedback and to try to make sure people in the community were aware of their forthcoming work.  As the wikipedia entry says, with the advent of widespread LaTeX and the development of the web, Paul Ginsparg (then at LANL) put together an html-based site for electronic sharing of preprints, initially at xxx.lanl.gov (back before "xxx" in URLs was the kind of thing filtered and blocked by employers).   In 2001 he moved to Cornell, and by then the lab was perhaps relieved to see the site, rebranded as the arXiv, shift to Cornell's library/repository infrastructure.   PIs my age remember (fondly? maybe?) the old days of the arXiv, when Prof. Ginsparg's rather dry sense of humor pervaded the site.  The skull-and-crossbones logo.  The "help"/FAQ pages that basically said, "if you can't figure out how to .tar.gz all of your necessary LaTeX files, and you can't figure out how to make your .eps figures small, maybe you should reconsider whether you're smart enough to be sharing your ideas here".  The arXiv was for preprints, without peer review (though interesting follow-on sites like Scirate now exist for organized commenting on the articles).

From these modest beginnings, the arXiv has grown enormously, including imitators/spin-offs such as chemrxiv and biorXiv and socarXiv.  The arXiv has recently become an independent nonprofit, hired a CEO, secured multiyear philanthropic support, and hosts over 3 million articles.  It's been interesting seeing some level of complaints online about some of these steps, but when the audience is so large, not everyone is going to be happy.  Rapid growth has been a major issue - see here for a graph of monthly submissions:

The exponential rise (except for a slight pandemic-correlated shoulder) has been problematic, especially recently.  One hallmark of the arXiv over the years has been its ability to function with comparatively minimal need for "moderation".  Early on, one consequence of the minimalistic help and moderate technical entry barrier was that it was unusual for fringe/pseudoscience to make its way onto the server.  (Hence the establishment of viXra.)  With the ease of cranking out properly formatted readable manuscripts using AI, clearly the arXiv has been struggling.  If 10-15% of submissions need some kind of human intervention or review, the support needs are rapidly outpacing the limited count of support staff.  There can be substantial backlogs.  To help deal with this, the arXiv recently updated its policies regarding AI-generated content (and AI cannot be a co-author, because the AI tools cannot take responsibility for content), and most recently has had to limit submission rates to two papers per month per submitting author.  These moves, too, have drawn some criticism (e.g. here).  Personally, I think the operators of the arXiv face an incredibly challenging environment and are doing the best they can - the idea that they are making moves because they are establishment sticks in the mud who don't understand the New Way of Doing Science is just wrong-headed.

It's completely unclear where all this is heading.  Exponential growth in nature signals instability and does not continue forever. If proponents of very heavily AI-driven research want to establish a repository specifically for that work, that's up to them.  [It is very on brand for the hard core AI advocates to argue that the arXiv is somehow morally obligated to host everything (regardless of hardware or personnel costs) so that future AI tools can read everything (a repository growing too quickly for human researchers to keep up) and summarize it.]

One overarching point that should come up in any arXiv discussion: The arXiv has become a global repository for an enormous amount of human knowledge, without charging anyone publication fees.  This should make interested parties think reallllllly hard about economic models of for-profit publishers.