- Some things have gotten better. We can work from our own laptops, and I think we're finally to the point where everyone at these things is computer literate and can use the online review system. The program officers do a good job making sure that the reviews get in on time (ahead of the meeting).
- Some things remain the same. I'm still mystified at how few people from top-ranked programs (e.g., Harvard, Stanford, MIT, Cornell, Cal Tech, Berkeley) I see at these. Maybe I just don't move in the right circles.
- Best quote of the panel: "When a review of one of my papers or proposals starts with 'Author says' rather than 'The author says', I know that the referee is Russian and I'm in trouble."
- Why does the new NSF headquarters have tighter security screenings that Reagan National Airport?
- The growth of funding costs and eight years of numerically flat budgets has made this process more painful. Sure looks like morale is not great at the agency. Really not clear where this is all going to go over the next few years. There was a lot of gallows humor about having "tax payer advocates" on panels. (Everyone on the panel is a US taxpayer already, though apparently that doesn't count for anything because we are scientists.)
- NSF is still the most community-driven of the research agencies.
- I cannot overstate the importance of younger scientists going to one of these and seeing how the system works, so you learn how proposals are evaluated.
Wednesday, October 25, 2017
Thoughts after a NSF panel
I just returned from a NSF proposal review panel. I had written about NSF panels back in the early days of this blog here, back when I may have been snarkier.
Subscribe to:
Post Comments (Atom)
15 comments:
Caltech, not Cal Tech!
The lack of big names comes about because anyone who applies for a NSF program is conflicted, so if you write proposals every year, you are not going to be asked to serve. I would also guess a lot of folks at Big Name Schools spend time on other panels (MRIs or even higher.)
To be honest, I would also guess a lot of people view serving as a waste of time, which I find ironic when they complain about their proposals being rejected.
Sen. Paul's proposal is a scary one, almost impossible to implement, and codifies the blatant hostility to expertise that has taken over parts of the Republican party.
Brad, the conflicted argument doesn't really hold water for those who aren't up for renewal, or for senior folks who are asked to review CAREER awards. I think there are a couple of issues. First, as you say, they probably prefer to spend their reviewing time on centers rather than single-investigator grants. Second, I strongly suspect that some fraction of big shots have decided that NSF is just too much small ball, and they prefer to spend their time on agencies and foundations that can support much larger amounts. Still, the contrast is interesting. At the DOE PI meeting I attended, for example, there was much better representation from big name places.
Wait until Sen. Paul and others realize that the new NSF headquarters has a gift shop. (Though they might like that - theoretically it could run as a cost center.) Finally you can get that NSF kitchen apron you've been wanting.
Doug, thank you for sharing your experience. All of what you say rings true to me, especially that it's really important for any applicant to experience how panels work including how tough the decisions can be. With regard to folks from the most prominent universities not participating in NSF panels, I agree that they are better represented at DOE events, but PI meetings are probably not the right comparison -- DOE Basic Research Needs workshops which are designed to draft guidance for future funding directions, or site visits to evaluate big programs, are somewhat more analogous. My personal experience is that I'm rarely asked to serve on NSF panels. For four years in a row recently I submitted regular single-investigator proposals (only because they were narrowly rejected three times), but as you point out that would not block me from other types of panels. There was a year when I was invited and responded after around a week, at which point the relevant panels were staffed.
Doug, I'll offer another perspective on serving on NSF panels. Coming from a regional state university that actively, or at least tries/used to, promote research but with increased teaching loads and less facilities as compared to a larger flagship or RI, I think serving on a panel is a priceless opportunity. There are generally less faculty at institutions such as mine and often you are not going to find someone else in your field or close to it that can mentor you with your grant writing. If you don't have such, trying to learn what goes in a competitive proposal is tough until you go serve on a panel. I also find that serving on a panel helps me to still feel as if I'm active in the research community.
In regard to the new NSF facility, how is it in terms of location? I remember that the program officer for my panel thought that it was bit far from the normal DC area locations. Are there nearby hotels and restaurants or do you have to stay somewhere else and take the Metro in each day? Jason
Hi David, thanks for your comment. I had a similar experience in the sense of repeated submissions to the regular program, but during that time I was asked for a couple of other programs. I'm not trying to call out anyone, and maybe my perceptions are off, but my data set is pretty large at this point, and it's a noticeable effect - assuming it's real, I wonder what the reasons are behind it.
Jason, I agree. I always try to encourage young faculty especially to see how these things really work. Regarding NSF, the location is fine. The metro station across the street seems quite new, so there is still a lot of building going on around the area. There were several nearby hotels (none particularly cheap, but that was true in Ballston, too), and an ok selection of restaurants. Given the NSF security, it's simpler to eat at their cafeteria/food court for lunch, which is fine if not exciting.
Hey Doug, I'm headed to an NSF panel on Monday, so your comments are very timely. We'll see if the restaurant scene stands up to scrutiny.
My observation about the distribution in universities matches Doug's. Personally, I am asked to 2-3 times PER YEAR to serve on panels. This may be because I'm reasonably close by. Its impossible for me to serve at that rate and so I decline some, which the PMs seem to understand.
DanM, I liked the Thai place near the Marriott Residence Inn. Peter, I think there is also a feedback loop at work - if you acquire the reputation for being a reasonable person who does these things competently, you get asked to do these things.
I think if the junior scientists showed up to the panels, they would be terrified and would start to consider quitting research seriously. Overall success rate at NSF is hovering around %6 and the average grant is 100K per year. The latter also partly explains why not a whole lot of the bigshots are on the radar. Is it really worth it at the end of the day?
I think the over-representation of ivy leagues isn't a good thing Doug. I'm not sure that it is a prerequisite for having higher quality science or proposals accepted for research. Also, if it were true that each panel always had ivy league representation on it, think of the exclusion of the rest of academia that would cause, just on a statistical basis. Last, being from Rice, you are the ivy league in the room. They don't need another when you're there ;).
But seriously, the panels should be a much broader, more diverse selection on people representing more of a cross-section of academic society that is in the US today, even small businesses, depending on the panel type (grants, SBIR, etc.).
--HD
Anon, thanks for the kind words :-) I'm not advocating for overrepresentation of top programs at these things. It's just that they seem significantly underrepresented, given the fraction of funded proposals that they make up. Broad representation is a good thing in general, as it prevents things from getting stale or too faddish.
The mediterranean place across the street from the AMC movie theater is not bad. Pretty decent beer list.
p.s. ASTROS
DanM - we actually had dinner together as a panel at that place. It was good, and I had some local beer I'd never seen before. So, how was the panel? Your impressions vs. mine? (PS - Astros!)
http://www.independent.co.uk/news/world/americas/nobel-prize-medicine-winner-2017-jeffrey-hall-left-science-lack-of-funding-biological-clocks-a7986441.html
Jeffrey Hall left science 10 years ago but has now won the prize for his work on biological body clocks. His interview in 2008 below. When Jeff was informed about the prize he was 10 yrs retired. He has introduced the term AI ( Actual investigator or s)
http://www.cell.com/current-biology/fulltext/S0960-9822(07)02369-X
Lines from the above web site of interest to all science academics.
You sound especially grumpy about scientific luminaries: why?
I can't help feel that some of these ‘stars’ have not really earned their status. I wonder whether certain such anointees are ‘famous because they're famous.’ So what? Here's what: they receive massive amounts of support for their research, absorbing funds that might be better used by others. As an example, one would-be star boasted to me that he'd never send a paper from his lab to anywhere but Nature, Cell, or Science. These submissions always get a foot in the door, at least. And they are nearly always published in one of those magazines — where, when you see something you know about, you realize that it's not always so great.
Celebrity ‘PI's,’ who are no longer Professors, have too much in the way of lavished resources — by which I mean too much money to do good work! They can and do hire very large numbers of workers, but it is at-best difficult closely to interact with and properly to supervise these bloated numbers of personnel. Such Actual Investigators (AIs) cannot easily gain their boss's attention; and the latter is unable to provide the required close, ongoing scrutiny of their research. There is huge pressure on the overworked, anxious AI to bring something ‘great’ to the boss, who wants everything to go to a vanity journal. One outcome of these antics is that some bizarre stuff is salted throughout this overly conspicuous subset of the literature.
Post a Comment