Reviewing- why, how, and how often?

I review lots of papers and proposals for various journals and funding agencies. While time allocation is a continual challenge, and while there is no good framework for rewarding this kind of professional service, I think it's important to do my fair share for several reasons. First, I'd like someone else out there to do me the same courtesy - well written, thorough, timely referee reports almost always improve the quality of scientific papers. Sometimes it's just a matter of the referee having fresh eyes and a different perspective; a referee can point out that something which seems obvious to you may not be clear to others. Second, reviewing is a way to keep abreast of what's going on out there in the community. Third, reading articles and proposals and having to write reviews is intellectually stimulating - it gets me to think about new things and areas that aren't necessarily my primary interest.

When writing a report, I try to produce something that's actually useful to the authors (as well as the editors in the case of journal articles). I briefly summarize the main points of the paper or proposal to indicate that I've actually read it and understand the key ideas. For a paper, I emphasize my overall opinion of the work. Then I point out anything that I found unclear or any parts of the argument that don't seem supported by the data or calculations, with an eye toward what would improve the manuscript. I rarely reject papers out of hand, since I rarely get manuscripts to review that I think are hopeless (though some are submitted to inappropriate journals). On the other hand, it's pretty rare that I think something is absolutely flawless (though if my comments are minor I don't ask to see the paper again). I truly don't understand why some people submit two-sentence referee reports that are dismissive - this doesn't help anyone. I also don't understand why a small number of people can be venomous in reviews. Ok, so you didn't like the work for some reason - why get nasty? Just explain rationally why you don't think the paper is right. The point of refereeing is not to fire off insults under the shelter of anonymity - that's what blogs and internet forums are for.

Proposals can be more work. The big questions are usually (1) Does the PI clearly articulate the science or engineering question that is under investigation? (2) Is the plan well considered and likely to lead to good science? (3) How much of this is new and how much is completely incremental? (4) Did the PI(s) include everything that they were required to (e.g., description of prior work, for NSF discussion of outreach and education)? Grade inflation in proposal refereeing makes this process more painful as well. I am well aware that labeling a proposal as merely "good" is the kiss of death.

I have a tough time saying "no" to refereeing requests, and I need to get better at it. Prompt refereeing is important, and it's better to decline to review than to sit on a manuscript for two months. Still, good refereeing is definitely needed. I'm sure many readers have had the experience of a manuscript sitting with editors for a long time because it's tough to find qualified reviewers with the right expertise who also have enough time to review promptly. It is a shame that there isn't some intelligent mechanism for rewarding refereeing. It shows up as a line or so on your CV, even though it's arguably more valuable than serving on some university committees. Ahh well. Off to write some reviews.