Monday, August 19, 2013

How to: Write a response to referees

I think I'm going to start a periodic series of "how to" posts.  First up, how to write a decent "response to referees" document.  While this is pretty much common sense, it's not bad to think about it a bit in the abstract, rather than in the heat of the moment of having just received some kind of (perceived) searing blast of criticism.  In brief, assuming you get some collection of referee reports, at least one or two of which are not particularly positive, and you intend to revise and resubmit:
  • Read the reports, and then put them aside for a day, as your white-hot rage over the terrible injustice that has befallen you fades, and in the cold light of reflection you realize that perhaps the manuscript you'd sent in is not, in fact, the greatest non-fiction prose writing since Churchill's six volume history of the Second World War.
  • Now that you're in a less annoyed frame of mind, read through the reviews again, carefully, trying to understand (a) what the reviewers are actually saying, and (b) what the reviewers want you to do (assuming that's not "dry up and blow away").  Often the answers to (a) will reveal either that the reviewers did not properly understand the main point or some subsidiary point of the paper.  Much as we like to grumble about referees, you may have to admit that the fault could lie in your presentation.  Were your figures unclear?  Did the abstract and the intro make your main point explicit, or did you bury the lede somewhere down in the conclusions?  Remember, scientific papers are not mystery stories.  Springing the cool observation on the reader after a lot of setup risks the reader not realizing that the observation is cool.  Moreover, often the answers to (a) will reveal that the reviewer has thought of a possible concern or objection that you either didn't consider, or you did consider but dismissed without pointing it out and explaining your reasoning.  An extremely important part of the response process is figuring out what the main point of the referee is, and realizing that frequently it's worthy of consideration.
  • Regarding (b) above, write down and make a list of what you think the referees want you to do, or what you think it would take to address the points that they raise.    Then consider whether you want to or should do all of those things.  Sometimes the referees can be very demanding.  (We've all seen this.)  You have to use your judgment, and remember that referees are not generally gratuitously mean.  I'd say the default position should be to do what they want, unless what they want is really considered unreasonable by you and your coauthors.  This list, by the way, is a headstart on the eventual "list of changes" that you'll need to provide when you resubmit.
  • When you sit down to write your response, have the referee remarks right there.  In fact, it's a good idea to use copy/paste to intersperse your point-by-point responses.  That way you can be sure you didn't miss anything, and you are forced to write your response in an order that will seem logical to the referee.  
  • Always (always) thank the referees for your time.  Seriously.  You know what refereeing is like, and you'd like to be thanked, admit it.  
  • Point out that after this process you believe the paper is much improved (it will be, too, assuming the referees were really on point and not just asking you to cite their seminal work on the topic at hand), and if possible explain why.  (e.g., we believe that our main point is now much clearer)
  • Always be polite and professional.  If you fly off the handle in your response, even if the referee is overtly hostile, it won't do you any favors with other referees or the editor.  Similarly, just as tone is difficult to convey in email, I suggest avoiding attempted jokes or sarcasm.  This is a professional communication - keep it that way.
  • Try to be timely about revisions.  It's much better to get revisions done while everything is fresh in your mind, rather than letting things linger.  (Don't write them in the heat of the moment, though.)
In your accompanying cover letter when you resubmit, make sure that you emphasize the changes you made in response to the referees.  Also, it doesn't hurt to point out to the editor if you think a referee either missed the mark or seems not to be objective, but it would be best to do so in a very professional way.  Calling the referee an idiot won't win you any friends, particularly since the editor likely chose the referee.  Still, if you really think the referee made serious mistakes, or was not competent, or didn't read the paper, you should bring that to the editor's attention in a professional way.  Again, this kind of response is an important part of your repertoire of professional communications, so it's best to get in the habit of writing them well.

That's it for now.  I'm sure I've left out points - please feel free to bring them up in the comments.


Anonymous said...

What about the reviewer question that is not a question? I had one recently where, interspersed among comments that were generally on-point, we had one that was basically a one-line sentence that stated something but didn't actually ask anything or give us much of a hint of what the reviewer thought was wrong or wanted addressed. Is there any way at all to ask for clarification on such a thing that doesn't make you look like an idiot for not understanding what the reviewer wanted?

Angry said...

What you say does seem like common sense, but I''m surprised at how obnoxious responses to the referee can be...almost as if authors forget that they're *not* anonymous!

I often hear people condemn referees for somehow "going too far." But it never makes much sense to me...I don't argue much, not wanting to give away hints about what I may have refereed. A referee's job is to give an opinion on readability, clarity, accuracy, etc, but somehow others think the referee should just correct their grammar and make sure that proper attribution is given.

Douglas Natelson said...

Anon., yes, sometimes you can get a real head-scratcher. My advice is just be honest. If the reviewer makes an ambiguous statement, just politely ask for clarification. If the reviewer is merely pontificating, you can either agree, disagree, or just let it go.

Angry, I agree with you completely.

Gautam Menon said...

Unrelated to this specific point, but I would be very interested in a "How to" post on how theorists and experimentalists should interact en route to and during a collaboration. How does one discuss things such as author order, role of students, relative weightage of experiment and theory on the paper and so on.

Akshay said...

Thanks Douglas, that was really helpful. I had a recent experience where I waited a month before responding to the reviewer's comments, only to realise that I'd forgotten most of the responses I had thought of initially. So the point about not lingering too long before responding is very relevant.

Could you give a brief example of professionally pointing out a lack of objectivity in the reviewer's comments? I'm personally having trouble framing it appropriately.

sylow said...

I always amend the manuscript somewhere for every remark made by the referees. That makes the folks feel good. That was an advice from my Phd advisor years ago and I follow it. Another point is that I never submit a separate cover letter. My response to the referees follows my cover letter. It is one single pdf file and that is what most journals ask for as far as I know. On the other hand, I appreciate when the editor sends me back the response letter when I am the referee. I would like to make the decision. The editor accepted the paper without even sending me the response letter a couple of times even though my report was quite detailed and required a lengthy and nontrivial response. What surprised me was that once it was a fairly high profile journal (i.e. IF>10).

Anonymous said...

What if the refrees are forcing the authors to cite particular papers that possibly affiated with thm in a directl/indirect way. It is so annoying, is not it??

Anonymous said...

What do you do when an editor appointed three referees, all of whom lacked a basic understanding of the physics involved in the paper? The referees' ignorance is demonstrable, but I'm afraid the editor might not believe that so many poor referees were chosen for the same paper.