Monday, October 04, 2010

"Definitively inaccurate": One more comment about NRC rankings

One last post before the Nobel in physics is announced tomorrow.... As many people in the academic blogosphere have reported, there are some serious issues with the NRC rankings of graduate programs.  Some of these seem to be related to data entry, and others to nonuniform or overly simplistic interpretations of answers to survey questions.  Let me give a couple of examples.  I'm in the physics and astronomy department at Rice, and for several years I've helped oversee the interdisciplinary applied physics graduate program here (not a department - applied physics does not have faculty billets or its own courses, for example).  I filled out faculty NRC paperwork, and I was also in charge (with a colleague) of filling out the "department"-level NRC paperwork for the applied physics program.  I know, with certainty, that some of the stats for the two programs are very very similar, including the allocation of work space to graduate students and the approximate completion rates of the PhD program.  However, while these seem to show up correctly in the applied physics NRC data, they are both skewed bizarrely wrong (and very unfavorably, like the completion rate in the NRC data is too low when compared with reality by at least a factor of two!) in the physics & astronomy departmental NRC data.  Now, overall the department did reasonably well in the rankings, and if one looks particularly at just the research stuff per faculty member, physics and astronomy did quite well.  However, this issue with student data really stinks, because that's what some sites geared toward prospective students emphasize.  It's wrong, there's no fixing it, and it looks like it will be "definitively inaccurate" (to borrow a phrase from Douglas Adams) for at least a decade. 

1 comment:

Brad Holden said...

Our department has, apparently, no graduates that have ever gone on to an academic position. Technically it is N/A, because the data are missing, but it seems to be listed as 0.

This bumps the department down in the automated tools to generate rankings and, as you say, there is nothing to be done about this for the next decade+.