Well, since several folks are commenting on the h parameter, I might as well put in my two cents. The h number is defined here. In brief, if you've published h papers (and no more) that each have h or more citations, then your h number is, well, h. In principle, your h number is not supposed to count self-citations (though once h is above 10 or so, that becomes pretty irrelevant anyway). In some fields (e.g. CS) where people tend to publish on public electronic archives rather than in journals, citations of those preprints are counted. The idea is that the H number is a metric of scientific performance and impact, and is more robust than mere citation counting. Steady output that people actually cite is rewarded more than being one co-author on a Science paper that happened to get 750 citations. There are variations, too. You can calculate the h number divided by a person's years of "professional experience", or actually figure out dh/dt. For a fair comparison between people, one should normalize h numbers by subfield. In condensed matter physics, a typical person near tenure time has an h of around 10. In mid-career, an h of around 20-30 is about the average, and exceptional people like National Academy members tend to have h values higher than 50. The h number can be skewed in certain cases. Some people publish little, but their work can have enormous impact. Others, such as materials growers, can have enormous h numbers because they supply materials used by dozens of experimental groups.
Obviously trying to quantify a person's scientific impact and productivity in one number is a crude and rough thing to do, just as the subject GREs and qualifying exams are often crude indicators of actual aptitude. Just as I think the physics GRE is only really good at identifying outliers (the best 2.5% do very well on it; the worst 2.5% do very poorly; the middle 95% get scores that don't seem to correlate with their actual talent or ability), the h number is similar. I would never dream of assigning too much importance to it in tenure cases. As in grad school or postdoc or faculty applications, detailed letters of recommendation are far more useful, and in my experience correlate much better with actual performance. However, if someone has an h number far outside the expected norm in either direction, I'd like to know that. For example, I heard recently of an externally appointed dean at a research university where the faculty were rather shocked to find that the dean's h number was about 4. Unsurprisingly, people who have had vastly larger scientific impacts don't really like being told what to do or have their decisions scrutinized by someone who has essentially been a professional administrator.
Anyway, I wouldn't lose too much sleep over h numbers. They just get a lot of attention because they're a relatively new idea, and they do seem superior to the previous crude metric, citation counting.
15 comments:
I think this whole h-number business (or citation counting, in that matter) is a direct consequence of an extremely tight job market for faculty positions. In an environment where people typically receive hundreds of applications for an assitant professor position in research universities, there is a need to quantify applicants numerically this way or that way. Someone who has extremely high h-number or citations may be doing garbage after all.
Ed witten, undoubtedly the smartest living person in the world, has an h-number of 110. He has 60000+ citations and al lthe prizes you can think of(except nobel) and he is sitting in Einstein's desk in IAS at Princeton. However, his papers have no practical significance for any human being except for a bunch of ortodox string theorists(which is pseudo-science). If they cannot discover supersymmetry next year in LHC, where will all these citations and h-numbers go? Is this the direction where we want to go in the future as scientists? The easiest way to reduce this pile of applicants in USA is to put an upper age limit for senior faculty like europe. Official retirement age should be 65 or so. There are professors stil lholding their positions at the age of 90 in this country. That is ridiculuous. Once those issues are addressed, this h-number nonsense will disappear automatically... I think that is the take home lesson from this.
anonymous,
h-like indicators have drawbacks, like pushing people to all work on the same mainsteram subject.
But in various places (maybe not in the US) candidates are too often chosen thanks to personal reasons that have little to do with scientific merit. In these cases, having a semi-quantitative measure of scientific merit is helpful.
I agree with the second respond above. And also with original post.
Most people tend to complain about h-index, but it's way better than total number of citations or total number of publications.
At the same time, a lot of decisions are made very subjectively. I don't really understand why letters of recommendations are considered important. I think they say more about the person who wrote them than the person described in the letter. Many professors simply don't care what happens to their postdocs, in fact there's a clear conflict of interest where a talented postdoc becomes a competitor. In government labs the advisor may want to keep postdoc as a staff member and thus he may be conflicted about writing a good letter for a super-talented postdoc as well.
Finally, some people are too busy to write a detailed letter, and most letters say about the same thing (how many people can be in top x% percentile, really?).
Subjectiveness of faculty appointments has been documented in details in several research papers - issues such as gender, or personal connection to search committee seem to play a key role. In proving the connections the scientists relied on factors similar to h-factor - such as various measures of publication record. Why wouldn't search committees take into account these type of measures, at least in early stages of candidate evaluation, rather than go for absolutely subjective or un-normalized measures, like letters of recommendations or self-confidence during presentations.
Incoherent ponderer, you are wrong about the personal decision. Personal decision plays a role only if you can survive the initial elimination process where the search committee goes down from hundreds of applicants to maybe half a dozen in merely a few hours. That initial elimination is purely numerical and if you happen to be in the "short list" of half dozen applicants, then they will start considering "personal" things like recommendations etc. Do you think search committee is reading 1000 recommendation letters if they received 300+ applicants? Come on... What I was trying to say in my first comment is that h-number of a certain person may not tell you anything about him as a "scientist". These bigshot string theorists like Witten, Polchinski and Vafa all have h-numbers around 100 but they are not "scientists". That was my point.
Two points. First, I think it's a mistake, anonymous, to declare that Witten, Polchinski, etc. are not scientists and thus argue that their high h numbers completely invalidate h as a metric. I give string folks grief all the time, but there is no question that those individuals have had an enormous impact on the current research being done in their field. If you think that their field is math rather than science, that's fine, but it's silly to discount Witten's obviously huge influence on a big community (which is really what the h number is supposed to quantify, anyway).
Second, do you really run searches the way you describe at your institution? Someone does some kind of numerical calculation on each folder, ranks them, and takes the top ten or something? I ask only because I've never seen a search done that way. I also think your application numbers are off, and reflect the bad job market of the early '90s more than today. I've been on several searches, and typically we get something like 120 applicants. Of those, there are always 30 or 40 who don't remotely fit the description in the ad (essentially they're spamming all places that are advertising), and they can be eliminated immediately. Then it's usually pretty challenging work to whittle down the rest. I don't know of anyone who actually looks up all the remaining candidates and does some kind of ranking by publications or citations.
anonymous - my argument is that as far as assigning a number to quantify someone's research credential, h-index is as good as any other external measure. And it's relatively easy and quickly to compute.
For younger folks this may not be a good measure anyhow - if someone published some key papers in the last year or two, they might not be reflected in h-index.
So for faculty decisions, a sum of publication's impact factors or a similar measure may be a much better choice. We do this unofficially anyways - someone who was a first author on a Science or Nature publication and a couple of PRL's will get higher on the list of candidates than someone with a lot of Physica B articles.
Finally - I agree that letters may come in only after pre-screening, but it doesn't make it less arbitrary - final selection is the only thing that really counts. Maybe physics departments should go the way of materials science dpts that request a list of recommenders but don't require letters for everyone - they request them themselves once the pool is narrowed down.
Considering that a lot of candidates are forced to apply to 30 or 40 schools, this really cuts down on the amount of work is required of recommenders in assisting their protege.
doug, i wonder if you could make a more detailed post about the inner workings of search committee. I don't mean you should reveal any secrets, but I am always amazed how little is known about what is essentially the most important career-making (or breaking) mechanism in academia. There's a lot of rumors but very little information, which tends to hurt many potential candidates. Rumor mills and internet have helped somewhat, but for example how much weight is attributed to publications vs. teaching or letters or "service"? Do most searches narrowly looking for someone in specific sub-field, or are searches typically more open than specified in the ad for outstanding candidates? How many people have to agree on a choice of candidate? Do the faculty agree on lowest common denominator (a candidate nobody feels strongly against) vs. selecting the one that generates most enthusisasm, even if some search committee members feel strongly agains? etc. etc.
Ponderer, I second your suggestion to Doug. But, having also served on said committees, I would expect that the characteristics of the search can be very different at different universities, according to the particular culture of each department. These things also change from year to year as the makeup of the search committee changes, and as the needs of the department change. I suspect that the range of different ways in which searches are conducted is probably very large. For example, with respect to each of the questions you posed, I bet the answer to each one is: sometimes yes, sometimes no.
It is not unusual for top schools like Harvard and MIT to receive about 300 applicants for a tenure-track assistant professor position in high energy theory(buzzword for string theory nowadays) therefore my estimation is not farfetched. In fact, less than 5% of a;l high-energy theory Ph.D's end up in academic positions these days so it is a fairly tight job market. These 300 candidates are initially judged numerically this way or that way. It can be h-number, citation count or bean counting(i.e. number of publications) Once the committee reduce these to a "short-list" of about a dozen candidates, they start reading recommendation letters etc. Some candidates are married to another scientist and they may require the school to hire their wife as well. It depends on the circumstance. As for condensed matter faculty positions, number of applicants may go down to as low as 100 for second tier schools.
As for string theory, it has 10^500 vacuum states(so called anthropic landscape), therefore you can get any result you want from it. It also would not predict anything experimentally even if we had the technology to build a galaxy size particle accelerator tomorrow thus it is not "science". Lack of falsifiability allowed it to flourish and survive for two decades now. Whether you can call it "math" is a completely different story but I fail to see how you can call a string theorist "donner professor of science" for instance. These people are garnering lots of citations for sure but it does not change the fact that string theory is not even wrong... In fact, it is not even a "theory". It is no more than a speculation.
I'll try to ignore anonymous' thinly veiled implication that my institution is second rate. I also don't want to defend string theory in any serious way, since when I'm grumpy I tend to agree more than disagree with what's been said. (Though to be fair: the best string theorists are fully aware of the fact that physics is an experimental science, and would like nothing better than to come up with testable predictions. Only the worst of the lot are truly happy and comfortable with the current situation.)
I will point out that my institution is nationally in the top 10 in AMO physics, and the numbers of applicants for searches in that area, even theory (which can draw on both AMO and CM theory), are still more in line with what I said above, rather than the 300 number.
Later tonight if I have the time I'll try to whip up a quick primer on searches, though Dan is right: every search is strongly shaped by the department culture and particular circumstances.
the number (100-120) is typical for top ivy league schools in northeast. Most candidates applying for faculty positions in CM exp would probably apply to Rice as well, so I don't think there's a drop-off, and if anything, geography may be more important than ranking in such considerations. On the other hand, Chad commented that they expect as many as 200 applications for AMO position at his school, which is way more than I would have guessed.
This may be different for theorists, I am talking mostly about CM exp.
The number may increase if the position is open to multiple fields - nanoscience, physical chemistry or materials science, for example, which may attract physics Ph.D. as well as bio/chem/matsci people.
Regardless of whether the actual number is 100 or 200 or 300 applicants for a single open position, this strikes me as insane, especially considering that most if not all candidates are basically geniuses who spent the past 10 years working very hard to have a 1 in 100 shot at a job that doesn't even pay all that well (compared to what lawyers or MBA's make after spending a fraction of the PhD's time in terms of training).
As I wrote before, the tightness of the faculty job market comes from two facts, namely
1. There is no official retirement age for senior faculty members in USA unlike europe therefore many people do not retire until they die basically. As a result, the average age of a tenured professor in research universities in USA is about 55 now. Believe or not, that number had been going up for the past two decades.
2. Not coincidentally, the government funding for university research had been flat or falling in the same time period and industrial funding is almost nonexistent outside schools like Berkeley and MIT. In fact, more papers are coming from Europe right now than USA in top-tier journals like Science& Nature these days and unless the funding situation improves soon, USA will be behind pacific rim as well in 10 years.
Not surprisingly, economic realities dictate the situation even in ivory towers.
These H parameters are called Hybrid Parameters and mainly used in transistor. Transistor has a vast panel of experienced electrical engineering tutors who specialize in hybrid parameters and can explain the different concepts to you effectively.
I agree with the second respond above. And also with original post.
Post a Comment