So, what's the problem, beyond the obvious issues of computing efficiency and the fact that we don't know how to write down an exact form for the exchange-correlation part of the functional (basically where all the bodies are buried)?
Well, the noninteracting states that people like to use, the so-called Kohn-Sham orbitals, are seductive. It's easy to think of them as if they are "real", meaning that it's very tempting to start using them to think about excited states and where the electrons "really" live in those states, even though technically there is no a priori reason that they should be valid except as a tool to find the ground state density. This is discussed a bit in the comments here. This isn't a completely crazy idea, in the sense that the Kohn-Sham states usually have the right symmetries and in molecules tend to agree well with chemistry ideas about where reactions tend to occur, etc. However, there are no guarantees.
There are many approaches to do better (e.g., some statements that can be made about the lowest unoccupied orbital that let you determine not just the ground state energy but get a quantitative estimate of the gap to the lowest electronic excited state, and that has enabled very good computations of energy gaps in molecules and solids; time-dependent DFT, which looks at the general time-dependent electron density). However, you have to be very careful. Perhaps commenters will have some insights here.
The bottom line: DFT is intellectually deep, a boon to many practical calculations when implemented correctly, and so good at many things that the temptation is to treat it like a black box (especially as there are more and more simple-to-use commercial implementations) and assume it's good at everything. It remains an impressive achievement with huge scientific impact, and unless there are major advances in other computational approaches, DFT and its relatives are likely the best bet for achieving the long-desired ability to do "materials by design".
Hi Doug,
ReplyDeleteThanks again for this great service to DFT that you are putting out here.
I have thought a lot about what, in principle, is completely determined by the ground state density. I have often wondered about the subtlety of whether or not we can get excited state (non-equilibrium) information? And if so, just which information can we capture? After much contemplation and meditation, I have come to the following conclusion: the complete DFT functional, of the Hohenberg-Kohn variety (i.e., in the absence of a TIME-DEPENDENT POTENTIAL) contains, in principle, exact information about all the eigenstates of a given ‘static’ Hamiltonian. This is because the Hamiltonian is a unique functional of that ground state density, and all the eigenstates (i.e., not just the electronic ground state, but all the excited eigenstates of that static Hamiltonian) are determined by the Hamiltonian. This point took me a while to wrap my head around – but exact, pure DFT does indeed imply that the entire electronic eigenspectrum (not just the ground state) is encoded by the ground state density of any time-independent Hamiltonian / potential. Of course, in practice, we don’t have the exact functional, and we are nowhere near close to getting all of this possible information – DFT of the Kohn Sham variety is mostly restricted to ground state / near ground state properties, which is still quite a lot.
BUT, if you have a time-dependent external potential, then this static average density is not enough – that is when you have to work harder and do things like TDDFT. I guess the heart of the difficulty stems from the same reason time-dependent quantum mechanics is much harder than time-independent quantum mechanics in the first place. With time-independence, we know we can just solve for a stationary eigenspectrum, with a clear-well defined ground state. With a time-dependent Hamiltonian, on the other hand, the problem is much messier, you have to worry about things like unitarity, which in fact, if you follow your nose, takes you all the way down to the foundations of why we need quantum field theory to describe nature as opposed to straight-up quantum mechanics that you learn in your first undergrad courses. There is a generalization of the Hohenberg-Kohn theorem, called Runge-Gross, which states that you can completely determine everything with the time-dependent electronic density – but then the question becomes, WHICH time-dependent electronic density? For static systems, HK says you look specifically at the GROUND STATE electronic density. But what is the analogue of a ‘ground state’ for time-dependent spectrum? That’s where it gets sticky, and in short, leads to one of the key conceptual difference of TDDFT from regular static DFT – the time-dependent electronic density completely encodes the time-dependent Hamiltonian FOR A GIVEN INITIAL CONDITION – you need the initial condition along with the Hamiltonian to determine the unique time-dependent density.
Finally, I would like to comment that in many ways, DFT can be viewed a particular useful application of the ‘Effective Action’ approach to physical theories in general, and electronic structure specifically (http://www.physics.rutgers.edu/~haule/papers/rmp.pdf). Indeed, there is a related ‘Classical DFT’, which applies the same strategy to study the statistical mechanics of complex classical fluids, including some very sophisticated hydrodynamic properties. Thus, I would like to emphasize that DFT is more than just a useful computational tool – to those passionate in the field, it is a whole new conceptual framework of thinking about many-body quantum and statistical mechanics problems in general, not unlike field theory. I would specifically address this to some commenters who have asked, why is Kohn-Sham DFT useful at all when we have limited schemes to approximate the exact XC functional? In my view, DFT’s value is not simply (or even primarily) practical, but also just as much aesthetic/ intellectual – it provides a whole new way of thinking about physics in general.
ReplyDeleteThanks, Tahir! I really appreciate your comments, and now I want to learn about "classical DFT".
ReplyDeleteYour welcome, Doug. The pleasure is mine! I would recommend this useful Annual Review for those interested in Classical DFT, specifically in the context of Soft Matter Physics: http://www.engr.ucr.edu/~jwu/papers/annurev.physchem.58.032806.pdf
ReplyDeleteBest,
Tahir
Hi, You may also like this review by Antoine Georges, which emphasizes the effective field theory viewpoint to both classical and quantum dft/dmft:
ReplyDeletehttp://arxiv.org/abs/cond-mat/0403123
Electronic DFT is a lot more powerful because of the "transferability" of effective Kohn-Sham potentials: Since they represent electron-electron correlation and exchange, the same functional that works for a small molecule would work for much larger ones. This transferability is lost both in TDDFT due to the dependence on initial conditions or even electronic ground state DFT at finite temperatures, and in classical statistical mechanical systems, where the functionals are likely dependent on the physical context.
I just spotted your very nice blog. So I'm chiming in on a topic I know a little about :)
ReplyDeleteRegarding the excited states - there is also ensemble density functional theory which allows access (in principle) to excitation energies in a similar way to conventional groundstate Kohn-Sham theory (one can also use it for open-system and finite-T DFT using the same basic mechanics). This avoids various worries about initial states in time-dependent DFT.
The difficulty is that the space of approximations is much greater than for groundstate DFT, so coming up with decent density functional approximations (e.g. GGAs) is much more challenging. "Work is ongoing"
On the other hand there is current-DFT, which is in some sense better behaved than conventional time-dependent DFT, but at the expense of requiring the vector current density rather than a single density. I've never been entirely sure why it is not more widespread, but suspect that the additional numerical challenges it throws up make it ill-suited to the popular planewave _and_ Gaussian basis sets are to blame. Which is a little sad.