Discussion meeting on Machine Learning

In this introductory talk, I will cover the basic principles that govern machine learning (ML) methods. I will discuss many of the common methods within ML, their successfully applications and limitations, along with visualisations of the methods in action. I will also give particular focus to deep-learning methods, including the most widely used types of deep neural networks. I will also cover what constitutes machine learning good-practice, and how common pitfalls can be avoided, with illustrative examples. This will lead to the interactive session later in the meeting, where implementations of the methods can be deployed using python to solve problems relating to simple one-dimensional model systems.

Deep neural networks are emerging as a powerful tool in materials science and quantum chemistry, combining the accuracy of electronic structure methods with computational efficiency. Going beyond the simple prediction of chemical properties, neural network potentials can be applied to perform fast molecular dynamics simulations including solvent effects, model response properties and generate novel structures with desired properties. On several examples, I will demonstrate how this opens a clear path towards increased synergy of machine learning and quantum chemistry as well as designing workflows tightly integrated with experiment.

n recent years machine learning methods have greatly accelerated the theoretical discovery of new stable materials. Models for the prediction of material properties have evolved from simple elemental descriptors and decision tree models, over convolutional networks to graph neural networks. Graph neural networks for crystal structures typically use the atomic positions and the atomic species as input. Unfortunately, this information is not available when predicting new materials, for which the precise geometrical information is unknown. Crystal-graph attention networks replace these precise bond distances with embeddings of graph distances. This allows for the application in high-throughput studies based on both compositions and crystal structure prototypes without using relaxed structures as input. With these techniques we have discovered more than 5000 materials that are thermodynamically stable. The predictions have been confirmed by density-functional theory calculations.

Spectroscopy is central to the natural sciences and engineering as one of the primary methods to investigate the real world. We show that deep learning methods can learn to predict spectra (particularly photoemission spectra) directly from the structures of molecules. We show an application of this model in searching for interesting molecules in a different dataset and spend some time talking about the drawbacks of this method and future work.

In this talk I will present recent work on machine learned energy functionals, which can be used to predict energies, forces and molecular properties on an equal footing. On one hand, this concept is realized in a coarse grained scheme yielding an accurate and effcient charge equilibration method. On the other hand, ML-based density functional approximations working with the full electron density will be presented.

The density-functional toolkit (DFTK) is an open-source Julia code for multidisciplinary research on density-functional theory methods. Over the past months one of our ongoing efforts has been to make the code algorithmically differentiable, that is enable the ability to compute derivatives of arbitrary output quantities with respect to arbitrary input quantities. Since this does in particular also enable the computation of "unusual" derivatives, such as the sensitivity of a band gap with respect to the parameters of the XC functional, this opens (amongst others) the prospect to design novel DFT models in a data-driven fashion. Following the emerging opportunities of scientific machine learning one could for example imagine integrating a neural networks as a component of the exchange-correlation functional into the DFT model itself. In this talk I will sketch the current
status of algorithmic differentiability in DFTK and hint at possibilities such as the aforementioned ones enabled by an automatically differentiable DFT code.

In this talk I will present recent work on machine learned energy functionals, which can be used to predict energies, forces and molecular properties on an equal footing. On one hand, this concept is realized in a coarse grained scheme yielding an accurate and effcient charge equilibration method. On the other hand, ML-based density functional approximations working with the full electron density will be presented.

I will discuss machine learning in electronic structure theory. I will review recent efforts to use ML to find density functionals for ground-state density functional theory (DFT), both for weakly and strongly correlated systems. I will end with ideas about machine learning in time-dependent (TDDFT).

One of the biggest challenges in condensed matter physics is to calculate properties of materials taking into account the quantum many-body nature of matter. While the Coulomb interaction is universal, its effects cannot be separated from the specific material under analysis, which leads to a huge theoretical and computational effort. Recently, Machine Learning (ML) has raised new hopes as a tool that could be used to screen and predict properties of broad classes of materials. Indeed, the statistical structure of ML tools is particularly well suited to deal with quantum mechanics, thanks to the statistical information encoded in the state vector, and in recent years ML has been used for addressing various classes of problems. In quantum physics and chemistry, ML has been mainly used to predict energies and forces starting from the atomic composition. Another research line lies in the domain of Density Functional Theory (DFT): here the system is described by its electronic density, in other words, its properties are functionals of the density. These functionals are in general not known and must be approximated. So far, ML has been mainly used with the aim to determine energies as density functionals. The work described in this talk was aimed at designing a density functional. However, the underlying strategy chosen differs from the usual approach in two ways: first, instead of being observable specific, we concentrated on an important building block, the one body reduced density matrix (1-RDM). The knowledge of this quantity allows one to directly access much useful information, such as the kinetic and exchange energies. We hence investigated the functional connecting the 1-RDM to the density. Second, instead of using ML merely as a clever interpolation tool, we asked a methodological question, namely "Is it possible for the model maker to learn with the machine?” This would allow one to build functionals that could then be used without creating huge data-sets beforehand. One fact is essential for modeling the 1-RDM, i.e. its diagonal is the density. Consequently, the wanted functional must be derived by exploiting the functional constraints connecting the diagonal to the off-diagonals. The fact that the object on which to build the functional is contained in the matrix to be determined assures the existence of strong correlations in between the values of the entries. This is analogous to the existing relations among the pixels of an image. Therefore, the success of ML techniques in extracting spacial information in image processing tasks was at the basis of the work. First we tried to learn from the way the machine structures the data in order to create new analytical models. Then we moved to considering how the human can inform the machine by embedding in it some pre-existing theoretical knowledge. For the discussion we used a simple two sites system, whose 1-RDM can be found exactly in some limiting cases. Many other results were derived for characterizing this system, including the functional in non-limiting cases and the observation of its applicability as an auxiliary system for the determination of the desired functional in a system with a higher number of grid-points.

This session is devoted to live hands-on. You will find in this repository https://github.com/JackWetherell/deep-learning-hands-on-rest the python code, as well as the instructions on how to use it.

Short instructions:

• git clone https://github.com/JackWetherell/deep-learning-hands-on-rest.git
• cd deep-learning-hands-on-rest
• pip install -r requirements.txt
• jupyter notebook
• This should open a browser page. If it doesn't, search for a line like this to copy/paste in your browser:
  http://localhost:8888/home/sottile/jupyter/?token=2ad6e037cf103ca7719f315ed7eabaebc0d827776bbddc34