DALL·E 2, Imagen, and Applications to Chemistry

In the past two months, DALL·E 2 has taken over the internet. From Bart Simpson edited into Egyptian art to Donald Trump as the Lorax, text-to-image AI produces amazing results. Caption: “Panda weaving a basket made of cyclohexane”, DALL·E 2 Are these an impressive-but-gimmicky party trick? Or can these innovations be harnessed for applications in scientific domains? Many […]

Transformer Retrosynthesis

In drug discovery, there are two main approaches to hit finding: 1) virtual screening of existing small molecule libraries and 2) generative design of new molecules. Generative molecule design can result in better binders, but it may be unknown how to synthesize them. The task of retrosynthesis – designing a synthesis pathway for a molecule […]

A new state-of-the-art model for molecular conformer generation

In structure-based drug discovery, most methods rely on two key elements of accuracy: accurate protein structure modeling and accurate drug structure modeling. AlphaFold is able to predict protein structures with unprecedented accuracy. But drug structure modeling lags behind, with current models for conformer generation only providing 67% accuracy on a common molecular conformer benchmark. GeoDiff predicts drug conformations with […]

Design of protein binding proteins from target structure alone

How do you design a protein that binds to another protein given only the target protein structure? Until recently, you could use Rosetta to manually craft a protein using expert heuristics. However, this process is laborious, expensive, and does not generalize. Researchers at the Institute for Protein Design recently published a groundbreaking work outlining a systematic process […]

The Protein Folding Problem – is it Solved?

In CASP14, DeepMind presented the results of AlphaFold, a deep neural network designed for protein structure prediction. During the experiment, AlphaFold predicted structures with an average deviation of  ~1 Å from the C-alpha atoms of experimentally solved structures. Now, ~1 Å is often also used as the resolution to denote high accuracy experimental structures (they […]

Predicting Transition State Structures with Tensor Field Networks and Transfer Learning

The year 2021 was a Pandora’s Box for machine learning in chemistry. DeepMind put the chemistry world on notice when it published its approach to the protein folding problem [1]. I expect that we will continue to see machine learning approaches quickly dominate the well-defined, data-rich problems in chemistry. However, there are other challenges that are harder to […]

New work from the creators of AlphaFold pushes the frontiers of Density Functional Theory

Recent advances in computational chemistry have modeled non-covalent chemical interactions like protein folding with increasing accuracy [1]. But for chemical reactions that involve bond-breaking and bond-forming, modeling is still inaccurate and computationally expensive. For chemical bonds, density-functional theory (DFT) is a field of study seeking to find a “functional” that accurately maps the electron density […]

Recent advances in computational chemistry have modeled non-covalent chemical interactions like protein folding with increasing accuracy [1]. But for chemical reactions that involve bond-breaking and bond-forming, modeling is still inaccurate and computationally expensive. For chemical bonds, density-functional theory (DFT) is a field of study seeking to find a “functional” that accurately maps the electron density of an atomistic system to its energy.

In this blog, we explore a mathematically rigorous foundation that proves the existence of an exact functional, and for decades, researchers have handcrafted functionals to approximate the true functional. If we can find a better functional, we can accurately model chemical reactions at a subatomic scale, increasing our understanding of chemical interactions and opening new avenues for chemical synthesis.

deepmind alphafold

DeepMind recently published a paper that introduces their own machine learning functional, which they call DM21 [1]. Their functional achieved state-of-the-art performance on several different benchmarking tasks. It was designed to satisfy some known constraints of quantum mechanics. Crucially, it allows future researchers to incorporate new constraints, allowing the model to increase in accuracy as scientific knowledge advances. Electrons have fixed properties of charge and spin (“up” or “down”). A system can’t have an overall fractional charge or fractional spin, but a region of a system can. Although many constraints such as these are known, it is difficult to manually design a functional around them. This problem is well suited for deep learning, where the constraints can be expressed as data and a functional can be trained to reproduce the energies of a system while obeying the constraints. DeepMind designed their functional with these constraints in their mind and attribute much of their success to ensuring that the constraints are satisfied.

DeepMind’s functional is shown below in Fig 1. Feature vectors of the atomistic system are calculated at certain points and are fed into a simple neural network. The outputs of the neural network at the different points are added together to get a most difficult contribution to the energy. The rest of the energy can then be calculated using mathematically exact methods to get the total energy. The algorithm was trained on a dataset of highly accurate electron densities and energies. After training, the functional can be used to predict the energy of molecules. DeepMind has made the code for their functional available for free [2].

deepmind dft
Fig 1: Deep Mind’s Functional

I expect this functional to have a significant impact in the field of computational chemistry. The combination of state-of-the-art performance and easy usability make this functional an excellent tool for computational chemistry. Highly accurate energies are important for molecular dynamics simulations, allowing more accurate investigation of chemical reactions. This network will also likely inspire further investigation of machine learning functionals and approaches to solving the DFT problem. It’s a big step toward accurate computational simulations.

1. Highly accurate protein structure prediction with AlphaFold, John Jumper et. al, https://www.nature.com/articles/s41586-021-03819-2
2. Pushing the Frontiers of Density Functionals by Solving the Fractional Electron Problem, James Kirkpatrick, Brendan McMorrow, David H. P. Turban, Alexander L. Gaunt, James S. Spencer, Alexander G. D. G. Matthews, Annette Obika, Louis Thiry, Meire Fortunato, David Pfau, Lara Román Castellanos, Stig Petersen, Alexander W. R. Nelson, Pushmeet Kohli, Paula Mori-Sánchez, Demis Hassabis, Aron J. Cohen, Science, DOI: https://doi.org/10.1126/science.abj6511
3. Deepmind-Research GitHub page, https://github.com/deepmind/deepmind-research/tree/master/density_functional_approximation_dm21