Discovering Physical Concepts with Neural Networks

Despite the success of neural networks at solving concrete physics problems, their use as a general-purpose tool for scientific discovery is still in its infancy. Here, we approach this problem by modeling a neural network architecture after the human physical reasoning process, which has similarities to representation learning. This allows us to make progress towards the long-term goal of machine-assisted scientific discovery from experimental data without making prior assumptions about the system. We apply this method to toy examples and show that the network finds the physically relevant parameters, exploits conservation laws to make predictions, and can help to gain conceptual insights, e.g., Copernicus’ conclusion that the solar system is heliocentric.

Figure 1

Learning physical representations. (a) Human learning. A physicist compresses experimental observations into a simple representation (encoding). When later asked any question about the physical setting, the physicist should be able to produce a correct answer using only the representation and not the original data. We call the process of producing the answer from the representation “decoding.” For example, the observations may be the first few seconds of the trajectory of a particle moving with constant speed; the representation could be the parameters “speed $v$” and “initial position $x_0$” and the question could be “where will the particle be at a later time $t^{\prime }$?” (b) Neural network structure for SciNet. Observations are encoded as real parameters fed to an encoder (a feed-forward neural network, see SM [3]), which compresses the data into a representation (latent representation). The question is also encoded in a number of real parameters, which, together with the representation, are fed to the decoder network to produce an answer. (The number of neurons depicted is not representative.)

Figure 2

Quantum tomography. SciNet is given tomographic data for one or two qubits, as shown in part (a) and (b) of the figure, respectively, and an operational description of a measurement as a question input and has to predict the probabilities of outcomes for this measurement. The plots show the root mean square error of SciNet’s measurement predictions for test data as a function of the number of latent neurons. In the tomographically complete case, SciNet recovers the number of (real) d.o.f. required to describe a one and a two qubit state (which are two and six, respectively). Tomographically incomplete data can be recognized, since the prediction error remains high as one increases the number of latent neurons.

Figure 3

Heliocentric model of the solar system. SciNet is given the angles of the Sun and Mars as seen from Earth at an initial time $t_0$ and has to predict these angles for later times. (a) Recurrent version of SciNet for time-dependent variables. Observations are encoded into a simple representation $r(t_0)$ at time $t_0$. Then, the representation is evolved in time to $r(t_1)$ and a decoder is used to predict $a(t_1)$, and so on. In each (equally spaced) time step, the same time evolution network and decoder network are applied. (b) Physical setting. The heliocentric angles ${\varphi }_E$ and ${\varphi }_M$ of the Earth and Mars are observed from the Sun; the angles ${\theta }_S$ and ${\theta }_M$ of the Sun and Mars are observed from Earth. All angles are measured relative to the fixed star background. (c) Representation learned by SciNet. The activations $r_{1,2}(t_0)$ of the two latent neurons at time $t_0$ [see Fig. 3] are plotted as a function of the heliocentric angles ${\varphi }_E$ and ${\varphi }_M$. The plots show that the network stores and evolves parameters that are linear combinations of the heliocentric angles.