Interpretable conservation law estimation by deriving the symmetries of dynamics from trained deep neural networks

Understanding complex systems with their reduced model is one of the central roles in scientific activities. Although physics has greatly been developed with the physical insights of physicists, it is sometimes challenging to build a reduced model of such complex systems on the basis of insight alone. We propose a framework that can infer hidden conservation laws of a complex system from deep neural networks (DNNs) that have been trained with physical data of the system. The purpose of the proposed framework is not to analyze physical data with deep learning but to extract interpretable physical information from trained DNNs. With Noether's theorem and by an efficient sampling method, the proposed framework infers conservation laws by extracting the symmetries of dynamics from trained DNNs. The proposed framework is developed by deriving the relationship between a manifold structure of a time-series data set and the necessary conditions for Noether's theorem. The feasibility of the proposed framework has been verified in some primitive cases in which the conservation law is well known. We also apply the proposed framework to conservation law estimation for a more practical case, that is, a large-scale collective motion system in the metastable state, and we obtain a result consistent with that of a previous study.

Figure 1

Schematic diagram of the mapping structure of a DNN with $d_{\mathrm{in}}=2$ and $d_{\mathrm{out}}=1$. The DNN is trained to map two-dimensional data distributed on a black curve to one-dimensional output space. The arrows indicate the compression direction of the input space in the mapping from the input to the output.

Figure 2

Schematic diagram of the proposed framework.

Figure 3

Schematic diagram of method of extracting invariant transformation using autoencoder.

Figure 4

Results of case (i): half sphere. (a) Data set used for the evaluation. There are 1671 samples. (b) Black dots represent sampling distributions $D_a$ obtained by method 1 and red curves represent fitting curves estimated by method 2. Each graph shows 12 combinations of six transformation variables $a_{ij}$. (c) Eigenvalues of the principal components of the distribution $D_a$. (d) Principal component vectors of the distribution $D_a$ with their eigenvalues greater than zero.

Figure 5

Results of case (ii): Constant-velocity linear motion. (a) Conceptual diagram of constant-velocity linear motion. (b) Distribution of time-series data on $q\text{–}p$ plane. (c) Black dots represent sampling distribution $D_a$ obtained by method 1 and the red line represents the fitting curve estimated by method 2. (d) Eigenvalues of the principal components of the distribution $D_a$. (e) Principal component vectors of the distribution $D_a$ with their eigenvalues greater than zero.

Figure 6

Results of case (iii): Two-dimensional central force system. (a) Conceptual diagram of two-dimensional central force system. (b) Distributions of time-series data on $q_1\text{–}{p}_2$ and $q_2\text{–}{p}_1$ planes. (c) Black dots represent sampling distribution $D_a$ obtained by method 1 and the red line represents the fitting curve estimated by method 2. (d) Eigenvalues of the principal components of the distribution $D_a$. (e) Principal component vectors of the distribution $D_a$ with their eigenvalues greater than zero.

Figure 7

Results of case (iv): Collective motion system. (a) Simulation snapshot of torus-type collective motion. The simulation data were applied to the proposed method. (b) Distributions of time-series data on $q_1\text{–}{p}_2$ and $q_2\text{–}{p}_1$ planes. (c) Black dots represent the sampling distribution $D_a$ obtained by method 1 and the red line represents the fitting curve estimated by method 2. (d) Eigenvalues of the principal components of the distribution $D_a$. (e) Principal component vectors of the distribution $D_a$ with their eigenvalues greater than zero.

Figure 8

Qualitative transition of sampling results due to the increase in noise intensity. The figures describe the qualitative transition of the case (iii) central force system where rotation symmetry exists. (a) Distributions of MSE with different noise intensities. (b)–(d) Sampling results of $a_{11}$ and $a_{21}$ at each noise intensity.