Machine learning time-local generators of open quantum dynamics

In the study of closed many-body quantum systems one is often interested in the evolution of a subset of degrees of freedom. On many occasions it is possible to approach the problem by performing an appropriate decomposition into a bath and a system. In the simplest case the evolution of the reduced state of the system is governed by a quantum master equation with a time-independent, i.e. Markovian, generator. Such evolution is typically emerging under the assumption of a weak coupling between the system and an infinitely large bath. Here, we are interested in understanding to which extent a neural network function approximator can predict open quantum dynamics - described by time-local generators - from an underlying unitary dynamics. We investigate this question using a class of spin models, which is inspired by recent experimental setups. We find that indeed time-local generators can be learned. In certain situations they are even time-independent and allow to extrapolate the dynamics to unseen times. This might be useful for situations in which experiments or numerical simulations do not allow to capture long-time dynamics and for exploring thermalization occurring in closed quantum systems.

Introduction-The investigation of isolated quantum systems out of equilibrium is a central problem in physics [1,2]. Often, partial information that concerns a spatially localized subsystem S is sufficient to study dynamical effects related to relaxation or thermalization. This information is encoded in the so-called reduced quantum state ρ S , [c.f. Fig. 1(a,b)], whose time dependence is obtained by "integrating out" the remainder of the evolving many-body system, which is often referred to as bath B [see Fig. 1(c)]. In certain settings, e.g. for systems which are weakly interacting with a large environment [3,4], the dynamics of the reduced state is effectively described by a quantum master equation [5][6][7]. In the simplest manifestation, the dynamics of the reduced quantum state ρ S is then governed by a timeindependent generator L acting only on S: Here, U t is the unitary evolution operator of the full many-body system, the state ρ S ⊗ ρ B is the initial state of the many-body system in product form, and Tr B indicates the average over the bath B. However, the generator does not need to be time-independent and, in most generic instances [8], it may depend locally on time, L t , as sketched in Fig. 1(c).
In this paper, we show how simple neural network architectures can learn time-local generators that govern the local dynamics of a closed quantum spin system. The input of the networks consists of the time-dependent average values of the reduced system observables from which the dynamical generator is estimated. Firstly, we consider an architecture which provides a time-averaged Markovian generator time-dependent generator ?
Reduced dynamics

Model II
FIG. 1. Reduced local dynamics of a many-body quantum system. The reduced state ρS describes local degrees of freedom of a quantum spin system. The remainder of the system, B, takes the role of a "bath", with initial state ρB. We consider two scenarios: (a) Model I: spin chain with open boundary conditions, where the interaction strength V and range α are varied [see Eq. (2)]; (b) Model II: spin chain with closed boundaries and nearest-neighbor interaction, where the system-bath coupling V and the initial temperature β of the bath are modified. (c) The dynamics of the reduced state, ρS(t), is obtained by tracing out the degrees of freedom of B and to be learned by a neural network. The learned generator may be either Markovian (time-independent) or timedependent.
generator L corresponding to an effectively Markovian description of the system dynamics. Our findings indicate that this can yield indeed a valid approximation for the generator even beyond typical scenarios justifying a Markovian weak-coupling assumption [3,4]. Once the generator is known, the network can further be exploited to make predictions for times which go beyond the previously analyzed time frame, as is sketched in Fig. 2. To study settings with a time-dependent generator, L t , we use a different neural network architecture which is based on so-called hypermodels and allows to assess the time- Effective time-independent generator: Training and extrapolation. To obtain a time-independent generator we use a multi-layer perceptron (MLP). The inputs are the time-dependent expectation values of the reduced system observables -for a single spin contained in the Bloch vector v b -in a given training time window (shaded blue region) and for several initial Bloch vectors (top to bottom). The MLP learns the "propagator" M [θ * ], which depends on the optimized network parameters θ * but not on the actual time.
Considering an Euler integration step of length dt, this provides a matrixL representing the action of an "averaged" generator on the Bloch vector. When the reduced dynamics is Markovian, this learned generator allows for the network to make predictions for unseen future times (red shaded region).
dependence of the generator of the reduced dynamics. Our work links to recent efforts aiming at understanding quantum dynamics with neural networks [9][10][11][12][13][14][15][16][17][18][19][20]. Our approach, which uses machine learning tools and provides a directly interpretable object such as the physical generator of the dynamics, highlights a possible route for the application of neural networks in the study of the long-time dynamics of local observables in closed quantum systems. Moreover, it may find applications in the context of quantifying the degree of non-Markovianity in reduced quantum dynamics [8].
Many-body spin models-For illustrating our ideas, we consider two one-dimensional systems (in the following referred to as Model I and II) consisting of N interacting spins, which are inspired by recent state-of-the-art quantum simulator experiments with Rydberg atoms [21][22][23][24][25] or trapped ions [26][27][28]. Model I has open boundary conditions [see Fig. 1(a)]. It features power-law interactions and is described by the Hamiltonian Here, n i = (1 + σ z i )/2 is a projector on the "up"-state of the i-th spin and σ x,y,z i are Pauli matrices. The parameter α accounts for the interaction range, i.e. the powerlaw decay of the interaction potential, and V controls the overall interaction strength with respect to the strength Ω of a transverse field term. The reduced system S, for this model, is the middle spin of the chain, as shown in Fig. 1(a). Model II is a closed spin chain [sketched in Fig. 1 n j n j+1 + Ω σ x 1 + V n 1 (n 2 + n N ).
(3) Here, the spin with label 1 is singled out as reference spin (forming the system S) and the interaction strength with its neighbors is given by V , while the interaction strength among all other spins is V . Moreover, we assume that the strength Ω of the transverse field acting on the reference spin can be controlled. Both models capture a whole variety of different scenarios, including short-and long-range interactions, absence and presence of translational invariance as well as weak and strong coupling between local and bath degrees of freedom. They moreover encompass a number of standard scenarios often explored in quantum many-body physics, such as the so-called PXP-model [29][30][31], the Ising model in the presence of longitudinal and transverse fields, and all-to-all connected spin systems with resemblance to the Dicke model [32] or the Lipkin-Meshkov-Glick model [33].
Our aim is to investigate the reduced dynamics of local degrees of freedom, as for instance a single spin S, as depicted in Fig. 1(a,b). To this end, we assume that the system is initialized at time t = 0 in the state ρ = ρ S ⊗ρ B . In our studies concerning Model I, we assume the bath to be in the infinite temperature state, ρ B ∝ 1 B . When considering Model II, instead, we assume a finite temperature situation with ρ B ∝ e −βHII , where the HamiltonianH II is the one of Eq. (3) with Ω = V = 0, and β is the inverse temperature. We let the state ρ evolve according to the unitary dynamics ρ(t) = U t ρU † t , with U t = e −iHt from which we obtain the full dynamical information about the subsystem S. Throughout we will make use of the Bloch vector representation of the subsystem's density matrix: Here the Bloch vector v b (t) = 1, σ x k t , σ y k t , σ z k t is given in terms of the expectation · t with respect to the full quantum state at time t, and k labels the system spin [see Fig. 1 In the following, we will characterize the generator of this reduced dynamics; that is, we want to find a generator acting solely on S that reproduces (or approximates) the dynamics of v b (t) for the two models under consideration [ Fig. 1(a,b)]. For Model I we explore for which parameter combinations (V, α), the local generator, which is in principle time-dependent L t , can be approximated by an effective "averaged" time-independent one, L t ≈L. We remark here that, while B acts as a fictitious bath for subsystem S, the situation described here is far from that of typical weak-coupling limits. Indeed, B is a finite quantum system and its Hamiltonian has a discrete spectrum. As such, B can hardly be thought of as an infinite Markovian bath of bosonic oscillators, whose state is unaffected by the interaction with S. The only aspect in common with a weak-coupling limit is that the interaction strength can be made small. However, this is not sufficient to argue that the local dynamics may be described by time-independent generators.
For Model II we consider the case in which an "isolated" spin (on site 1) interacts with B with an independently tunable strength V . Experimentally such a situation can be realized, e.g., by exploiting the distance dependence of dipolar interactions of atoms in Rydberg states [24]. Also for Model II we investigate how well the generator of the reduced dynamics is approximated by a time-independent model when the initial state of B (parametrized by the inverse temperature β) and the interaction strength V is varied. In addition, for this scenario, we use a hypermodel in order to analyze regimes that necessitate a time-dependent generator L t .
Network architectures and training-In order to learn dynamical generators for the reduced dynamics of subsystem S, we employ machine learning algorithms. Our approach is completely data-driven, i.e. the network has no prior information about the physical system. For learning time-independent generators, we use a linear multi-layer perceptron (MLP) architecture (see Fig. 2) which turns out to be the simplest possible one. Every data-point of our data-set D contains the triple ( is the value of the Bloch vector at a given time-point t, and v b is the value of the Bloch vector after a further discrete time step The input of our model is v b , and the output is the vec- is a 4 × 4-matrix (matching the dimension of the Bloch vector), that depends on the parameters θ of the network. To optimize the network, we introduce the following loss function, which is given by the norm of the difference between the next time step v b and the output of the network: where the expectation is taken over the training data D. Minimizing the above function thus provides a model with x 2 + y 2 + z 2 < 1, i.e. we consider non-pure initial states. We split our data-set such that we consider the 80% of it for training, i.e. finding the best parameters of our model. The remaining 20% form the validation set to assess the performance of the model built in the training phase.
To encode and quantify the time-dependence of dynamical generators, L t , we use a different architecture and consider a so-called hypermodel, which computes network weights based on a context input. In our case, the context input is the time t and a multi-layer perceptron (MLP) F with non-linear activation functions transforms this input to the 4 × 4 matrix of the generator M [F (t; θ)].
The data-set is organized in the same way as for the time-independent model. The only difference with respect to the previous case is that we also pass the information about the actual time step t. The network parameters θ are optimized by minimizing the loss function By training the model in this way we obtain a timedependent representation of the propagator of the local dynamics, i.e. M [F (t; θ * )] = 1 + L t dt. For the training set, we choose quantum states as in the previous case. For the evaluation data-set, instead, we consider with c being a random number, c ∈ (0.01, 0.7) [34]. Time-independent generators-With the above numerical approach, we can now investigate under which circumstances the generator of the dynamics of the subsystem S is time-independent. We consider first the open spin chain [see Fig. 1(a) and Eq. (2)] and explore different values of V and α. To test the performance of the model after the training, we study the time-averaged norm difference between the exact Bloch vector v ex b -obtained by simulating the full many-body spin chain dynamicsand the time-evolution of the same quantity as predicted by our model v mod b : computed by numerical integration. As shown in Fig. 3(a), the error is small for shortrange (large α) and weakly interacting (with small ratio V /Ω) spin chains. This means that our time-independent model captures the relevant features of the reduced dynamics, and suggests that for finite systems with sufficiently weak interactions, the dynamics of local degrees of freedom can indeed be effectively described by a timeindependent generatorL. Furthermore, for this Markovian regime of the dynamics, our model allows to make predictions for times exceeding the training time.
We now turn to the discussion of Model II [see Fig. 1(b) and Eq. (3)]. Using the same procedure explained above, we can learn the time-averaged dynamical generator for subsystem S. In Fig. 3(b) we summarize the results of this analysis. We display the time-averaged error, as defined in Eq. (8), for different parameter choices, which in this case are the (inverse) temperature β of the initial state ρ B and the interaction strength V . We furthermore take Ω = Ω. As shown in the figure the dynamics is well approximated by a time-independent generator, L, for weak interactions V and for small β. Here also the extrapolation to times that exceed T train is possible. The data in Fig. 3 shows that in certain parameter regimes the approximation of the dynamics through a time-independent generator does not work well. Here significant deviations are even observable during the training period.
Time-dependent time-local generators-We are now interested in analyzing how strongly the dynamical generator depends on time, in those instances where the model introduced in the previous section fails. To this end, we adopt a hypermodel which can take as input the information about the running time. In this way, the neural network is able to learn an optimal parametrization of the "propagator" M [F (t; θ * )] = 1 + L t dt for the The time-dependence is quantified through the derivative of Lt with respect to time, expressed by the positive quantity ξ(t) (see text for definition). Panel (a) shows three example curves corresponding to Model II (N = 9). The function ξ(t) depends only on the parameters of the model and is independent from the initial condition chosen. Generally, the stronger the interaction V , the more pronounced the time dependence. This trend is clearly visible in panel (b), where we show the averaged value of ξ(t) in the interval 0 to T = 10/Ω, denoted as Ξ. In order to obtain the density plot we averaged, also in this case, over 5 different values of c in Eq. (7), also in this case dt = 0.01Ω.

Bloch vector dynamics which explicitly depends on time.
Here, the matrix L t encodes the action of the generator on the Bloch vector and, essentially, implements the differential equations obeyed by the entries of the Bloch vector v b . This architecture allows to learn and accurately reproduce the dynamics of Model II for all studied parameter regimes (see Supplemental Material for examples). This is not surprising, but gives us a handle for analyzing the time-dependence of the dynamical generator L t . To this end we consider the positive quantity ξ(t) = Tr(L † tLt ), where the dot denotes the derivative with respect to time. This can only be zero wheṅ L t = 0, i.e. when the action of the generator is not depending on the running time. To quantify an overall time-dependence within a time window [0, T ], we define the time-averaged value Ξ = 1 T T 0 dt ξ(t). In Fig. 4 we show the corresponding data. For strong interactions V , ξ(t) is non-zero throughout which indicates a strong explicit time-dependence. This is also reflected in a large Ξ. For weak interactions, on the other hand, ξ(t) remains small for all times considered. The oscillation we attribute to the finite size of the system. This confirms that, in such parameter regime, it is indeed possible to approximate the dynamics of the Bloch vector by means of a time-independent matrix, as considered in the previous section.
Conclusions-We have presented two simple examples of neural network architectures that can learn the dynamical features of reduced quantum states. When such time evolution is effectively Markovian, the network can find a suitable approximation for the generator of the local dynamics. Here one can extrapolate the dynamics of reduced degrees of freedom to times that were unexplored during the training procedure. This possibility is particularly promising for applications in combination with tensor networks, which perform extremely well for short times. A neural network could learn the time-independent generator during this time interval and then possibly extrapolate to long times. In principle, this might enable the exploration of the onset of stationary or thermalization regimes. When the dynamics is not of Markovian type, we have shown that hypermodels can recover the generator of the reduced quantum dynamics. This allows to quantify non-Markovian effects [35], which manifest in an explicit time-dependence of the dynamical generator Ref. [8]. Furthermore, being able to reconstruct the generator of the reduced dynamics, as we have done here with hypermodels, makes it possible to explore different measures of non-Markovianity based on non-divisibility criteria for quantum dynamical timeevolutions [36][37][38][39].
A possible future development in this regard could be the application of more advanced machine learning methods for learning time correlations in time series. This can be achieved with models borrowed from language studies (e.g. long-short term memory recurrent neural networks (LSTM) [40] and transformers [41]) or with algorithms geared towards more interpretable models by either learning analytical expressions of the differential equation [42] or by modelling the time-evolution dynamics directly [43].
Acknowledgments-We acknowledge financial support In this supplemental material we report a more detailed analysis concerning the effectiveness of the hypermodel in predicting the dynamics of the Bloch vector within the training time-window. In addition, we provide the hyperparameters of the used models.

HYPERMODEL VS. EXACT DYNAMICS
In the main text we analyze the time-dependence of time-local generators parametrizing the "propagator" M [F (t; θ * )] = 1 + dtL t for the Bloch vector, using a hypermodel architecture. The matrix L t encodes all the information concerning the dynamics of local (one-site) observables. As stated in the main text, a suitable measure to quantify the dependence on time of this matrix is given by Ξ = 1 T T 0 dt ξ(t), where ξ(t) = Tr(L † tLt ). However, this can be a faithful measure of the time-dependence of the physical generator, only if L t correctly implements the dynamics of the Bloch vector. It is thus important to check that the prediction of the hypermodel matches the exact dynamics, in the time window [0, T ] in which we want to analyze the generator L t . To this end, we report in Fig. 1 a detailed comparison between the predicted time evolution of the Bloch vector components and the exact ones for six different parameter regimes. We start from a generic initial state of the form (1) and for N = 7. As it can be observed the agreement between the model prediction (dotted line) and the exact dynamics (full line) is excellent. This fact enables us to use the local operator learned using the hypermodel for studying the time-dependence of Lt.
with x 2 + y 2 + z 2 < 1, in particular we choose x = 0.6, y = 0.3 and z = 0.4, and we consider different interactions and initial bath temperatures. In the training time-window [0, T ], we observe an excellent agreement between the two curves for T = 10/Ω. In Fig. 2 we report the time evolution of the quantity ξ(t) for the same parameter regimes. As it can be observed for weakly interacting systems the generator L t is almost time-independent. In this regime, as discussed in the main text, it is possible to predict the local dynamics using simple time-independent architectures. Increasing the interaction, the parameter Ξ grows accordingly signaling that the dependence on time of L t becomes stronger. The training procedure for the hyper-model is slightly different with respect what we have done for the timeindependent case, in particular it is different the differentiation between training and evaluation set. In this case, in fact, we generate two different types of data-sets, the first data-set for training consists in 100 trajectories with randomly chosen initial state ρ 0 of the the same form reported in Eq. (1). For evaluation we select, instead, randomly data-points from trajectories generated from initial states ρ ev 0 of type with 0.01 < c < 0.7. In this way we are performing a separation of the data-sets in the Hilbert space. This is contrary to what we had for the time-independent case where the data-set was given by separated time-intervals of the same trajectory.