Reconstruction of three-dimensional scroll waves in excitable media from two-dimensional observations using deep neural networks

Scroll wave dynamics are thought to underlie life-threatening ventricular fibrillation. However, direct observations of three-dimensional electrical scroll waves remain elusive, as there is no direct way to measure action potential wave patterns transmurally throughout the thick ventricular heart muscle. Here we study whether it is possible to reconstruct simulated scroll waves and scroll wave chaos using deep learning. We trained encoding-decoding convolutional neural networks to predict three-dimensional scroll wave dynamics inside bulk-shaped excitable media from two-dimensional observations of the wave dynamics on the bulk's surface. We tested whether observations from one or two opposing surfaces would be sufficient and whether transparency or measurements of surface deformations enhances the reconstruction. Further, we evaluated the approach's robustness against noise and tested the feasibility of predicting the bulk's thickness. We distinguished isotropic and anisotropic, as well as opaque and transparent, excitable media as models for cardiac tissue and the Belousov-Zhabotinsky chemical reaction, respectively. While we demonstrate that it is possible to reconstruct three-dimensional scroll wave dynamics, we also show that it is challenging to reconstruct complicated scroll wave chaos and that prediction outcomes depend on various factors such as transparency, anisotropy, and ultimately the thickness of the medium compared to the size of the scroll waves. In particular, we found that anisotropy provides crucial information for neural networks to decode depth, which facilitates the reconstructions. In the future, deep neural networks could be used to visualize intramural action potential wave patterns from epi- or endocardial measurements.

Figure 1

Deep learning-based reconstruction of scroll wave chaos inside a three-dimensional volume from partial observations of the dynamics on its surface. Scroll wave chaos is a model for the electrophysiological dynamics underlying ventricular fibrillation. Computer simulations were performed in isotropic and anisotropic bulk-shaped excitable media. A neural network (NN) predicts scroll wave dynamics underneath the bulk's surface from a short temporal sequence ($t_1,...,t_5$) of two-dimensional observations (here shown for top layer, see also Fig. 2).

Figure 2

Surface observations and projections of three-dimensional scroll wave chaos (“turbulent” parameter regime) in a bulk medium. (a) Observation of the top surface of the bulk (layer 1) in single-surface mode. (b) Observation of the top and bottom surfaces of the bulk (layers 1 and 24) in dual-surface mode. (c) Observation of the electrical activity and mechanical motion on the top surface of the bulk (layer 1) in single-surface mode (red: Motion vectors). In (a)–(c), the medium is opaque and does not allow observations of the dynamics inside the medium below the top layer. (d) Observation of the projection (transillumination) of the three-dimensional dynamics along the depth of the bulk. The projection is calculated by summing the values in all 24 layers along the $z$ axis for a specific $(x,y)$ coordinate and dividing the sum by the number of layers. All layers 1–24 are cross sections in the $x-y$ plane.

Figure 3

Observations of three-dimensional scroll wave (“laminar” parameter set) on the top and bottom surfaces of an opaque medium, and in the projection of the full dynamics in a transparent medium in the bulk's $z$ direction (depth). In each case, the neural network analyzes a short sequence of five snapshots, which are sampled at discrete times (red) over the period of the scroll wave from the simulation data, see Sec. 2b for details. In the simulations, one rotational period corresponds to about 500 simulation time steps.

Figure 4

Predictions of three-dimensional “laminar” scroll wave dynamics from two-dimensional observations using deep convolutional encoding-decoding neural network (U-Net) in an anisotropic excitable medium ($128\times 128\times 24$ voxels). (a) Ground-truth scroll wave dynamics (five random representative snapshots). The simulations exhibit scroll waves with meandering and curved vortex cores, see also Fig. 5, wavebreak, and dissociation between the top and bottom surface dynamics, see Fig. 6. (b) Predictions from two-dimensional wave pattern visible only on the top surface of the bulk when the medium is completely opaque. The reconstruction accuracy decreases slightly with increasing depth (wave pattern becomes fuzzy toward the bottom). (c) Predictions from two-dimensional projection of the whole three-dimensional dynamics along the $z$ axis in a transparent medium. The prediction accuracy is slightly better than in (b), particularly toward the bottom layers of the bulk. Overall, the predictions and the ground truth are visually difficult to distinguish from each other. The data were not seen by the network during training. See Supplemental Videos 1–4 for corresponding scroll wave dynamics [34].

Figure 5

Predictions of three-dimensional scroll wave dynamics (“laminar” parameter set) and their vortex filaments from either single- or dual-surface observations in a thick, opaque, anisotropic excitable medium ($128\times 128\times 40$ voxels), see also corresponding Supplemental Video 2 [34]. (a) Ground-truth scroll wave dynamics. (b) Prediction from the top surface only. (c) Prediction from both the top and bottom surfaces. In dual-surface mode, the network is able to recover the dynamics sufficiently well. Arrow indicates direction of cross-sectional view. (d) Ground-truth vortex filaments (gray) and vortex filaments calculated from predicted scroll wave dynamics (red). In single-surface mode, predictions become unreliable (wave pattern becomes fuzzy or vortex filaments do not match) toward the bottom of the bulk. Reconstructions performed with U-Net.

Figure 6

Different electrical wave patterns as seen in the (a) top layer, (b) bottom layer, and (c) projection of all layers of a three-dimensional bulk. Bulk sizes are $128\times 128\times 24$ voxels (left and right) and $128\times 128\times 40$ voxels (center), respectively. With increasing bulk thickness or smaller scroll waves, the top and bottom layers are dissociated because the dynamics become increasingly three-dimensional. The projection is calculated for a given $(x,y)$ coordinate by averaging the $u$ values ($u\in [0,1]$) along the depth ($z$ direction) of the bulk. Data from left to right shown in Figs. 4, 5, and 9, respectively. The left and center snapshots are from the “laminar” parameter regime, while the ones on the right are from the “turbulent” chaotic parameter regime.

Figure 7

Bulk thickness and transmurality of scroll wave dynamics. “Turbulent” scroll wave chaos with different bulk thicknesses $d_z=\{8,12,16,20,24\}$, also shown in corresponding cross sections. The dynamics are quasi-two-dimensional with $d_z=8$. Dissociation between top and bottom layers starts to emerge at $d_z=12$ as the dynamics become increasingly three-dimensional. At $d_z>12$ the dynamics are fully three-dimensional.

Figure 8

Predictions of (“turbulent”) electrical scroll waves in subsurface layers of anisotropic (left) and isotropic (right) bulk tissue with dimension $128\times 128\times 24$ voxels. (a) Ground-truth scroll wave dynamics (representative snapshots). (b) Prediction in dual-surface mode analyzing the top and bottom layers of an opaque bulk tissue. (c) Prediction in single-surface mode analyzing the top surface layer of an opaque bulk. (d) Prediction analyzing the $z$ projection of the dynamics along its depth (or $z$ axis) in a transparent bulk. The depth profile of the prediction error (RMSE: Root mean-squared error) along the $z$ axis is shown to the right of each prediction. The reconstruction is successful in anisotropic transparent media but fails in isotropic transparent media, as shown in (d). In opaque media, the reconstruction performs sufficiently well in dual-surface mode with larger errors emerging at midwall of the bulk, as shown in (b). In transparent isotropic media, the subsurface prediction fails because the network is unable to infer the depth of the layers. Cross sections intersect the bulk at its center. All reconstruction with U-Net.

Figure 9

Predictions of electrical scroll waves within subsurface layers of a bulk-shaped “turbulent” anisotropic excitable medium from observing either (a) the top layer of the bulk or (b) the projection of the three-dimensional wave pattern in a transparent bulk along its depth (or $z$ axis). The bulk's dimensions are $128\times 128\times 24$ voxels and predictions were performed with the U-Net architecture. Predictions are shown for the 24 layers along the $z$ axis of the bulk, where the first layer is the top layer and the $24\mathrm{th}$ layer is the bottom layer. First column: The five two-dimensional frames ($t_1,...,t_5$) which are the input for the neural network prediction. Second column: Ground-truth (GT) electrical excitation wave pattern within cross-sectional layers (1–24), of which layers 2–24 cannot be observed. Third column: Prediction of the current cross-sectional layer (1–24) by the neural network. Fourth column: Absolute difference per voxel between prediction and ground truth.

Figure 10

Average reconstruction error over bulk depth (RMSE: Root mean-squared error along $z$ axis) in [(a), (b), (d), and (e)] opaque or [(c) and (f)] transparent excitable media with anisotropy (with varying bulk depths of $d_z\in \{8,12,...,40\}$). All reconstructions were performed with U-Net. [(a)–(c)] “Laminar” scroll wave dynamics as shown in Figs. 4 and 5: (a) Single-surface mode, see also Figs. 4 and 5. (b) Dual-surface mode. (c) Projection, see also Figs. 4 and 5. “Turbulent” scroll wave chaos as shown in Fig. 7. (d) Single-surface mode, see also Figs. 8, and 9. (e) Dual-surface mode, see also Figs. 2 and 8. (f) Projection, see also Figs. 8 and 9. An error of 0.1 (RMSE) corresponds to a mean absolute error (MAE) of about $5\%$. In opaque excitable media, the reconstruction error increases approximately linearly with depth. In transparent media with anisotropy the error remains flat and below 0.1. We trained separate U-Net neural networks for each combination. See also Supplemental Fig. S1 for a comparison with isotropic excitable media and Supplemental Fig. S2 for a comparison of the deep learning-based reconstruction with a naive reconstruction in which the top layer is simply repeated in each following layer [34].

Figure 11

Comparison of reconstruction errors obtained with different imaging configurations in opaque anisotropic excitable medium (with thickness $d_z=24$ layers) with (a) “turbulent” scroll wave chaos and (b) “laminar” scroll wave dynamics. In the different configurations, the network (U-Net) analyzes (i) in single-surface mode the electrics on the top layer (blue), (ii) in single-surface mode the electrics and motion on the top layer (green), (iii) in dual-surface mode the electrics on both top and bottom layers (orange), and (iv) the $z$ projection of the three-dimensional electrics (red). All reconstruction errors were calculated per depth as root mean-squared error (RMSE).

Figure 12

Average reconstruction error per layer (depth) over time in an anisotropic excitable medium with thickness $d_z=24$ shown for 5 different depths. Left: In opaque media and single-surface mode, the prediction error increases and fluctuates more with increasing depth. Right: In transparent media, the prediction error stays small throughout the different depths. Both plots derived for the “laminar” scroll wave dynamics as shown in Fig. 4.

Figure 13

Noise does not pose a limitation for the deep learning-based reconstructions (performed with U-Net on “laminar” scroll wave dynamics). Reconstructions succeed in the presence of various noise levels ($\sigma =0.05,...,0.2$) in both (a) opaque and (b) transparent anisotropic excitable media. Top: Gaussian noise with standard deviation $\sigma$ was added onto the input images. Bottom: Reconstruction error profiles rise slightly with higher noise (light pink curve: $\sigma =0.0$, black curve: $\sigma =0.2$).

Figure 14

Reconstruction errors obtained with different neural network architectures for “turbulent” scroll wave chaos in a bulk with thickness of $d_z=24$. (a) Steep increase of reconstruction error with all networks (encoder-decoder, U-Net, TransUNet, and MIRNet) in opaque excitable media from a single surface (top). (b) Low and relatively flat reconstruction errors (below $10\%$) with all networks in transparent anisotropic excitable media (projection). MIRNet performs slightly better than the other networks. All reconstruction errors stated as root-mean-squared error (RMSE).

Figure 15

Prediction of bulk thickness from surface observations of scroll wave chaos in [(a) and (c)] transparent bulk medium with projection observations and [(b) and (d)] opaque bulk medium with top surface observations. The bulk thickness was predicted using either [(a) and (b)] a regression (black line shows ideal prediction $d_{\text{prediction}}=d_{\text{true}}$) or [(c) and (d)] a classification neural network, respectively (all in anisotropic media). In transparent media, the thickness can be predicted from observations as shown in Fig. 6, whereas in opaque media neither the regression nor classification neural networks predict the thickness correctly. (e) Exemplary projection images for bulk thicknesses or depths $d_z=8$, $d_z=16$, and $d_z=32$. Due to the averaging, the contrast of the waves decreases with increasing depths.