Matrix Optimization on Universal Unitary Photonic Devices

Universal unitary photonic devices can apply arbitrary unitary transformations to a vector of input modes and provide a promising hardware platform for fast and energy-efficient machine learning using light. We simulate the gradient-based optimization of random unitary matrices on universal photonic devices composed of imperfect tunable interferometers. If device components are initialized uniform randomly, the locally interacting nature of the mesh components biases the optimization search space toward banded unitary matrices, limiting convergence to random unitary matrices. We detail a procedure for initializing the device by sampling from the distribution of random unitary matrices and show that this greatly improves convergence speed. We also explore mesh architecture improvements such as adding extra tunable beam splitters or permuting waveguide layers to further improve the training speed and scalability of these devices.

Figure 1

Mesh diagram representing the locally interacting rectangular mesh for $N=8$. The inputs (and single-mode phase shifts at the inputs) are represented by blue triangles. Outputs are represented by purple squares. The MZI nodes are represented by red dots labeled with sensitivity index ${\alpha }_{n\ell }$ (e.g., ${\alpha }_{44}=7$ is the most sensitive node). The nodes represent the Givens rotation $U_n$ (in orange) at vertical layer $\ell$ (in green). Each photonic MZI node can be represented with 50:50 beam splitters $B$ (red) and phase shifters $R_{\theta },R_{\varphi }$ (orange), with required ranges $0\le \theta \le \pi$ and $0\le \varphi <2\pi$.

Figure 2

(a) The sensitivity index ${\alpha }_{n\ell }$ for $N=64$. (b) Checkerboard plot for the average reflectivity $⟨r_{n\ell }⟩$ in a rectangular mesh. (c) Decomposition of Ref. [10] for a Haar-random matrix yields phases close to cross state in the middle of the mesh. (d) The Haar phase ${\xi }_{n\ell }$ for the rectangular mesh better displays the randomness. (e),(f) Field measurements (absolute value) from propagation at input 32 in (e) Haar and (f) uniform random initialized rectangular meshes with $N=64$.

Figure 3

(a) Plot of the relationship between ${\xi }_{\alpha }$ and $\theta$. (b) We show that phase-shift standard deviation ${\sigma }_{\theta ;\alpha }$ decreases as $\alpha$ increases. (c) A plot of ${\sigma }_{\theta ;\alpha }$ as $\alpha$ increases. (d) The transmissivity of a MZI component as a function of a periodic Haar phase has a power-law relationship. The periodic Haar phase ${\tilde{\xi }}_{\alpha }$ is mapped to the Haar phase by a function $\xi :\mathbb{R}\to [0,1]$ as discussed in Appendix pp7.

Figure 4

(a) A $16\times 16$ rectangular mesh (red). Extra tunable layers (green) may be added to significantly reduce convergence time. (b) A $16$-input, $30$-layer permuting rectangular mesh. The rectangular permutation layer is implemented using either waveguide crossings or cross-state MZIs (gray).

Figure 5

Elementwise absolute values of unitary matrices resulting from rectangular ($U\sim {\mathcal{U}}_R$) and permuting rectangular ($U\sim {\mathcal{U}}_{\mathrm{PR}}$) meshes, where meshes are initialized with uniform-random phases.

Figure 6

We implement six different optimizations for $N=128$, where we vary the choice of permuting rectangular mesh (PRM) or rectangular mesh (RM); the initialization (random ${\theta }_{n\ell }$ or Haar-initialized ${\theta }_{n\ell }$); and photonic transmissivity error displacements [$ϵ=0$ or $ϵ\sim \mathcal{N}(0,0.01)$, where ${\sigma }_{ϵ}^2=0.01$ is the variance of the beam-splitter errors]. The conditions are 20,000 iterations, Adam update, learning rate of $0.0025$, batch size of 256, simulated in tensorflow. (a) Comparison of optimization performance (defaults are Haar initialization and ${ϵ}_{n\ell }=0$ unless otherwise indicated). The optimized error magnitude spatial map for (b) rectangular mesh shows higher off-diagonal errors than (c) permuting rectangular. The optimized ${\theta }_{n\ell }$ phase shifts (see Appendix pp7) for (d) rectangular meshes are close to zero (cross state) near the center as opposed to (e) permuting rectangular meshes, which have a striped pattern (likely due to initialization). Note that, by $\mid \cdot \mid$, we refer to the elementwise norm.

Figure 7

A comparison of test error in tensorflow for $N=128$ between rectangular (RM), permuting rectangular (PRM), and redundant rectangular (RRM) meshes for 20,000 iterations, Adam update, learning rate of $0.0025$, batch size of 256. Ideal denotes Haar-initialized ${\theta }_{n\ell }$ with $ϵ=0$. $\delta N$ is the additional layers added in the redundant mesh. We stop the $\delta N=128$ run within 4000 iterations when it reaches convergence within machine precision. Redundant meshes with 32 additional layers converge better than permuting rectangular meshes and, with just 16 additional layers, we get almost identical performance.

Figure 8

Triangular mesh for $N=8$ using (a) $2N-3$ vertical layers $\ell$, showing the sensitivity index ${\alpha }_{n\ell }$, and (b) $N$ diagonal layers $m$, showing the transmissivity basis ($t_n$ in red) and the measurement basis ($x_n$ in purple).

Figure 9

Given $\eta =0.001$, we compare bandsizes for rectangular [$U\sim {\mathcal{U}}_R(N,N)$], permuting rectangular [$U\sim {\mathcal{U}}_{\mathrm{PR}}(N)$], and redundant meshes [$U\sim {\mathcal{U}}_R(N,2N)$]. Permuting rectangular meshes match the bandsize of Haar random matrices.

Figure 10

A comparison of test mean-square error for $N=128$ between redundant rectangular meshes with error $ϵ$ for a $256$-layer mesh for 20,000 iterations, Adam update, learning rate of $0.0025$, and batch size of 256, simulated in tensorflow.

Figure 11

A comparison of test mean-square error for $N=64$ between SVD devices using rectangular (SVDRM) and permuting rectangular (SVDPRM) meshes for 20,000 iterations, Adam update, learning rate of $0.005$, and batch size of $128$, simulated in tensorflow. Unless otherwise noted, the default setting is Haar-random-initialized ${\theta }_{n\ell }$ with ${\sigma }_{ϵ}=0$.

Figure 12

Comparison of learned matrix errors and learned ${\theta }_{n\ell }$ weights after 20,000 iterations for the Adam update at learning rate $0.0025$ and batch size $256$ for the simple unitary network. We consider two meshes: (1) rectangular mesh (RM) and (2) permuting rectangular mesh (PRM). We consider three conditions for each mesh: (1) ideal (with Haar random unitary initialization); (2) photonic beam-splitter error displacement $ϵ\sim \mathcal{N}(0,0.01)$; (3) random initialization.

Figure 13

Comparison of learned, normalized ${\theta }_{n\ell }$ distributions for $N=128$ rectangular (RM) and permuting rectangular (PRM) meshes with ${\mathcal{P}}_{\alpha }(\theta /2)$ PDFs for $\alpha =(N+1)/3=43$ (the average sensitivity index) and $\alpha =\lfloor N/(2\log N)\rfloor =9$ respectively. Note that permuting meshes have a larger tolerance, which eventually results in faster mesh optimization.

Figure 14

Rectangular decomposition for even ($N=8$) and odd ($N=7$) meshes, showing the diagonal $x,y$ basis. Values for ${\alpha }_{n\ell }$ are shown in red above each MZI, with values for $s_x[y]$ shown in blue below. The critical boundaries of $x,y=\lfloor N/2\rfloor$ separating the different quadrants are drawn in green. (Boundaries are offset for visual clarity.)