Adjoint-based machine learning for active flow control

We develop neural-network active flow controllers using a deep learning partial differential equation augmentation method (DPM). The end-to-end sensitivities for optimization are computed using adjoints of the governing equations without restriction on the terms that may appear in the objective function. In one-dimensional Burgers' examples with analytic (manufactured) control functions, DPM-based control is comparably effective to standard supervised learning for in-sample solutions and more effective for out-of-sample solutions, i.e., with different analytic control functions. The influence of the optimization time interval and neutral-network width is analyzed, the results of which influence algorithm design and hyperparameter choice, balancing control efficacy with computational cost. We subsequently develop adjoint-based controllers for two flow scenarios. First, we compare the drag-reduction performance and optimization cost of adjoint-based controllers and deep reinforcement learning (DRL) -based controllers for two-dimensional, incompressible, confined flow over a cylinder at $\text{Re}=100$, with control achieved by synthetic body forces along the cylinder boundary. The required model complexity for the DRL-based controller is 4229 times that required for the DPM-based controller. In these tests, the DPM-based controller is 4.85 times more effective and 63.2 times less computationally intensive to train than the DRL-based controller. Second, we test DPM-based control for compressible, unconfined flow over a cylinder and extrapolate the controller to out-of-sample Reynolds numbers. We also train a simplified, steady, offline controller based on the DPM control law. Both online (DPM) and offline (steady) controllers stabilize the vortex shedding with a 99% drag reduction, demonstrating the robustness of the learning approach. For out-of-sample flows ($\text{Re}=\{50,200,300,400\}$), both the online and offline controllers successfully reduce drag and stabilize vortex shedding, indicating that the DPM-based approach results in a stable model. A key attractive feature is the flexibility of adjoint-based optimization, which permits optimization over arbitrarily defined control laws without the need to match a priori known functions.

Figure 1

DRL: interaction of the agent and environment in a Markov decision process.

Figure 2

Illustration of the DPM optimization steps in Algorithm lg2, with $\tau =T/2$ as an example.

Figure 3

Top left: Relative training loss for a priori and DPM-trained models ${\epsilon }_{\mathrm{rel}}=J_n/J_0$, where $J_0$ is the norm of targets. Remaining quadrants: at $t/T=1$, (a) Solution using best-case a priori ML model trained for 2000 iterations; (b) Solution using DPM model with matching training relative loss (43 iterations); (c) Solution using DPM model trained for 2000 iterations.

Figure 4

In-sample performance: a priori ML (left) and DPM (right) controllers targeting an advecting Gaussian function. Top to bottom in each group: Instantaneous snapshots, uncontrolled baseline solution, controlled Burgers solution, analytical target, and error in the controlled solution.

Figure 5

Out-of-sample performance: a priori ML (left) and DPM (right) controllers targeting a multiscale Fourier function. Top to bottom in each group: Instantaneous snapshots, uncontrolled baseline solution, controlled Burgers solution, analytical target, and error in the controlled solution.

Figure 6

Top: Error of DPM, a priori, and MMS controlled solutions, showing means and confidence intervals $\pm \sigma$ obtained using 10 random processes. Bottom: Instantaneous comparisons of analytical and controlled solutions (left) and control terms (right) at $t/T=1$; DPM models used training window $\tau /T=0.02$.

Figure 7

Left: Geometrical configuration of 2D confined flow, illustrating the unstructured mesh density. Right: Boundary conditions (not to scale).

Figure 8

Location of the body-force actuators (red) in the 2D confined cylinder configuration. The pressure sensors for actuation are located within the same regions.

Figure 9

Mean drag coefficients, obtained by averaging over $t_{\mathrm{test}}=9000\mathrm{\Delta }t$, of DPM flow controllers optimized over different numbers of training iterations $N$. Test models had $<0.01\%$ change for $N>19$. Confidence intervals are obtained from ten independently initialized tests per model.

Figure 10

Left: Instantaneous velocity-magnitude snapshots for the baseline uncontrolled $\mathrm{Re}=100$ flow, DPM-controlled flow (L1H100; 4.6 parameters per point), and DRL-controlled flow (L2H512; 19.4k parameters per point). Right: Instantaneous body-force control magnitudes.

Figure 11

Left: Time evolution of uncontrolled and (in-sample) controlled instantaneous drag coefficient. Right: Mean and RMS drag coefficient and mean lift coefficient for flow times $t\ge 2.5$. The red star indicates the time after which the DPM $C_d^{\prime }<5\%$.

Figure 12

Left: Mesh nodes for the full domain. Right: Zoomed view of the downstream domain sector.

Figure 13

Mean drag coefficient $\langle C_d\rangle$ (left) and Strouhal number $S_t$ (right) at different Reynolds numbers. Data in Ref. [63] are compiled from experimental, DNS, and analytical theories.

Figure 14

Illustration of the DPM-based active-control framework.

Figure 15

Velocity magnitude, pressure, and vorticity magnitude snapshots for the baseline and online-DPM controlled flow. The controlled fields for the simplified-DPM controller are visually similar to those for the online-DPM control.

Figure 16

Time-series drag coefficient $C_d$ and statistical measures of flows controlled by online and simplified DPM actuators.

Figure 17

Sequential snapshots of velocity (blue), pressure (orange), and control forces (red and green) along the cylinder boundary. The control-force axes are cut to show the full extents.

Figure 18

Drag reduction using the online ($\mathrm{Re}=100$-trained) and constant (simplified) DPM controllers for out-of-sample Reynolds numbers.