Simulation of stochastic systems via polynomial chaos expansions and convex optimization

Polynomial chaos expansions represent a powerful tool to simulate stochastic models of dynamical systems. Yet, deriving the expansion's coefficients for complex systems might require a significant and nontrivial manipulation of the model, or the computation of large numbers of simulation runs, rendering the approach too time consuming and impracticable for applications with more than a handful of random variables. We introduce a computationally tractable technique for computing the coefficients of polynomial chaos expansions. The approach exploits a regularization technique with a particular choice of weighting matrices, which allows to take into account the specific features of polynomial chaos expansions. The method, completely based on convex optimization, can be applied to problems with a large number of random variables and uses a modest number of Monte Carlo simulations, while avoiding model manipulations. Additional information on the stochastic process, when available, can be also incorporated in the approach by means of convex constraints. We show the effectiveness of the proposed technique in three applications in diverse fields, including the analysis of a nonlinear electric circuit, a chaotic model of organizational behavior, and finally a chemical oscillator.

Figure 1

RLC circuit with stochastic parametric uncertainty and stochastic input. Layout of the considered electric circuit.

Figure 2

RLC circuit with stochastic parametric uncertainty and stochastic input. Nonlinear characteristic of capacitance $C\left(v_C\right)\left(H\right)$.

Figure 3

RLC circuit with stochastic parametric uncertainty and stochastic input. Comparison between the exact covariance function $C_D(t_1,t_2)$ of the stochastic input (solid line) and the covariance function ${\tilde{C}}_D(t_1,t_2)$ obtained with the Karhunen-Loeve expansion (dashed line).

Figure 4

RLC circuit with stochastic parametric uncertainty and stochastic input. Examples of ten different realizations of the stochastic input $i_D\left(t\right)$ computed with the Karhunen-Loeve expansion.

Figure 5

RLC circuit with stochastic parametric uncertainty and stochastic input. Distance $\parallel {\mathbf{a}}^{*(\nu +1)}-{\mathbf{a}}^{*\left(\nu \right)}{\parallel }_{\infty }$ between the expansion's coefficients computed with two subsequent numbers of data $\nu$, as a function of $\nu$. The plot is related to the voltage at time instant $t_5$, $v_C(t_5,\theta )$.

Figure 6

RLC circuit with stochastic parametric uncertainty and stochastic input. Estimate of the $L_2$ norm of the error between the values of $\widehat{v}\left(\theta \right)$, computed with the chaos expansions estimated with different values of $\nu$, and the true process $v\left(\theta \right)$. The error estimate is expressed as $\%$ of the average $E\left[v\right(\theta \left)\right]$. The plot is related to the voltage at time instant $t_5$, $v_C(t_5,\theta )$.

Figure 7

RLC circuit with stochastic parametric uncertainty and stochastic input. Mean values at $t=t_i,i\in \{1,10\}$ of (a) current $i_L\left(t\right)$ and (b) voltage $v_C\left(t\right)$ obtained either with MC simulations of the model (dashed line with $\circ$) or with the coefficients of the term of degree 0 in the PCEs ${\widehat{i}}_L(t_i,\theta ),{\widehat{v}}_C(t_i,\theta )$ (solid line with $*$) and ${\widehat{i}}_L^{LS}(t_i,\theta ),{\widehat{v}}_C^{LS}(t_i,\theta )$ (dotted line with $⊳$).

Figure 8

RLC circuit with stochastic parametric uncertainty and stochastic input. Variances at $t=t_i,i\in \{1,10\}$ of (a) current $i_L\left(t\right)$ and (b) voltage $v_C\left(t\right)$ obtained either with MC simulations of the model (dashed line with $\circ$) or with the coefficients of the PCEs ${\widehat{i}}_L(t_i,\theta ),{\widehat{v}}_C(t_i,\theta )$ (solid line with $*$) and ${\widehat{i}}_L^{LS}(t_i,\theta ),{\widehat{v}}_C^{LS}(t_i,\theta )$ (dotted line with $⊳$).

Figure 9

RLC circuit with stochastic parametric uncertainty and stochastic input. Comparison between the empirical pdfs of variable $i_L(t_{10},\theta )$ estimated with $100000$ MC simulations with the system model (left) and with $100000$ MC evaluations of the PCEs ${\widehat{i}}_L(t_{10},\theta )$ (middle) and ${\widehat{i}}_L^{LS}(t_{10},\theta )$ (right).

Figure 10

Distance $\parallel {\mathbf{a}}^{*}(\beta ,\zeta )-{\mathbf{a}}^{*}{\parallel }_{\infty }$ between the expansion's coefficients computed with different values of the exponent $\zeta$ in the weights $w\left(l_k\right)=\frac{l_k^{\zeta }}{{\overline{l}}^{\zeta }}$, and with different values of the scalar weight $\beta$. In particular, the cases $\zeta =1$ (solid line), 2 (dotted), 3 (dashed), and 4 (dash-dot) are shown. The plot is related to the voltage at time instant $t_5$ in the first example of the paper, $v_C(t_5,\theta )$.

Figure 11

Stochastic model of innovative search. (a) Mean values and (b) variances of the new ideas $NI$ at the observation periods $t=4,...,30$, estimated either with $100000$ standard MC simulations (dashed line with $\circ$) or with the polynomial chaos expansions computed with the convex optimization approach (solid lines with $*$).

Figure 12

Stochastic model of innovative search. Level curves of the pdf of new ideas $NI$ as a function of the observation period, estimated by means of (a) $100000$ standard MC simulations and (b) polynomial chaos expansions computed with the convex optimization approach.

Figure 13

Chemical oscillator. (a) Mean values and (b) variances of the expected number $\overline{A}(t,\theta )$ of molecules of protein $A$ estimated every 5 s up to 50 s of simulation, either with $10000$ standard MC simulations (dashed line with $\circ$) or with the polynomial chaos expansions computed with the convex optimization approach (solid lines with $*$).

Figure 14

Chemical oscillator. (a) Mean values and (b) variances of the variance ${\sigma }_A^2(t,\theta )$ of molecules of protein $A$ estimated every 5 s up to 50 s of simulation, either with $10000$ standard MC simulations (dashed line with $\circ$) or with the polynomial chaos expansions computed with the convex optimization approach (solid lines with $*$).

Figure 15

Chemical oscillator. Level curves of the pdf of the average value $\overline{A}(t,\theta )$ of the number of protein $A$ molecules as a function of time, estimated by means of (a) $10000$ standard MC simulations and (b) polynomial chaos expansions computed with the convex optimization approach.

Figure 16

Chemical oscillator. Level curves of the pdf of the variance ${\sigma }_A^2(t,\theta )$ of the number of protein $A$ molecules as a function of time, estimated by means of (a) $10000$ standard MC simulations and (b) polynomial chaos expansions computed with the convex optimization approach.