Optimal dynamic incentive scheduling for Hawk-Dove evolutionary games

The Hawk-Dove evolutionary game offers a paradigm of the trade-offs associated with aggressive and passive behaviors. When two (or more) populations of players compete, their success or failure is measured by their frequency in the population, and the system is governed by the replicator dynamics. We develop a time-dependent optimal-adaptive control theory for this dynamical system in which the entries of the payoff matrix are dynamically altered to produce control schedules that minimize and maximize the aggressive population through a finite-time cycle. These schedules provide upper and lower bounds on the outcomes for all possible strategies since they represent two extremizers of the cost function. We then adaptively extend the optimal control schedules over multiple cycles to produce absolute maximizers and minimizers for the system.

Figure 1

Twelve regions in the off-diagonal $(a_{12},a_{21})$ plane [29] (delineated by dashed lines) define which game is being played. We choose $a_{22}$ at the origin (without loss). Starting at $t=0$ in the Hawk-Dove square, indicated by the initial large (red) dot, what are the paths that minimize and maximize aggression at time $t=T$? The solid (red) curve is one example of a path that crosses into four different regions of the plane.

Figure 2

Dynamics of the uncontrolled ($u_1=0, u_2=0$) Hawk-Dove evolutionary game. (a) Phase portrait associated with the aggressor population $x_1$. Both Hawk and Dove dominance ($x_1=1,0$) are unstable fixed points, while the mixed state $x_1=1/3$ is the evolutionarily stable strategy (ESS). (b) Hawk dynamics for various initial conditions. $T=1$ is the end of one control cycle and also the linear growth rate of the Hawk-Dove system.

Figure 3

Maximizing (solid blue) and minimizing (dashed red) trajectories for nine initial conditions. Also shown is the (dashed black) curve for the uncontrolled Hawk-Dove trajectory which lands in between the maximum and minimum at $T=1$. The top bar (dark blue) corresponds to $u_1=1$, with white indicating $u_1=-1$. The second bar (light blue) corresponds to $u_2=1$, with white indicating $u_2=-1$ associated with the maximizing control schedule. The third bar (dark red) corresponds to $u_1=1$, with the white bar indicating $u_1=-1$. The fourth bar (light red) corresponds to $u_2=1$, with white indicating $u_2=-1$ associated with the minimizing control schedule. All schedules are bang-bang. (a) $x_1\left(0\right)=0.01$, (b) $x_1\left(0\right)=0.05$, (c) $x_1\left(0\right)=0.1$, (d) $x_1\left(0\right)=0.3$, (e) $x_1\left(0\right)=0.5$, (f) $x_1\left(0\right)=0.7$, (g) $x_1\left(0\right)=0.9$, (h) $x_1\left(0\right)=0.95$, and (i) $x_1\left(0\right)=0.99$.

Figure 4

Maximizing (solid blue) and minimizing (dashed red) trajectories for nine initial conditions over $n=5$ cycles, with solid (blue)/dashed (red) curves joining the end values after each cycle. The adaptive schedule for the $(n+1)\text{st}$ cycle is calculated based on the endpoint of the $n\text{th}$ cycle. The dashed black curve shows the uncontrolled Hawk-Dove trajectory. (a) $x_1\left(0\right)=0.01$, (b) $x_1\left(0\right)=0.05$, (c) $x_1\left(0\right)=0.1$, (d) $x_1\left(0\right)=0.3$, (e) $x_1\left(0\right)=0.5$, (f) $x_1\left(0\right)=0.7$, (g) $x_1\left(0\right)=0.9$, (h) $x_1\left(0\right)=0.95$, and (i) $x_1\left(0\right)=0.99$.

Figure 5

Maximizing (solid blue) and minimizing (dashed red) trajectories for the two special initial conditions $x_1\left(0\right)=0.08,0.79$. For the larger initial condition, the maximizing schedule (top two bars in blue) produces an absolute maximum $x_1^{\text{max}}$ for $T=1$. For the smaller initial condition, the minimizing schedule (bottom two bars in red) produces an absolute minimum $x_1^{\text{min}}$ for $T=1$. The dashed (black) curve shows the uncontrolled Hawk-Dove trajectory. The top bar (dark blue) shows $u_1=1$, and the white bar shows $u_1=-1$. The second bar (light blue) shows $u_2=1$, and the white bar shows $u_2=-1$ associated with the maximizing control schedule. The third (dark red) bar shows $u_1=1$, and the white bar shows $u_1=-1$. The fourth (light red) bar shows $u_2=1$, and the white bar shows $u_2=-1$ associated with the minimizing control schedule. (a) $x_1\left(0\right)=0.08$, and (b) $x_1\left(0\right)=0.79$.

Figure 6

Maximizing (solid blue) and minimizing (dashed red) trajectories for the two special initial conditions $x_1\left(0\right)=0.08,0.79$ over $n=5$ cycles. Notice that the minimizing trajectory (dashed red) shown in (a) exactly repeats for each cycle $[x_1\left(0\right)=x_1\left(T\right)]$ since the schedule is an absolute minimizer, while the maximizing trajectory (solid blue) shown in (b) exactly repeats for each cycle $[x_1\left(0\right)=x_1\left(T\right)]$ since the schedule is an absolute maximizer. The dashed (black) curve shows the uncontrolled Hawk-Dove trajectory. (a) $x_1\left(0\right)=0.08$. The dashed (red) horizontal line indicates an absolute minimizer at $x_1^{\text{min}}$. (b) $x_1\left(0\right)=0.079$. The solid (blue) horizontal line indicates an absolute maximizer at $x_1^{\text{max}}$.

Figure 7

Minimizing sequence of four games (Deadlock; Leader; Prisoner's Dilemma; Game 9) associated with initial condition $x_1\left(0\right)=0.08$ that produces the absolute minimizer. Open dots show the starting point, and the solid dot shows the ending point.

Figure 8

Maximizing sequence of four games (Prisoner's Dilemma; Leader; Deadlock; Game 9) associated with initial condition $x_1\left(0\right)=0.79$ that produces the absolute maximizer. Open dots show the starting point, and the solid dot shows the ending point.

Figure 9

(a) Hawk initial condition $x_1\left(0\right)$ vs Hawk frequency at final time $x_1\left(T\right)$ for maximizing (solid blue) and minimizing (dashed red) schedules. The vertical dashed line at $x_1\left(0\right)=0.65$ marks the maximum difference between the minimizer and the maximizer. (b) Change in Hawk frequency as a function of initial condition. Points above the line $x_1\left(T\right)/x_1\left(0\right)-1=0$ represent an increase over time and points below this line represent a decrease over time. The two intersection points $x_1\left(0\right)=0.08$ and $x_1\left(0\right)=0.79$ mark the absolute minimizer and maximizer initial conditions for $T=1$.

Figure 10

$x_1^{\text{max}}$ (solid blue) and $x_1^{\text{min}}$ (dashed red) as a function of cycle time $T$. The dashed horizontal line at $x_1=1/3$ is the ESS for the uncontrolled Hawk-Dove system where the two curves meet as $T\to 0$.