Optimizing information flow in small genetic networks

In order to survive, reproduce, and (in multicellular organisms) differentiate, cells must control the concentrations of the myriad different proteins that are encoded in the genome. The precision of this control is limited by the inevitable randomness of individual molecular events. Here we explore how cells can maximize their control power in the presence of these physical limits; formally, we solve the theoretical problem of maximizing the information transferred from inputs to outputs when the number of available molecules is held fixed. We start with the simplest version of the problem, in which a single transcription factor protein controls the readout of one or more genes by binding to DNA. We further simplify by assuming that this regulatory network operates in steady state, that the noise is small relative to the available dynamic range, and that the target genes do not interact. Even in this simple limit, we find a surprisingly rich set of optimal solutions. Importantly, for each locally optimal regulatory network, all parameters are determined once the physical constraints on the number of available molecules are specified. Although we are solving an oversimplified version of the problem facing real cells, we see parallels between the structure of these optimal solutions and the behavior of actual genetic regulatory networks. Subsequent papers will discuss more complete versions of the problem.

Figure 1

Input proteins at concentration $c$ act as transcription factors for the expression of output proteins, $g$. The diffusive noise in transcription factor concentration and the shot noise at the output both contribute to stochastic gene expression. The regulation process is described using a conditional probability distribution of the output knowing the input, $P\left(g\mid c\right)$, which can be modeled as a Gaussian process with a variance ${\sigma }_g^2\left(c\right)$. In this paper we consider the case of multiple output genes $\left\{g_{\text{i}}\right\}$, $i=1,\dots ,M$, each of which is independently regulated by the process illustrated here with the corresponding noise ${\sigma }_{\text{i}}^2$.

Figure 2

(Color online) Information capacity for one (activator) input and one output. The information is $I={\text{log}}_2{\tilde{Z}}_1+A$, with $A$ independent of the parameters; the map shows ${\tilde{Z}}_1$ as computed from Eq. (37), here with $C\equiv c_{\text{max}}/c_0=1$. We see that there is a broad optimum with cooperativity $n_{\text{opt}}=1.86$ and $K_{\text{opt}}=0.48c_0=0.48c_{\text{max}}$.

Figure 3

(Color online) The optimal solutions for one gene controlled by one transcription factor. The optimization of information transmission in the small noise limit depends on only one parameter, which we take here as the maximum concentration of the input molecules, measured in units determined by the noise itself [$c_0$ from Eq. (20)]. Panel (a) shows the optimal input/output relations with $c_{\text{max}}/c_0=0.3,1,3,10,30,100,300$; activators shown in blue (solid line), repressors in green (dashed line). Although the input/output relation is defined for all $c$, we show here only the part of the dynamic range that is accessed when $0<c<c_{\text{max}}$. Panel (b) shows the optimal distributions, $P_{TF}^{\ast }\left(c\right)$, for each of these solutions. Panel (c) plots ${\text{log}}_2{\tilde{Z}}_1$ for these optimal solutions as a function of $c_{\text{max}}/c_0$. Up to an additive constant, this is the optimal information capacity, in bits.

Figure 4

Approximate results for the optimal values of $K$ (a) and $n$ (b) compared with exact numerical results for activators (black lines) and repressors (gray lines). As explained in the text, we can use our analytic approximations to determine, for example, the optimal $K$ assuming $n$ is known (large $c_{\text{max}}$ with known $n$ results) or we can simultaneously optimize both parameters (large $c_{\text{max}}$ results); results are shown for both calculations.

Figure 5

(a) Mean concentration of the transcription factor when we optimize information transmission subject to a constraint on the maximum concentration. Results are shown for one input and one output, both for activators and repressors. The dashed black line shows equality. (b)–(d) Comparing two formulations of the optimization problem for activators (black lines) and repressors (gray lines) calculated with a finite dynamic range ($c_{\text{max}}$—circles and solid lines) and constrained means (crosses and dashed lines). The panels show the relative information in panel (b), the optimal value of $K$ in panel (c), and the optimal value of the Hill coefficient in panel (d). In panel (c), approximate results for $K$ are shown as a function of $⟨c⟩$ from Eqs. (56, 58).

Figure 6

(Color online) Optimal input/output relations for the case of five independent target genes activated by the TF at concentration $c$. Successive panels [(a)–(e)] correspond to different values of the maximal input concentration as indicated $\left(C=0.3,1,3,5,10\right)$. Panel (f) summarizes the optimal values of the $K_{\text{i}}$ as a function of $C=c_{\text{max}}/c_0$: as $C$ is increased, the $K_{\text{i}}$ of the fully redundant input/output relations for $C=0.3$ bifurcate such that at $C=10$ the genes tile the whole input range.

Figure 7

(Color online) The case of two target genes. The maps show contour plots of relative information $\left({\text{log}}_2{\tilde{Z}}_1\right)$ as a function of the $K$ values of the two genes: $K_1$ and $K_2$. In each map, the upper right quadrant (A-A) contains solutions where both genes are activated by a common TF, in the lower left quadrant (R-R) both genes are repressed, and the other two quadrants (A-R) contain an activator-repressor mix. The maximal concentration of the input molecules is fixed at $c_{\text{max}}/c_0=0.1$ in map (a), at 0.5 in map (b), and at 1 in map (c). We see that, for example, only at the highest value of $c_{\text{max}}$ does the two-activator solution in the upper right quadrant correspond to distinct values of $K_1$ and $K_2$; at lower values of $c_{\text{max}}$ the optimum is along the “redundant” line $K_1=K_2$. The redundancy is lifted at lower values of $c_{\text{max}}$ in the case of repressors, as we see in the lower left quadrants, and the mixed activator/repressor solutions are always asymmetric. At large $c_{\text{max}}$ we also see that there are two distinct mixed solutions.

Figure 8

The relative information for stable solutions for two genes as a function of $c_{\text{max}}$ [panel (a)]. The inset shows the difference in information transmission for two activators and the mixed case relative to the two repressors. In panel (b), the optimal $K_1$ and $K_2$ are plotted as a function of $c_{\text{max}}$ for two activators (squares) and two repressors (circles). The bifurcation in $K$ is a continuous transition that happens at lower $c_{\text{max}}$ in the case of two repressors.

Figure 9

The relative information for different values of $c_{\text{max}}$ as a function of the number of genes, $M$, shown in panel (a). At low $c_{\text{max}}$ the genes are redundant and so the capacity grows as $\left(1/2\right){\text{log}}_{2}M$; at high $c_{\text{max}}$, the increase in capacity is larger but bounded from above by 1 bit. The differences in information for various combinations of activators and repressors are comparable to the size of the plot symbols. In panel (b), the relative information for different numbers of genes as a function of $c_{\text{max}}$. At higher $M$, the system can make better use of the input dynamic range.

Figure 10

(Color online) The optimal probability distribution of inputs, $P_{TF}^{\ast }\left(c\right)$. In red (dotted line), plotted for $C=0.3$. In blue (solid line), plotted for $C=30$. Different lines correspond to solutions with $2,3,\dots ,9$ genes. At low $C$ (red dotted line), the genes are degenerate and the input distribution is independent of the number of genes. At high $C$ (blue solid line), where the genes tile the concentration range, the optimal input distribution approaches ${\left(c/c_0\right)}^{-1/2}$ (dashed line) as the number of target genes increases.

Figure 11

(Color online) Parameter variations away from the optimum. Results are shown for a two gene system focusing on the solution with two activators. (a) Analysis of the Hessian matrix for $c_{\text{max}}/c_0=0.3$, where the two genes are redundant. Top four panels show the variation in information ($\delta I$ in bits) along each dimension of the parameter space (thick red line) and the quadratic approximation. (b) As in (a), but with $c_{\text{max}}/c_0=10$, where the optimal solution is nonredundant. We also show the eigenvectors and eigenvalues of the Hessian matrix. (c) Distribution of information loss $\Delta I$ when the parameters $K_1$ and $K_2$ are chosen at random from a uniform distribution in $\text{ln}K$, with widths as shown; here $c_{\text{max}}/c_0=10$. (d) As in (c), but for variations in the Hill coefficients $n_1$ and $n_2$.