Optimal inference strategies and their implications for the linear noise approximation

We study the information loss of a class of inference strategies that is solely based on time averaging. For an array of independent binary sensors (e.g., receptors, single electron transistors) measuring a weak random signal (e.g., ligand concentration, gate voltage) this information loss is up to 0.5  bit per measurement irrespective of the number of sensors. We derive a condition related to the local detailed balance relation that determines whether or not such a loss of information occurs. Specifically, if the free-energy difference arising from the signal is symmetrically distributed among the forward and backward rates, time integration mechanisms will capture the full information about the signal. As an implication, for the linear noise approximation, we can identify the same loss of information, arising from its inherent simplification of the dynamics.

Figure 1

The two-level system with transition rates is shown on the left. The change $x$ of the free-energy difference between the the states due to the signal is shown on the right.

Figure 2

Full information of trajectory (blue open circle) versus coarse-grained information without knowledge of the number of jumps (red solid line). The results are shown for four different sensory systems, for which the rates are either asymmetrically $\alpha =0$ (a, c) or the symmetrically $\alpha =1/2$ (b, d) influenced by the signal $x$. The upper panel (a, b) shows the single sensor case ($N=1$). The lower panel (c, d) displays the results for a sensor array with $N=100$ binary sensors, where $\mathcal{I}={\mathcal{I}}_N$ and $\tilde{\mathcal{I}}={\tilde{\mathcal{I}}}_N$. $\mathcal{I}$ is obtained numerically by simulating ${10}^6$ individual trajectories to estimate Eq. (3) [or Eq. (8)]. For the coarse-grained mutual information $\tilde{\mathcal{I}}$ we use the approximation from Eq. (12). Parameters: Rates $w_{01}\left(x\right)\equiv e^{(1-\alpha )x}, w_{10}\left(x\right)\equiv e^{-\alpha x}, \alpha =0$ (left panel), $\alpha =1/2$ (right panel), $N=1$ (upper panel), $N=100$ (lower panel), signal standard deviation ${\mathcal{E}}_{\mathrm{x}}\equiv 0.3$ (${\mathcal{E}}_{\mathrm{x}}^2=0.09\ll 1$), $p_0=1/2$, and ${\omega }_{\mathrm{y}}=2$.

Figure 3

Two levels of coarse graining. First, ${\mathit{y}}_t\equiv (y_t^{\left(1\right)},y_t^{\left(2\right)},...,y_t^{\left(N\right)})\to Y_t=Y\left[\mathit{y}\left(t\right)\right]$. Second, a linear noise approximation $Y_t\to {\tilde{Y}}_t$.

Figure 4

Schematic view of marginal distribution $\mathit{m}$ (green dashed lines) and conditional distribution $\mathit{m}|x$ (blue dotted ellipse).

Figure 5

Nonlinear approximation ${\mathcal{I}}^{\infty }$ versus exact result with asymmetric rates ($\alpha =0$). The numerical solution of the exact mutual information $\mathcal{I}$ is indicated as blue open circles. The red solid line indicates the approximation from Eq. (11) of the main text, which is here equivalent to Eq. (B20), and assumes a narrow-width Gaussian distribution of $x$. Parameter: $w_{01}=e^x,w_{10}=1, \alpha =0$. (a) Signal standard deviation ${\mathcal{E}}_{\mathrm{x}}=0.3$ [same as in Fig. 2 in the main text]. (b) ${\mathcal{E}}_{\mathrm{x}}=2$.

Figure 6

Nonlinear approximation ${\mathcal{I}}^{\infty }$ versus exact result with asymmetric rates ($\alpha =0$). The signal is uniformly distributed within $-1\le x\le 1$. Parameter: $w_{01}=e^x,w_{10}=1, \alpha =0$.