Engineering sensorial delay to control phototaxis and emergent collective behaviors

Collective motions emerging from the interaction of autonomous mobile individuals play a key role in many phenomena, from the growth of bacterial colonies to the coordination of robotic swarms. For these collective behaviours to take hold, the individuals must be able to emit, sense and react to signals. When dealing with simple organisms and robots, these signals are necessarily very elementary, e.g. a cell might signal its presence by releasing chemicals and a robot by shining light. An additional challenge arises because the motion of the individuals is often noisy, e.g. the orientation of cells can be altered by Brownian motion and that of robots by an uneven terrain. Therefore, the emphasis is on achieving complex and tunable behaviors from simple autonomous agents communicating with each other in robust ways. Here, we show that the delay between sensing and reacting to a signal can determine the individual and collective long-term behavior of autonomous agents whose motion is intrinsically noisy. We experimentally demonstrate that the collective behaviour of a group of phototactic robots capable of emitting a radially decaying light field can be tuned from segregation to aggregation and clustering by controlling the delay with which they change their propulsion speed in response to the light intensity they measure. We track this transition to the underlying dynamics of this system, in particular, to the ratio between the robots' sensorial delay time and the characteristic time of the robots' random reorientation. Supported by numerics, we discuss how the same mechanism can be applied to control active agents, e.g. airborne drones, moving in a three-dimensional space.


I. INTRODUCTION
The interaction between several simple autonomous agents can give rise to complex collective behaviors. This is observed at all scales, from the organization of bacterial colonies [1,2] and the foraging of ants and bees [3] to the assembly of schools of fish [4] and the collective motion of human crowds [5]. Inspired by these natural systems, the same principles have been applied to engineer autonomous robots capable of performing tasks such as search-and-rescue in disaster zones, surveillance of hazardous areas and targeted object delivery in complex environments [6][7][8][9][10][11].
Complex behaviors can emerge even if each agent follows very simple rules, senses only its immediate surroundings and directly interacts only with nearby agents, without having any knowledge of an overall plan [12,13]. For example, while performing their swim-and-tumble motion, chemotactic bacteria are able to climb a chemotactic gradient, e.g. in order to move towards regions rich in nutrients, by simply adjusting their tumbling rate depending on the chemical concentration they sense [2,14]. Furthermore, by releasing chemoattractant molecules into their surroundings, they are capable of generating a chemical gradient around themselves to which other cells can respond, e.g. in order to create bacterial colonies [1]. Similarly, simple mechanisms are at work in the organization of flocks of birds, schools of fish, and herds of mammals, whereby complex collective behaviors result from each animal reacting to signals sent by its neighbors. A similar approach has also been fruitfully explored in order to build artificial systems with robust behaviors arising from interactions between very simple constituent agents [6,10,11,[15][16][17][18]. Complex behaviors emerging from agents obeying simple rules have the advantage of being extremely robust: for example, even if one or more agents are destroyed, the others can continue to work together to complete the task at hand; agents can also be removed or added mid-task without significantly affecting the final result.
Here, we experimentally and theoretically demonstrate that it is possible to engineer the individual and collective behavior of autonomous agents whose motion is intrinsically noisy by making use of the delay in their sensorial feedback cycle. That is, we show how the delay between the time when an agent senses a signal and the time when it reacts to it can be used as a new parameter for the engineering of large-scale organization of autonomous agents. This proposal is inspired by the motion of chemotactic cells, which are able to climb a chemical gradient by adjusting a different parameter, i.e. their tumbling rate, in response to the concentration of molecules in their surroundings. We demonstrate that the collective behavior of a group of phototactic robots, capable of emitting a radially decaying light field, can be tuned from segregation to aggregation and clustering by controlling the delay with which they adjust their propulsion speed to the light intensity. More precisely, we show that this transition occurs as the ratio between the robots' senso-arXiv:1511.04528v1 [physics.bio-ph] 14 Nov 2015 rial delay time and characteristic time of their random reorientation crosses a certain critical value.

II. SINGLE AGENT
We start by considering a single autonomous agent that moves in a plane and whose orientation is subject to noise. This happens naturally in the case of microswimmers -microscopic particles capable of selfpropulsion such as motile bacteria and cells [19,20] as the direction of their motion changes randomly over time because of the presence of rotational Brownian motion [2]. Similarly, autonomous robots, animals, and even humans can undergo a random reorientation when moving in the absence of external reference points (a striking example of this is an experiment where blindfolded people who were asked to walk in a straight line spontaneously moved along bent trajectories [21]). Such motion is known as active Brownian motion and can be modelled by the following system of stochastic differential equations [12,20,22,23]: where (x t , y t ) is the position of the agent in the plane at time t, φ t is its orientation, v is its speed, τ is the reorientation characteristic time (i.e., the time after which the standard deviation of the agent's rotation is 1 rad), and η t is a white noise driving the agent's reorientation, as shown in Fig. 1a. The reorientation time τ can be associated to an effective reorientation diffusion constant D R = τ −1 , which, in the case of microswimmers, often coincides with the rotational diffusion constant of the particle. Furthermore, we will assume that this agent moves in the presence of an external intensity field to which it reacts by adjusting its speed as a function of the instantaneous intensity it senses. We have realized this experimentally by using a phototactic robot (Elisa-3 [24]) moving within the light gradient generated by a 100 W infrared lamp, which emitted a radially symmetric light intensity radially decaying with a characteristic length R = 35 cm, as shown in Fig. 1b. This robot measures the local light intensity I t = I(x t , y t ) corresponding to its position (x t , y t ) at time t using 8 infrared sensors evenly distributed around its circumference, and adjusts its propulsion speed v (I) accordingly, while randomly changing its orientation with a characteristic reorientation time τ = 1 s. Its motion can be described by modi-fying Eqs. (1) as Fig. 1b shows also a sample trajectory (line) superimposed onto the picture of the robot. The function v(I) is plotted in Fig. 1c; its functional form is where v 0 = 60 cm s −1 is the maximum speed (corresponding to a null intensity), I c = 90 mV is the characteristic intensity scale (measured in volts) over which the velocity decays, and v ∞ = 3 cm s −1 is the residual velocity (in the limit of infinite light intensity). It can be seen in Fig. 1b that the runs between consecutive turns are longer in the low-intensity (high-speed) regions while they are shorter in the high-intensity (low-speed) regions. The result is that over a long period of time, the robot spends more time in the high-intensity regions. As we will see, this behavior is in agreement with our theoretical results given in Eq. (7). This is also in agreement with the behavior of chemotactic cells whose explorative behavior decreases when they reach regions with ideal conditions and reduce their locomotion activity in favor of other metabolic activities [2]. We now proceed to add a delay δ in the agent's response to the measured intensity, which is the main novelty of our work. With this addition, the equations describing the motion of the robot become: The idea of introducing a sensorial delay is inspired by the way in which bacteria react to a chemotactic gradient; in fact, chemotactic bacteria make a comparison of the number of molecules they detect around themselves at consecutive times in order to decide how to adapt their motion [2,14,25]. The presence of sensorial delays is typically ignored, or treated as a nuisance to be controlled [26], while only few theoretical works have considered its possible constructive effects but in situations different from the one studied in this work [27,28]. By introducing a delay long enough so that the robot has enough time to randomize its direction of motion before responding to the sensorial input by changing its speed, we can observe that the motion becomes more directed  towards the high-intensity (low-speed) region, as can be observed by comparing the trajectories in Fig. 2a (with delay δ = +5τ ) to that in Fig. 1b (without delay).
Things become even more interesting if a "negative" delay is introduced, i.e. if a prediction of the future measured intensity is employed to determine the current robot speed. While it is straightforward to see how a positive delay is introduced (e.g. by a delay in the transmission of the signal or by a lapse time before reacting to the signal), the introduction of a negative delay is less intuitive. In fact, a negative delay can be rationalized as a prediction of the future state of the system, which can be done based on the signal received up to the present time. For example, in the case of our robots, a negative delay is introduced by linearizing the light intensity measurement as a function of time and extrapolating it into the future, i.e. I(t − δ) ≈ I(t) − δI (t), where both I(t) and I (t) are known at time t; higher order predictor algorithms are also possible making use of more information about the evolution of the intensity measured up to the present. We show the corresponding trajectory in Fig. 2b, where δ = −5τ . In this case, the robot escapes from the highintensity region and moves towards the edge, where the infrared lamp intensity is lower (and the speed higher).
In order to quantify these observations, we have measured the effective radial drift of the robots, which is calculated [29] as where r is the radial coordinate, r n are samples of the robot's radial position and ∆t is the time step between samples. The results are shown in Figs. 2c and 2d. For positive delay (red circles), the negative drift for large radial distance shows that the robot tends to move to-wards the central high-intensity region. For negative delay (blue diamonds), the positive drift shows that the robot escapes from the central high-intensity region. We have also theoretically calculated the radial drift for an autonomous agent whose motion is governed by Eqs. (4) (see Appendix A), obtaining where v(r) = v(I(r)) and we have assumed a radially symmetric intensity distribution. The solid lines plotted in Figs. 2c and 2d show that there is a good agreement between these theoretical predictions and the experimentally measured data. We have further corroborated these results with numerical simulations, whose results are shown in Fig. S1 in the Supplementary Information and are in good agreement with the experimental results shown in Fig. 2. The numerical simulations were performed by solving the finite difference approximation of Eqs. (4) [20,30]. The delayed sensorial measurement was evaluated by Taylor-expanding the measured intensity about the agent's location and extrapolating the corresponding past/future value. We can also theoretically derive the approximate steady-state probability distribution of the agent's position (see Appendix A), which exists and equals provided that the normalization constant Eq. (6) confirms our initial observations that the larger the positive delay is (solid lines in Fig. 3a), the more time the agent spends in the low-speed (high-intensity) regions. On the other hand, the more negative the delay is (solid lines in Fig. 3b), the more time the agent spends in the high-speed (low-intensity) regions. Interestingly,  7)] moving in a radial intensity field (inset in (a), the agent is confined in a circular well with radius 100 cm indicated by the gray border) as a function of the sensorial delay time: (a) for δ > −τ the agent tends to spend more time in the low-speed (high-intensity) central region; (b) for δ < −τ the agent spends more time in the highspeed (low-intensity) peripheral region; for δ = −τ the probability distribution is uniform (black line). These results are corroborated by numerical simulations of autonomous agents shown by the symbols. For each case, we have simulated a very long trajectory (10 8 s) to obtain an accurate and smooth distribution.
we note that there is a cutoff value at δ = −τ for which the probability distribution of the agent is uniform (black solid line in Fig. 3b). We have further corroborated these results with numerical simulations shown by the symbols in Figs. 3a and 3b.
We emphasize that the qualitative change of the particle's behavior occurs at a negative delay, i.e. δ = −τ . Introduction of negative delays is thus crucial for the described transition. On the other hand, positive delays also influence the system's behavior strongly. While without delay the particle spends more time in slow regions, a positive delay makes this tendency more pronounced, as seen clearly at the quantitative level from Eq. (7). This tendency persists, albeit in a weaker form, for small negative delays −τ < δ < 0 and gets reversed at the critical value δ = −τ .

III. MULTIPLE AGENTS
We can now build on these observations to engineer the large-scale organization of groups of robots. In order to do this, each robot must be able not only to sense the local intensity, but also to create a luminosity field. Thus, we have equipped each robot with 6 LEDs evenly placed around its circumference (EDEI-1LS3), as shown in Fig. 4a, which emit infrared light (wavelength 850 nm) so that each robot generates a decaying light intensity around itself. The LEDs are arranged so that the robot measures only the light intensity emitted by the other robots. A phototactic robot capable of measuring this light intensity will be able to move in the resulting field similarly to the case discussed above, i.e. that of the light intensity generates by a static infrared lamp. We stress that each robot only measures the local intensity without being aware of the positions of the other robots.
We have experimentally studied how three autonomous robots organize by reacting to the cumulative light field created by all of them as a function of their sensorial delay. For a positive sensorial delay (δ = +3τ , Fig. 4b), the three robots gradually move towards each other and form a dynamic cluster, which remains stable over time. A single robot's tendency to spend more time in the highintensity regions when there is positive delay leads to multiple robots forming clusters because of their preference for high-intensity regions. For a negative delay (δ = −3τ , Fig. 4c), the three robots tend to move away from each other, dispersing and exploring a much larger area. In order to understand this behavior in a more quantitative way, we have also simulated a larger number of trajectories for a group of three agents and plotted the average distance between the agents as a function of time for various sensorial delays. The results are reported in Fig. 4d: for positive delays, as the agents tend to come together and form a cluster, their average distance decreases over time; for negative delays, as the agents move apart and explore a larger area, their average distance increases. The qualitative change of the agents' behavior occurs at a strictly negative value of the dimensionless parameter δ/τ = −1 (see Eq. (7)). While introduction of negative delays is thus crucial for the described transition from aggregation to segregation, positive delays also influence the system's behavior strongly by enhancing the tendency of the agents to aggregate. Importantly, not only a light field, but any radially decaying scalar (e.g. chemical, acoustic) field created by the autonomous agents can be used in order to achieve this kind of control over their behavior.
In order to explore the scalability of this mechanism, we have simulated the behavior of an ensemble of 100 robots. Each robot emits around itself a Gaussian intensity field that decays radially, and responds to the locally measured cumulative intensity by adjusting its speed. The long-term behavior and the large-scale organization of these ensembles of agents significantly depend on the sensorial delay, as shown in lay, they move collectively by forming clusters (Figs. 5a and 5b). On the other hand, for negative delays, they move away from each other in order to reduce the intensity each of them measures and are thus able to explore the space more effectively (Figs. 5c and 5d). The possibility of tuning the sensorial delay can be exploited, for example, in a search-and-rescue task by setting initially a negative delay so that the robots can thoroughly explore the environment and, at a later stage, a positive delay so that the robots can be collected into clusters to share the gathered information. Collecting all robots can also be easily achieved by sending a strong signal capable of eclipsing the signals emitted by the robots themselves. It is also possible to adjust the behavior of the agents by altering the intensity-speed relation to something different than Eq. (3). For example, instead of a monotonically decreasing relation, it is possible to use a relation with a minimum at some specific value. As can be seen in Fig. S2 in the Supplementary Information, this alters the agent's behavior so that it spends more time where the intensity corresponds to the minimum speed. In this way, it is possible to control where the agent will spend most of its time, which may be useful, e.g., for targeted delivery. Furthermore, in the presence of multiple agents capable of emitting a radially decaying intensity field, changing the intensity-speed relation permits one to control various features of the clusters such as their characteristic size, as shown in Fig. S3 in the Supplementary Information.
Our results can also be extended to the threedimensional case, where they still hold with only minor adjustments. This could be important when considering airborne objects (e.g. drones, flying insects, birds) or underwater objects (e.g. fish, submarine robots). In three dimensions, the autonomous agent motion can be modelled by the set of equations where (x t , y t , z t ) is the position of the agent at time t, θ t and φ t are its azimuthal and polar orientations respectively, and η (1) t and η (2) t are independent white noises. Similar equations but without delay have already been considered, e.g. in Ref. 31 to describe active Brownian motion in three dimensions. The last two equations describe (accelerated) Brownian motion on the surface of the unit sphere (see Supplementary Information). From this model we obtain the approximate steady-state probability distribution (see Appendix A and Supplementary Information), which exists and equals ρ 0 (x, y, z) = 1 provided that the normalization constant M = v(x, y, z) −(1+2 δ τ ) dx dy dz < ∞.
Comparing Eq. (9) and Eq. (7), we note that the main difference is that in the three-dimensional case the uniform distribution occurs for δ = −0.5τ instead of for δ = −τ . Otherwise, the agents still exhibit a qualitatively different behavior for positive and negative sensorial delay, corresponding, respectively, to an effective drift towards high-intensity and low-intensity regions, as illustrated in Figs. S4 and S5 in the Supplementary Information. As in the two-dimensional case, also in the three-dimensional case it is possible to engineer this drift by changing the time delay in order to tune the collective behavior of a swarm from aggregation and clustering to segregation.

IV. CONCLUSION
We have demonstrated the use of delayed sensorial feedback to control the organization of an ensemble of autonomous agents. We realized this model experimentally by using autonomous robots, further backed it up with simulations, and finally provided a mathematical analysis which agrees with the results obtained in the experiments and simulations. Our findings show that a single robot, measuring the intensity locally, spends more time in either a high or a low-intensity region depending on its sensorial delay. Tuning the value of the delay permits one to engineer the behavior of an ensemble of robots so that they come together or separate from each other. The robustness and flexibility of these behaviors are very promising for applications in the field of swarm robotics [6,10,11,16,18] as well as in the assembly of nanorobots, e.g., for targeted delivery within tissues. Furthermore, since some living entities, such as bacteria, are known to respond to temporal evolution of stimuli [2,25], the presence of a sensorial delay could also explain the swarming behavior of groups of living organisms.