Heteroclinic channels

Heteroclinic channels are ensembles of trajectories that can connect saddle equilibrium points in phase space. Dynamical systems and their associated phase spaces can be used to describe natural phenomena in mathematical terms; heteroclinic channels, and the cycles (or orbits) that they produce, are features in phase space that can be designed to occupy specific locations in that space. Heteroclinic channels move trajectories from one equilibrium point to another. More formally, a heteroclinic channel is a region in phase space in which nearby trajectories are drawn closer and closer to one unique limiting trajectory, the heteroclinic orbit. Equilibria connected by heteroclinic trajectories form heteroclinic cycles and cycles can be connected to form heteroclinic networks. Heteroclinic cycles and networks naturally appear in a number of applications, such as fluid dynamics, population dynamics, and neural dynamics. In addition, dynamical systems are often used as methods for robotic control. In particular, for robotic control, the equilibrium points can correspond to robotic states, and the heteroclinic channels can provide smooth methods for switching from state to state.

Four saddle equilibrium points (gray) connected by heteroclinic channels (red).

Overview

Heteroclinic channels (or heteroclinic orbits) are building blocks for a subset of dynamical systems that are built around connected saddle equilibrium points. Homoclinic channels/orbits join a single equilibrium point to itself, whereas heteroclinic channels join two different saddle equilibrium points in phase space. The connection is formed from the unstable manifold of the first saddle (“pushing away” from that point) to the stable manifold of the next saddle point (“pulling towards” this point). Combining at least three saddle equilibria in this way produces a heteroclinic cycle, and multiple heteroclinic cycles can be connected into heteroclinic networks.

Heteroclinic channels have both spatial and temporal features in phase space. Spatial because they affect trajectories within a certain region around themselves, and temporal because the parameters of a heteroclinic channel affect how much time a trajectory spends along that channel (or more specifically, how much time it spends around one of the saddle points). The transient nature of heteroclinic channels is important for describing their “switching” nature. That is, some neighborhood around each equilibrium point can be defined as a separate state, and the heteroclinic channel itself presents a method of switching sequentially between these states.

Heteroclinic "switching" is an important descriptor for natural phenomena, especially in neural dynamics. It has also been used as an approach for designing robotic control methods which cycle between states, whether those states are pre-defined behaviors or transient states that lead to larger behaviors.

History

The mathematical image described above – a series of states with a functional mechanism for switching between them – also describes a phenomenon known as winnerless competition (WLC). Winnerless competition describes the switching phenomenon between two competitive states and was identified by Busse & Heikes in 1980 when they were investigating the change of phases in a convection cycle. However, the transient dynamics of WLC are widely agreed to first have been presented by Alfred J. Lotka, who first developed the concept to describe autocatalytic chemical reactions in 1910 and then developed an extended version in 1925 to describe ecological predator-prey relationships. In 1926, Vito Volterra independently published the same set of equations with a focus on mathematical biology, especially multi-species interactions. These equations, now known as the Lotka-Volterra equations, are widely used as a mathematical model to describe transient heteroclinic switching dynamics.

Heteroclinic cycles which describe the transition between at least three states were first described by May & Leonard in 1975. They identified a special case of the Lotka-Volterra equations for population dynamics . The re-emergence of heteroclinic cycles and the increased ability to do numerical computations as compared to the period of Lotka and Volterra, prompted a resurgence of interest in heteroclinic channels, cycles, and networks as mathematical models for transient sequential dynamics.

Heteroclinic channels have become models for neural dynamics. An example is Laurent et al. (2001) who described the neural responses of fish and insects to olfactory stimuli as a WLC system, where each stimulus and its response could be identified as a separate state within the space. The responses could be modeled in this way because of their spatial and temporal properties, which aligned with the spatiotemporal nature of WLC. Rabinovich et al. (2001) & Afraimovich et al. (2004) used WLC networks (via the Fitzhugh-Nagumo & Lotka-Volterra models, respectively) to connect the mathematical concept of stable heteroclinic channels (SHCs) to transient neural dynamics more generally, particularly other sensory processes and more abstract neural connections. Rabinovich et al.  (2008) expanded this idea to larger cognitive dynamic systems, and large-scale brain networks. Stable heteroclinic channels have also been used to model neuromechanical systems. The feeding structures and associated feeding processes (stages of swallowing) of marine mollusks have been analyzed using heteroclinic channels.

Biological models have always been a source of inspiration for roboticists, especially those interested in robotic control. Since robotic control requires defining and sequencing the physical actions of the robot, models of neural dynamics can be very useful. An example of this can be found in central pattern generators, which are widely used for rhythmic robotic motion. Heteroclinic channels have been used to replicate central pattern generators for robot control. Similarly, dynamic movement primitives, another common robotic motion control system, have been adapted and made more flexible by using heteroclinic channels. In more practical applications, stable heteroclinic channels have been directly used in the control of several biologically-inspired robots

Concepts

Dynamical systems

A dynamical system is a rule or set of rules that describe the evolution of a state (or a system of states) in time. The set of all possible states is called the state space. The phase space is the state space of a continuous system. Dynamical systems describe the state over time with mathematical equations, often ordinary differential equations. The current state at a particular time can be plotted as a point in phase space. The set of points over time can be plotted as a trajectory.

Stability

A heteroclinic channel itself can be asymptotically stable. That is, any point near the vicinity of the channel is attracted to the heteroclinic cycle at the core of the channel. Both heteroclinic channels and cycles can be robust (or structurally stable) if, within a given parameter range, they maintain a given behavior; however, this is not required.

Stochasticity

Noise is one input into a heteroclinic system to move it from one equilibrium to the next. The reason is that noise (or some other stochasticity) disturbs the system enough to move it into the vicinity of the next saddle equilibrium point in the sequence. The amount of noise required is inversely proportional to the “attractiveness” of the saddle points; the more attractive the stable part of the saddle is to the system state, the longer the trajectory will linger in its vicinity, and the more noise will be required to move the system’s state off of that attractive equilibrium point. There are also other ways of moving between the equilibrium points including parametric changes, or using sensory feedback.

Control theory

Control theory, in robotics, deals with the use of dynamical systems to control robotic systems. The goal of robotic control is to perform precise, coordinated actions using physical actuators in response to sensor input. Dynamical systems can be used to drive the robot to a desired state (or set of states) using sensor input to minimize actuator errors.

Mathematical definition

An equilibrium point in a dynamical system is a solution to the system of differential equations describing a trajectory that does not change with time. Equilibrium points can be described by their stability, which are often determined by the eigenvalues of the system’s Jacobian matrix. In general, the eigenvalues of a saddle point have non-zero real parts, at least one of the real parts is positive and at least one of the real parts is negative. Any eigenvalue with a negative real value indicates a stable manifold of the saddle which attracts trajectories, whereas any eigenvalue with a positive real value indicates the unstable manifold of the saddle which repels trajectories.

Phase space definition (from Heteroclinic orbit)

Let ${\textstyle {\dot {x}}=f(x)}$ be the ordinary differential equation describing a continuous dynamical system. If there are equilibria at $x=x_{0}$ and $x=x_{1}$, then a solution $\phi (t)$ is a heteroclinic connection from $x_{0}$ to $x_{1}$ if

$$\phi (t)\rightarrow x_{0}$$

as $t\rightarrow -\infty$

and

$$\phi (t)\rightarrow x_{1}$$

as $t\rightarrow +\infty$

This implies that the connection is contained in the stable manifold of $x_{1}$ and the unstable manifold of $x_{0}$.

Neural dynamics examples

Neural dynamics are the non-linear dynamics that describe neural processes, from single neurons to cognitive processes and large-scale neural systems.

Lotka-Volterra model

This model was first presented independently by Alfred J. Lotka for autocatalytic chemical reactions and then again for biological species in competition by Vito Volterra from a mathematical biology perspective. Originally, this model was only considered for two species: the two chemical species in the reaction, or a predator-prey situation in a shared environment.

The original equations were based on the logistic population equation, which is popularly used in ecology.

$${dx \over dt}=rx{\Bigl (}1-{x \over K}{\Bigr )}$$

where $x$ is the size or concentration of a species at a given time, $r$ is the growth rate and $K$ is the carrying capacity of that species.

Lotka incorporated a term for the interaction between species and, with some generalization, the series of equations can be written as follows:

$${dx_{i}(t) \over dt}=rx_{i}(t){\Biggl [}1-\sum _{j=1}^{N}\alpha _{ij}x_{j}(t){\Biggr ]}$$

In this definition, $x_{i}(t)$ is the size or concentration of the $i$-th species and $N$ is the total number of species. The interaction between each species is described by the matrix $\alpha$.

May-Leonard expansion

May and Leonard expanded the Lotka-Volterra equations by investigating the system in which three species interact with each other (i.e., $N=3$). They found that for a system in which each equilibrium point is a saddle with an $N-1$ dimensional stable manifold, and the unstable manifold connects the points sequentially, the equation above can be re-written as follows:

$${dx_{i}(t) \over dt}=x_{i}(t){\Biggl [}1-\sum _{j=1}^{N}\rho _{ij}x_{j}(t){\Biggr ]}$$

Explicitly for $N=3$, this becomes

${\begin{aligned}{dx_{1} \over dt}&=x_{1}[1-x_{1}-\alpha x_{2}-\beta x_{3}],\\{dx_{2} \over dt}&=x_{2}[1-\beta x_{1}-x_{2}-\alpha x_{3}],\\{dx_{3} \over dt}&=x_{3}[1-\alpha x_{1}-\beta x_{2}-x_{3}],\end{aligned}}$

where the coupling matrix, $\rho$, is given by

$$\rho ={\begin{bmatrix}1&\alpha &\beta \\\beta &1&\alpha \\\alpha &\beta &1\end{bmatrix}}.$$

In this model, the stability of the saddle equilibria can be easily determined. The stability requirements for the formation of a stable heteroclinic cycle are $\alpha +\beta \geq 2$ with either $\alpha >1$ or $\beta >1$.

It was noted in this work that the system never asymptotically reaches any of the equilibrium points, but the amount of time the trajectory spends at each equilibrium point increases with time. In ecological terms, this suggests that a single population would eventually “beat out” the other two. May & Leonard noted that this is not a practical result in biology (and also see).

Winnerless Competition framework

The “Winnerless Competition” framework (suggested by Laurent et al.) allowed a single neuron and/or a collection of synchronized neurons to be encoded between "on" and "off". Laurent et al. investigated olfaction in fish and insects, particularly olfactory reception, and some of the postsynaptic structures in the odor sensory system. They found that the processing (or encoding) of perceived odors occurred over at least three timescales: fast, intermediate, and slow. They posited that an odor encoding system should be reproducible, which requires it to be insensitive to (or rapidly forget) any initial state. This is only possible if the dynamical system is strongly dissipative, that is, it settles on a state quickly and is insensitive to internal noise. Conversely, a useful odor encoding system should be sensitive to small variations in input, which requires the system to be active. An active system uses external sources to allow small variations in initial states to grow with time. The winnerless competition framework allowed a single neuron (or node) to encode a stimulus (the “fast” timescale), or many stimuli could be encoded via stimulus-specific trajectories (the “slow” timescale).

The winnerless competition system was described by

$${dx_{i}(t) \over dt}=x_{i}(t){\Biggl [}1-\sum _{j=1}^{N}\rho _{ij}x_{j}(t){\Biggr ]}+S_{i}^{s}$$

where $x_{i}(t)$ and $x_{j}(t)$ characterize the activities of stimulus-specific groups $i$ and $j$, respectively, $N$ is the number of neurons being simulated, $\rho _{ij}>0$ characterizes the strength of inhibition by $i$ and $j$ (i.e., their interactions with each other), and $S_{i}^{s}(t)$ is the current input by a stimulus $s$ to $i$.

Winnerless competition required that the inhibitory connections in the $\rho$ matrix were asymmetrical and cyclic. For example, for $N=3$, if $\rho _{11},\rho _{22},\rho _{33}=1$ then $\rho _{12},\rho _{23},\rho _{31}>1$, and $\rho _{21},\rho _{32},\rho _{13}<1$.

Overall, this description produces a heteroclinic channel composed of several heteroclinic orbits (trajectories).

Transient neural dynamics

Sensory encoding via heteroclinic orbits (which are facilitated by heteroclinic channels) as described by Laurent et al. was extrapolated beyond the olfactory system. Rabinovich et al. explored winnerless competition as a spatiotemporal dynamical system corresponding to the activity of specific neurons or groups of neurons. They identified the added stimulus as the factor that would drive a trajectory from one node along the channel to the next. Without it, the system would reduce to a steady state in which one neuron (or neuronal group) was active whereas the others were quiescent.

Afraimovich et al. also developed winnerless competition using connected saddle points in phase space as a model for transient, sequential neural activity. They outlined how the saddle points should be defined, the conditions for heteroclinic connections between them and the conditions for heteroclinic sequence stability. They performed numerical simulations of the dynamics of a network with N = 50 neurons and used Gaussian noise as the external input. They found that the movement of a trajectory along each connection was initiated by the noise, and the speed of switching from one saddle to the next depended on the noise level.

Cognitive dynamics

The sequential switching property of stable heteroclinic channels has been expanded to describe higher-level transient cognitive dynamics, particularly sequential decision making. Rabinovich et al. first introduced this idea by applying the sequential switching that characterizes stable heteroclinic channels to the sequential decision making process seen in a fixed time game. The player takes sequential actions in a changing environment to maximize some reward. For a fixed time game, in order to maximize the reward, the player must encounter as many decision states as possible. This means that within a fixed amount of time, the trajectory must pass in the vicinity of as many saddle points, or nodes, as possible. When the trajectory reached the vicinity of a saddle point, a decision-making function was applied.

The reward was maximized by choosing appropriate system parameters. One of these was a decision-making rule that corresponded to the fastest motion away from the saddle, which was the shortest time to reach the next saddle. Additionally, there was an optimal level of additive noise; the noise was high enough that the trajectory could move away from each saddle quickly, but not so high that the trajectory would be directed off the cycle entirely.

A major point of this work was that, without significant external stimulus, the player was likely to find one of two extremes: ending decision-making quickly or reaching a cycle that runs through the entire allotted time. Behaviorally, this cycle translates to habit formation (on a cognitive level) and is sensitive to external stimuli that can change the trajectory’s direction at any time.

Rabinovich & Varona described sequential memory in a similar way. They also introduced “chunking”, which describes how the brain groups sequential information items into chunks at different hierarchical levels. They used stable heteroclinic channels as a framework for building these chunks into high level heteroclinic networks.

Neuromechanical models

Heteroclinic channels have also been used as a model for neuromechanical systems in animals, particularly the feeding structures in marine mollusks. Shaw et al. (2015) investigated potential models for the feeding behavior of Aplysia californica. They found that heteroclinic channels could more accurately match features of actual experimental data than other models such as limit cycles. Lyttle et al. (2017) showed that both the heteroclinic model and the limit cycle model of the Aplysia californica’s feeding system grant different advantages and disadvantages, such as robustness to perturbations and flexibility to inputs. They also showed that a reasonable model of the animal’s behavior could be made by switching between these modes, heteroclinic and limit cycle, using external sensory input, providing a dynamical basis for understanding both robustness and flexibility in motor systems.

Robotic control examples

Mathematical expansions of the framework are required for robotic control applications.

For higher dimensional systems, the connection/inhibition matrix $\rho$ can be generalized as:

$$\rho ={\begin{bmatrix}1&\alpha &\gamma &\gamma &\cdots &\gamma &\beta \\\beta &1&\alpha &\gamma &\gamma &\cdots &\gamma \\\gamma &\beta &1&\alpha &\gamma &\cdots &\gamma \\\gamma &\gamma &\beta &1&\alpha &\cdots &\gamma \\\vdots &\vdots &\vdots &\vdots &\vdots &\ddots \\\gamma &\cdots &&\gamma &\beta &1&\alpha \\\alpha &\gamma &\cdots &\gamma &\gamma &\beta &1\end{bmatrix}}$$

or formulations similar to this.

Appropriate saddle values must be assigned to make the system dissipative. The strength of the saddle can be characterized by its two largest eigenvalues: the single unstable eigenvalue, $\lambda ^{u}$, and the weakest stable eigenvalue, $-\lambda ^{s}$. The saddle value of the $i$-th node can be defined as

$$v_{i}={Re\lambda _{i}^{s} \over \lambda _{i}^{u}}$$

If $v_{i}>1$, the $i$-th node is dissipative and stable, and if $\textstyle \prod _{i=1}^{N}\displaystyle v_{i}>1$ the entire cycle will be stable.

SHCs for biologically inspired robotic control

SHCs have been used directly to control robots, particularly biologically-inspired robotic systems. SHCs have also been used to adapt existing robot control frameworks. In both instances, the special properties of SHCs were used to improve the associated control tasks. Some examples include integrated contact sensing to modulate the additive SHC noise, a combined Gaussian Mixture Model to inform SHC "switching", a central pattern generator which was adapted to be temporally sensitive, and a modified control framework which has an intuitive visualization property.

Contact sensing for additive noise modulation

SHCs make it possible to use sensory feedback for rapid choices in a high degree-of-freedom robotic system. For example, Daltorio et al. used SHCs as a controller for the simulated locomotion of a worm-like robot in a pipe. The robot's structure consisted of 12 actuated body segments, each with one degree of freedom: segment length. Each segment coupled its height to the length such that as the length decreased, the height increased. This structure was used to simulate peristaltic locomotion, as the segments’ actuation was coordinated to form a peristaltic wave down the robot, with each segment contracting one after the other down the robot body.

For this system, each body segment was associated with a saddle point in the SHC system. The multi-dimensional connection matrix was constructed so that each point inhibited its neighbors except the point immediately after it. This asymmetry caused the active SHC node to move “backwards” down the robot structure, while the body moved forward.

The controller was tested in multiple pipe-shaped paths where contact sensors on the robot could provide information on the environment. Contact sensing information was used to modulate the noise added to the system, which in turn allowed the activation sequence to be altered. This was key for highly coordinated movement across all segments.

Gaussian mixture model to inform "switching"

SHCs can be used to inform the switching among complex configurations. Petrič et al. used a combined Gaussian Mixture Model (GMM) and SHC system to control a spinal exoskeleton. The exoskeleton was designed as a quasi-passive system that physically supports the user to different degrees depending on the current pose or movement of the user. Different functional poses/movements were identified as the nodes within the SHC system. GMMs were used to indicate what the additive inputs for each SHC node should be, which would drive the system from one pose to the next.

Temporally sensitive central pattern generator

SHCs have been used as an alternative to central pattern generators for robotic control. Horchler et al. used SHCs to produce an oscillator whose behavior near each node could be manipulated using system parameters: additive noise and saddle values. This produced a cyclic controller that could spend more time at a particular node when needed. The controller's responsiveness to external input was demonstrated by pausing and resetting the cycle using additive noise.

Intuitive visualization property

Rouse & Daltorio replaced the underlying attractor points of dynamic movement primitives, another biologically-inspired robotic control method, with the saddle points of SHCs. This adaptive framework maintained the stability of the system. Additionally, it provided a visualization property which allowed the user to intuitively place saddle points in phase space to match a desired trajectory in the task space.

See also

• Dynamical systems
• Saddle equilibrium points
• Heteroclinic networks
• Heteroclinic orbits
• Lotka-Volterra equations
• Neuromechanical systems
• Control theory
• Biologically-inspired robotic systems
• Limit cycles

Further reading

• J. Lotka, Elements of Physical Biology. Baltimore: Williams and Wilkins, 1925.
• M. Krupa, “Robust Heteroclinic Cycles,” J. Nonlinear Sci, vol. 7, pp. 129–176, 1997.
• M. I. Rabinovich and M. K. Muezzinoglu, “Mutual Emotion-Cognition Dynamics,” Association for the Advancement of Artificial Intelligence, 07-Sep-2009. [Online]. Available: arXiv:0909.1144. [Accessed: 02-Nov-2021].
• Antonio Palacios (2007) Heteroclinic cycles. Scholarpedia, 2(1):2352.
• Eugene M. Izhikevich (2007) Equilibrium. Scholarpedia, 2(10):2014.
• James Meiss (2007) Dynamical systems. Scholarpedia, 2(2):1629.
• Frank Hoppensteadt (2006) Predator-prey model. Scholarpedia, 1(10):1563.
• Randall D. Beer (2009) Biologically inspired robotics. Scholarpedia, 4(4):1531.

External links

• Most viewed “heteroclinic” articles on Frontiers Media.
• A Matlab Toolbox for the Simulation, Analysis and Design of Continuous Dynamical Systems Based on Networks of Stable Heteroclinic Channels (SHCs)
• Code for the simulations and figures from Lyttle et al. (2017)