Projects:2018s1-107 Evolution of Spiking Neural Networks for UAV Control
Lachlan Bateman, Paul Elson, Matthew Astachnowicz and Alexander Makarowsky
Dr Hong Gunn Chew, Dr Braden Phillips and sponsor Dr David Howard (CSIRO)
Every unmanned aerial vehicle (UAV) contains a flight controller, which calculate roll, pitch, yaw and thrust signals from the UAV's current position and its desired position. A flight controller is often used by higher-level control structures in autonomous behaviours such as path planning and object avoidance.
This project investigates the use of a biologically-inspired computational model known as a spiking neural network (SNN) to perform UAV flight control. Like conventional artificial neural networks, an SNN consists of a network of small processing nodes called biological neurons. The key difference between spiking and conventional artificial neural networks lies in the inter-neuron signalling: spiking neurons communicate through precisely timed binary 'spikes' as opposed to regularly propagated continuous values. The processing power of a spiking neural network is believed to be significantly greater than that of conventional artificial neural networks when comparing networks of similar size .
Improving the performance of quadcopter flight controllers facilitates UAV applications in an ever-wider range of challenging environments, such as inside mine shafts. As each controller must be adjusted or 'tuned' to a given system, reducing the time required to tune a controller is also of enormous importance to deployment capability. High performance or fast-tuning flight controllers are of enormous military and civilian interest.
Proportional-integral-differential (PID) flight control schemes see widespread use for UAV flight control . While PID-based control schemes are generally easy to tune and implement, they are fundamentally linear control schemes. As quadcopter and hexacopter system dynamics are nonlinear, flight control has been shown to benefit from nonlinear control schemes which can compensate for nonlinearities in the underlying plant .
A nonlinear system of differential equations governs the relationship between a spiking neuron’s internal state, its inputs and its outputs. The nonlinearity of SNNs produces nonlinear functional approximation, and hence potentiates improved flight control performance in comparison to linear PID architectures. One such potential area of improvement is a common tradeoff between disturbance rejection and set-point tracking.
The nonlinearity of SNNs however comes at a cost: conventional artificial neural networks training algorithms cannot be directly applied to optimise the control network. This research uses a family of biologically-inspired search algorithms known as evolutionary algorithms (EAs) to develop high performance controllers.
Position and Speed Control of a DC Motor
Flight control of a quadcopter is a nonlinear, multiple-input multiple-output control problem. To build towards this goal, the position and speed control of a DC motor were adopted as preliminary stepping-stone problems, as they represent highly linear single-input single-output problems.
Spiking neural network controllers were developed both for simulated motors as well as for a physical DC motor. As both position and speed control are linear problems, it is possible to easily compare spiking neural network control performance to a variety of conventional techniques, such as optimal feedback controllers. The DC motor control problems also provide a great deal of insight into the level of accuracy required in simulation before tuned controllers can be readily transferred to hardware.
Angular Control of a 1D Propeller Arm
To progress from a linear single-input single-output (SISO) problem to a nonlinear SISO problem, a one-dimensional propeller arm was constructed. The one-dimensional arm resembles one component of a full UAV flight controller, namely the component which controls a UAV's height. The one-dimensional arm is also a highly nonlinear, as the dynamics of the plant change substantially with the angle of the arm.
The system was modelled in software to provide an easily-accessible platform for evolution experiments. The software model was improved iteratively until the solutions evolved in software were sufficiently transferable to the hardware system.
For both the motor control and 1D arm control experiments, a significant quantity of development work was required to integrate sensors, actuators and log data between experiments and controller evaluations. A Zynq combined FPGA/microprocessor was used throughout the project to perform such tasks. The Zynq also also facilitated the parallelisation of controller evaluations in software, which significantly reduced the time required to evolve high-performing controllers.
The timing precision of at which any SNN was simulated as well as any underlying fixed-point precision were found to both greatly affect controller performance. Understanding how each of these parameters affected the signal precision in terms of rounding proved critical to evolving high-performing controllers.
Early SNN control experiments on both physical and simulated DC motors led to performance that was quite poor when compared to conventional control techniques, such as well-tuned PID tuned or optimal control approaches. An example of an early manually-tuned speed control network is shown below.
The angle of the 1D arm in response to a 30 degree change in setpoint both under PID control and when controlled by a small-scale spiking neural network is shown below. The performance of the SNN controller was found to depend substantially upon the simulation frequency and the level of fixed-point precision used in the embedded platform.
A system diagram of one of high-performance control network topologies is shown below.
To the best of our knowledge, closed-loop control of the physical 1D arm represents the first time that a research group has successfully applied spiking neural networks to an unstable physical plant outside of simulation.
For control applications, SNNs much be simulated with a very high degree of timing precision to produce acceptable performance in terms of closed-loop stability and classical control metrics such as steady-state error. High simulation frequencies in turn necessitate the use of dedicated hardware, such as FPGAs, to produce the signal precision required to produce high-quality control responses.
Evolutionary algorithms have so far proven incapable of generating small-scale SNNs (<10 neurons) which are capable of controlling unstable systems, such as the 1D arm, which when provided solely with the control variable error. Providing small-scale SNNs with explicitly calculated error derivative terms does however result result in system stabilisation and effective plant control. Several recent research publications have also required a derivative term to stabilise simulated systems.
Future Research Opportunities
The next stage for this research is the transition from single-input single-output (SISO) control problems to larger scale multiple-input multiple-output control problems, such as the flight control of a quadcopter. The size of the control network required will likely be significantly larger than that of the SISO controllers.
Flight control will likely be first explored in simulation, on a suitable platform such as QrSim. Transitioning to the SNN control of a UAV in hardware will represent a significantly larger challenge, as there are several aspects of the full control system, such as gyroscopes for state estimation, which will require substantial effort to integrate into a full system.
- W. Maass, “Networks of spiking neurons: the third generation of neural network models” in Neural networks, Vol 10, no 9, 1997, pp.1659-1671
- A. Zulu and S. John, “A review of control algorithms for autonomous quadrotors”, in ArXiv Computing Research Repository - Systems and Control, 2016, submission 02622
- K. Nemirsky, Simulated Annealing-based Optimal Proportional-Integral-Derivative (PID) Controller Design: A Case Study on Nonlinear Quadcopter Dynamics, San Jose State University, 2017
- L. Sanchez, et al, “Nonlinear and optimal real-time control of a rotary-wing UAV”. In American Control Conference (ACC), June 27-29, 2012, Montreal, Canada, 2012, pp. 3857-3862
- Y, Li, K. Ang, G Chong, “PID control system analysis and design” in IEEE Control Systems, Vol 26, no. 1,, 2006, pp.32-41.