Visual Processing of Motion Information
Using insects, we can discover how visual systems extract local cues or features and then assemble these to detect coherent and meaningful patterns. Flying insects use the patterns of optic flow induced by self-motion to control flight, much as we use the patterns of flow projected on a video screen to control a plane that we pilot on a flight simulator or video game. The patterns are coded by large, motion sensitive neurons that can be anatomically identified by dye-injection and recorded from for extended periods. These neurons and the motion detection pathway are among the best described and understood of all visual pathways.
In such a well established system we can determine the ways in which neurons and neural circuits have evolved to operate effectively under different conditions. Does adaptation promote the efficiency with which higher order "pattern processing" circuits operate? We are investigating several aspects of adaptation in motion detecting networks. We combine electrophysiological recording methods with computer modelling, to establish what neurons do and why they do it.
Specific projects on insect motion processing:
Analysis of optical flow by HS neurons in the fly
Straw,
AD, O'Carroll, DC
A small set of 6 'HS' neurons within the 3rd optic ganglion
of dipteran flies mediate turning responses to wide-field rotational
motion (yaw). These neurons are unusual in that they do not fire
action potentials, but rather signal the direction and speed of
moving patterns via graded voltage responses. This transmission
mode requires a large axon diameter (ca. 6 micrometers), permitting
in-vivo intracellular physiological recordings from the axon.
This physiological preparation offers similar advantages (e.g.
accessibility, stability) to in-vitro brain slice preparations
frequently used to study network behaviour of neurons in mammals,
but with the additional advantage of intact sensory input. We
presently use computer-generated stimulation of HS neurons to
address two basic questions in visual processing: (1) How are
local properties of wide-field movement detectors 'tuned' in both
space and time to optimise detection of moving patterns during
flight? (2) How do adaptive
properties of local motion detectors aid the analysis and coding
of pattern speed? We have discovered that motion detectors located
at different parts of the receptive field of HS neurons use different
delay mechanisms to compensate for local differences in spatial
acuity and thus tune HS neurons to a globally optimal speed. Motion
adaptation is highly complex,
but a model for its role in producing robust responses to variable
patterns is gradually emerging from our work. We hope that this
project will lead to a robust model for analysis of complex optical
flow by 'tuned' filters. In the course of this research, Andrew
Straw has developed an extensive library of software for generating
moving patterns on modern computer graphics cards at high enough
frame-rates for insect vision research. Andrew has made this software
freely available for download, in the form of the VisionEgg
project.
Modelling biomimetic algorithms for velocity discrimination
in motion of natural scenes.
O'Carroll, DC, Rajesh,
S, Shoemaker, PA
After 30 years of physiological research, the visual processing
pathway mediating wide-field motion detection in insects is among
the best studied of all neural pathways. We are using knowledge
acquired about the key stages of motion analysis, in combination
with our recent studies of adaptive properties of insect motion
detectors, to develop and model 'biomimetic' algorithms based
on insect vision. The assumption is that 170 million years of
evolution have optimised efficiency in ways that may not be inherently
obvious if we take a 'bottom up' approach to the problem. Thus
we may be well served by 'reverse engineering' the relatively
simple brain of the fly. The aim is not to model specific biological
processes in detail, but rather to derive inspiration from the
neurobiological system to seek simple solutions to tasks that
have posed major challenges to traditional engineering approaches.
Our present models have two primary applications. Firstly, the
models can be run on a digital computer and used to predict behaviour
of the biological system under unique conditions and so propose
further experiments to test hypotheses about the operations that
take place. Secondly, we can take the key elements of the model
and implement them in hardware. In collaboration with Dr P.A.
Shoemaker at Tanner Research Inc., Pasadena, California, we are
presently developing algorithms for incorporation into analog
VLSI hardware based on local adaptive properties of insect motion
detectors. We aim to develop motion-processing chips with applications
in the area of flight control for autonomous aerial vehicles and
passive motion detection for surveillance. Analog VLSI has very
low energy consumption compared with digital computer technology,
so that the potential cost and size requirements of control systems
based on insect vision may be very modest, and suitable for adding
low-cost embedded control elements to a variety of vehicle types,
from miniature unmanned vehicles to collision avoidance detectors
that can be embedded into the bumper bars of future cars.
Neurophysiology and modelling of insect neurons involved
in target detection.
DuBois, RA, O'Carroll, Shoemaker, PA
The insect brain contains neural pathways for discrimination
of the movement of small targets and features. In some cases,
the physiology of these neurons is spectacular, with a selectivity
for small targets that rivals that of so-called 'hypercomplex'
cells in the mammalian cortex. Neurobiological research on these
small-target movement detector (STMD) neurons, and the basis for
their response selectivity remains in its infancy, however, compared
with the better-studied pathway for detection of wide-field motion.
This project aims to redress the deficiency in our understanding
of this system with a cross-disciplinary approach. We are combining
intracellular electrophysiological recording techniques from novel
neurons and dye-injection to reveal their neuroanatomy, with simulations
run on a digital computer to deduce models that explain the selectivity
of STMD neurons. We aim to develop biomimetic silicon circuits
in analog VLSI that capture similar behaviour.
Adaptation in the visual processing of motion information:
Ongoing work is aimed at refining our observation of tuning of motion detector responses to visual ecology (see above). In a related project, we are investigating the ways in which dynamic adaptation of the motion pathway might help 're-tune' the system to different pattern speeds. Adaptation in both this dynamic sense and as an evolutionary phenomenon, may represent fundamentally similar processes operating on different time scales, optimizing the performance of the visual system for the demands imposed by behaviour (see abstract).
Motion detection at low light levels:
We are also interested in how the light level at which an animal is active influences the properties of motion detectors. There are several different strategies which might tune a motion detector to similar image speeds, but these might operate most efficiently at high or low light levels. In collaboration with Jamie Theobald & Prof. Tom Daniel at University of Washington, and Eric Warrant and Dan-Eric Nilsson at the University of Lund in Sweden, we are investigating the ways in which optical eye design of diurnal versus nocturnal insects (particularly moths) and these neural properties combine to 'optimize' vision at different light levels.
The visual ecology of hoverfly territoriality: It is clear that not only the gross features of behaviour might affect response tuning (e.g. fast versus slow flight), but also subtle aspects of the way the animal interacts with its habitat. For example, the apparent speed of nearby objects may be much higher than more distant ones, due to the effects of 'motion parallax' . Thus an insect which moves slowly through thick undergrowth might experience similar or higher image speeds to one which flies rapidly but a long way above the ground. We are attempting to describe the structure of the world surrounding territorial hoverflies in a more detailed way, using video and image analysis techniques. In addition to addressing basic questions about whether the animal makes the job of detecting motion easier by selecting a micro-habitat to behave in, this approach will provide important baseline information which can be used in realistic computer simulations of motion detectors.
This image is a panoramic view of the world surrounding a male hoverfly (Episyrphus balteatus). The image was taken by placing a CCD camera at the position where a male hoverfly had been observed hovering for several minutes previously. The site was on Coe Fen in Cambridge, adjacent to the river Cam, which is visible towards the right hand side of this image.
NEW !! To see a Quicktime VR panorama of a hoverfly territory: click here (2.0 Mb, 5-10 minute download by modem, requires Quicktime plugin)
The cost of seeing: Finally,
we have been keeping a close eye on the price
paid for high visual performance. Our comparative work has selected
many species which excel at visual behaviour. These include species
with among the largest of all insect eyes. We are trying to estimate
how much of the energy expended in flight is spent lifting and
carrying the large mass of these eyes (see abstract).
