\subsection*{Oscillatory Networks}

\subsubsection*{Selective Amplfication}

Two factors control the amount of selective amplification that is
viable in networks such as those in
figure~\ref{B3:nonlinAmp}~and~\ref{B3:orientFig}. The most important
constraint on the recurrent weights is that the network should be
stable, {\it ie\/} the activity should not increase without bound. A
more subtle computational requirement is suggested by
figure~\ref{B3:sus}B. Although the {\it input\/} to the network is
flat, the {\it output\/} shows a well-tuned bump. The creation and
support of tuned output in the absence of tuned input, {\it ie\/}
hallucination or explicit symmetry breaking, is harmful from the
perspective of many computational operations. Avoiding this in the
network requires small recurrent weights, and therefore little
selective amplification.

This restriction can be significantly eased using the richer dynamical
behaviors available from networks with explicit inhibition, {\it ie\/}
networks in which the inhibitory cells have non-trivial time constants
as in equations~\ref{eq:excitpair} and \ref{eq:inhibpair} (Li \&
Dayan, 1999). For instance, figure~\ref{fig:ei} shows an example based
on the network of Ben-Yishai {\it et al\/} (1995) for which the
network {\it oscillates\/} in response to either a flat or a tuned
input (shown in figure~\ref{fig:ei}A).  Figures~\ref{fig:ei}B;D show
that, for the flat input, all the units have the same amplitude;
whereas figures~\ref{fig:ei}B;C show that, for the oriented input,
their amplitudes are appropriately tuned.  Figures~\ref{fig:ei}E;F
show that, in response to the same flat input, if the inhibitory time
constant $\tau_{\Imy}$ in equation~\ref{eq:inhibpair} is set to $0ms$,
then the network breaks symmetry. Avoiding this requires reducing the
recurrent weights and the selective amplification of the network.

\begin{figure}
\setlength{\unitlength}{.1in}
\begin{picture}(45,9)(0,0)
\put(0,0) {\psfig{figure=\home\here/figs/ei/inp.eps,width=1in}}
\put(10,0) {\psfig{figure=\home\here/figs/ei/outp.eps,width=1in}}
\put(20,0) {\psfig{figure=\home\here/figs/ei/respo.eps,width=1in}}
\put(30.5,0) {\psfig{figure=\home\here/figs/ei/respno.eps,width=1in}}
\put(32.25,3) {\psfig{figure=\home\here/figs/ei/finalresp.eps,width=0.625in}}
\put(38.75,3) {\psfig{figure=\home\here/figs/ei/timeresps.eps,width=0.625in}}
\put(0.5,9) {\sf  A}
\put(10.5,9) {\sf  B}
\put(20.5,9) {\sf  C}
\put(30.5,9) {\sf  D}
\put(32.5,9) {\sf  E}
\put(39.25,9) {\sf  F}
\end{picture}
\caption{{\small Selective amplification in an excitatory-inhibitory
    network. A) Noisy tuned (solid and dotted) and untuned (dashed)
    inputs to the network. B) Average response of the network to these
    inputs. Symbols 'o' and 'x' mark $0^\circ$ and $-37^\circ$ for use
    in (C). C) Activities over time of the units at $0^\circ$ (solid)
    and $-37^\circ$ (dashed) in response to the most strongly
    modulated input from (A). D) Activities of the same units to the
    flat input in B (the lines lie on top of each other showing that
    the network oscillates but fails to break symmetry.  E) Response
    of a reduced network (with $\tau_I=0ms$) to the same flat input.
    Symbols 'o' and 'x' mark $42^\circ$ and $6^\circ$ for use in (F).
    F) Activities over time of these units in the reduced network.
    The network is as in
    equations~\ref{eq:excitpair}~and~\ref{eq:inhibpair}~and~\ref{B3:orient},
    with $\tau_{\s}=\tau_{\Imy}=10ms$, $\hh^-=\zzz$. The excitatory
    weights are given by equation~\ref{B3:bfunc} with
    $B^+(\theta-\theta')=(1/\pi\rho_\theta)[5.9+7.8\cos(2(\theta-\theta'))]_+$,
    and the inhibitory weights by
    $B^-(\theta-\theta')=(1/\pi\rho_\theta)13.3$. After Li \& Dayan
    (1999).
\label{fig:ei}}}
\end{figure}

This can be understood by observing that for the reduced network
with symmetric, positive and negative connections, the only way to
avoid breaking symmetry is for the solution with $v(\theta)$ constant
to be linearly {\it stable\/} in response to flat input $h(\theta)$.
This forces the recurrent weights, and therefore the degree of
selective amplification to be small.  However, the network with
explicit inhibitory cells, and therefore asymmetric connections, can
avoid breaking symmetry while still having the solution with
$v(\theta)$ constant be linearly {\it unstable\/} in response to flat
input $h(\theta)$, provided that the instability leads $v(\theta)$ on
a limit cycle on which all the activities are the same (as in
figure~\ref{fig:ei}D).  This allows larger recurrent weights, and
therefore more selective amplification.

\subsubsection*{The Olfactory Bulb}

A simple example of an excitatory-inhibitory network with less well
ordered connectivity is the oscillatory model of the olfactory bulb
(Li \& Hopfield, 1989; Li, 1995). The analysis of this network makes
critical use of the phase-plane methods discussed at the end of
chapter~6.  Figure~\ref{fig:olf}A shows a network diagram of the first
stages of olfactory processing in vertebrates (although many elements
of the circuit are substantially conserved across invertebrates also).
Odor molecules are sensed by olfactory receptor neurons. These send
their axons to the olfactory bulb, terminating in glomeruli, where
they synapse onto mitral cells (and also onto local interneurons which
are not shown). The mitral cells form the excitatory population, and
are the projection cells from the bulb to primary olfactory cortex.
The mitral cells also synapse onto the inhibitory population of
granule cells (which are much more numerous).  The granule cells in
turn inhibit the mitral cells. Inputs from olfactory cortex to the
bulb synapse onto the granule cells and not the mitral cells,
suggesting an inhibitory top down control mechanism.  Although the
receptor neurons wire up in a beautifully precise way to glomeruli,
and although the connections between mitral and granule cells are
local, there is no obvious role for the topography in the connections.

