I define a ``population code'' or ``distributed representation'' to be a set of units for which the pattern of activity indicates the value of an underlying coded variable. This is in contrast to an ``analog representation'' in which the underlying variable is represented directly by the firing rates of one or more cells. The difference is illustrated in figure 1 (see [van Gelder 1991] for discussion). Perhaps the most common computational example of a distributed representation is the binary number system, in which a pattern of 1's and 0's represents an approximation to an analog value. In this paper I will consider a particular form of distributed representation in which each unit is active for only a small contiguous range of input values. A radial basis function representation with local basis functions has this form, as does the unary (but not the binary) number system. I will refer to such representations as having ``local receptive fields''.
Figure: a. ``Analog'', and b. ``Population'' coding of the number 3.
Population codes have been proposed as a model for information representation in certain regions of primate cortex [Georgopoulos et al. 1988,Andersen and Zipser 1988]. This paper seeks to provide the beginnings of a theoretical foundation for understanding such representations and their computational properties. In particular, I attempt to answer the following three questions: