The visual system is the largest sensory system in the brain. It is sophisticated and complex. The number of visual neurons expands from about a million retinal ganglion cells to about 8 billion cells in the cerebral cortex. fMRI studies reveal two streams of information in the visual cortex, a ventral stream related mostly to "what" an object is, and a dorsal stream related to "where" it is. But even in the retina the visual signal is split up into multiple channels that convey real-time information which would be difficult to compute sequentially from an image in the allotted time.

The human visual field is shown in the figure below, it's about 200 degrees horizontally and about 130 degrees vertically. The fields from each eye overlap in the central 120 degrees or so, and the high resolution foveal area is actually quite small, covering about 5 degrees of horizontal arc.

The human retina emits a little north of a million fibers from its ganglion cells. The ones we're discussing now, travel from the retina through the thalamus and into the occipital lobe of the cerebral cortex. (Others connect with the superior colliculus, which we discuss elsewhere, and other structures like the suprachiasmatic nucleus which is involved in light-related circadian rhythms, and the olivary pretectal nucleus which is involved with light-driven pupil constriction). The figure shows the general plan of the early visual system, specifically the pathway from the retina to the primary visual cortex.

The nerve fibers from each retina cross in the optic chiasm. The nasal hemifield from one is combined with the temporal hemifield from the other, and the result is that the left side of the brain processes the right side of the visual field, and vice versa. The retina "knows" its orientation and its subdivisions early in development, based on chemical gradients. Later in development there is also oscillatory neural activity that helps to self-organize the connections.

The retina is a layered structure, and it is inverted with respect to light - in other words the photoreceptors are on the inside, nearest the brain. Light has to travel through the ganglion cells to get to the photoreceptors. This figure shows the general orientation of the neurons within the retina.

The number of layers in the retina keeps increasing as our experimental methods get better, Ramon y Cajal said 5 but newer results bring the number closer to 10. Some layers contain cells, some layers contain fibers and synapses, and some contain both. The general connection pattern of the retina is photoreceptors => bipolar cells => ganglion cells. Laterally between the photoreceptors and the bipolar cells are the horizontal cells, and laterally between the bipolar cells and the ganglion cells are the amacrine cells. This pattern is schematized in the figure.

There are four main types of photoreceptors, a dozen kinds of bipolar cells, and about 20 different varieties of ganglion cells.

The electrical activity from the retina associated with an optical stimulus is shown in the figure below. Historically it was detected as early as the mid-1860's by placing electrodes on the cornea. This "electroretinogram" is also visible in other places (it can be detected with a special contact lens, and it is also visible using ordinary electrodes on the forehead and temple). The ERG is subject to volume conductor effects from the vitreous humor and other parts of the eye and nearby tissue.

Photoreceptors

The right eye is shown schematically in the figure below. The typical human eye is about 2.4 cm from front to back, it is approximately spherical but slightly wider than it is deep. The retina is in back of the eye, light has to travel through the vitreous humor to get to the photoreceptors. Not only that, but the retina is inverted with respect to light, the photoreceptors are in back along the choroid wall and the nerve fibers from the ganglion cells exit in the front and travel laterally to enter the optic nerve at the blind spot.

Humans have about 126 million photoreceptors, of which 120 million are rods and 6 million are cones. The distribution of photoreceptors in a human retina is shown in the figure.

The retinal photoreceptors hyperpolarize in response to light, they have a baseline membrane potential that goes negative as a result of ion influx. This process is shown in the figures below.

There are four main types of photoreceptors, their characteristics are shown below. The density of small color-sensitive photoreceptors is highest in the foveal region of the eye, while the density of wider field contrastive cells is highest in the periphery. The cones are called S, M, and L for blue, green, and red respectively, based on their preferred wavelength of light. They are arranged in a mosaic pattern within the retina.

The various photoreceptors form a mosaic pattern on the retina. The mosaic is organized in a radial manner, with a very high concentration of cones at the fovea, that tapers off in the periphery. The figure shows the mosaic pattern of color-sensitive cones in a normal retina (left) and a retina with protanopsia (in this case, no red cones). The foveal area is outlined.

Photoreceptors of all kinds synapse on bipolar cells, using glutamate as a neurotransmitter. In addition, they form gap junctions and triadic synapses with horizontal cells, of which there are several different kinds. Generally speaking only one or a few photoreceptors connect to a bipolar cell in the high resolution area of the fovea, whereas the number may be slightly higher in the periphery.

There are gap junctions between rods, and between rods and cones. These gap junctions are associated with various connexin proteins, that allow ions and small molecules to move between cells. Electrically, the movement of sodium, calcium, potassium, and chloride between cells creates a syncytium, whose properties can be modeled as one large membrane. Some of the gap junctions are rectifying, they only allow movement in one direction.

Bipolar Cells and Horizontal Cells

Photoreceptors feed forward to bipolar cells. There are two kinds of bipolar cells, those that respond to light onset, and those that respond to light offset. These two responses are provided by two different kinds of receptors that process the glutamate (neurotransmitter) released by photoreceptors. The sign-preserving (off-center) synapses use the ionotropic AMPA receptor, whereas the sign-reversing (on-center) synapses use the metabotropic mGlu-R6 receptor.

In addition to the differential response to photoreceptors, the horizontal cells provide an inhibitory surround field to the bipolar cells. The surround is wider than the center, because the horizontal cell dendrites branch more widely than the bipolar cell dendrites. The surround does not depend on the bipolar cells, it is generated by a cooperative process between the photoreceptors and the horizontal cells, as shown in the figure. Horizontal cells typically use GABA as an inhibitory neurotransmitter.

The surround field can be loosely modeled as a "sum of Gaussians", where the middle peak is determined by the viewing field of the photoreceptor and the edges are determined by the extent of the horizontal cell. In real life the situation is a bit more complicated, because of both of these patterns are dynamic, they undergo adaptation and other forms of plasticity that help the eye adjust to changing light conditions. In humans and primates which are trichromatic, there are two classes of horizontal cells, the H1 class that shapes M and L cone output, and the H2 class that shapes S cone output and contributes to S-M/L opponency (Chapot et al 1997).

In humans, the H1 horizontal cells have axons, and the H2 cells do not. The axons from the H1 cells terminate mainly on nearby rods, and can be strongly directional. Gap junctions play an important role in both the dendritic and axonal connections of horizontal cells. There are genetically engineered "knockout" mice missing connexin proteins, they show severe visual deficits, especially in low light.

Though there are only two main types of horizontal cells, there are at least 12 different kinds of bipolar cells in humans. One of them is a specialized rod bipolar cell that only contacts AII amacrine cells (20-80/cell) and does not connect directly with ganglion cells. The others feed forward to retinal ganglion cells.

The inhibitory surrounds of bipolar cells are generated by specialized synapses between the receptors, horizontal cells, and bipolar cells. These surrounds are generated when activity in one photoreceptor activates the connected horizontal cell, which then inhibits nearby receptors. In real life during changing light conditions, the horizontal cell synapses are part of the circuitry related to adaptive gain control (Scholl et al 2012, VanLeeuwen et al 2009). Bipolar cells also participate in adaptive gain control (Beaudoin et al 2007).

The synaptic web between the photoreceptors and horizontal cells sends signals with virtually no delay, there are gap junctions involved (identified by their connexin proteins) with time constants on the order of microseconds, rather than the milliseconds usually associated with synaptic transmission (Trenholm & Awatramani 2025, Gollisch & Meister 2010). Therefore the inhibitory surrounds of bipolar cells activate without much delay, after an optical stimulus.

Bipolar cells excite ganglion cells, they generally use glutamate as a neurotransmitter. The different types of bipolar cells segregate local information into at least 20 independent "channels" that exit the retina at the level of the ganglion cells. Each of these channels is topographically consistent, and the result is an overlay of various "views" of the retinal input. Each view is filtered in a different way, some carry color information, some carry motion information, and some carry general luminance. The channels remain segregated on their way to the visual cortex. This is a great resource for visualizing cells in the retina.

Neither retinal photoreceptors nor horizontal cells generate action potentials. Instead they use graded potentials, and the overall shape of response is mostly sigmoidal with a nearly linear region in the middle. Bipolar cells, on the other hand, can use both graded and action potentials, as can certain kinds of amacrine cells. As far as we know right now, retinal ganglion cells communicate with the LGN exclusively via action potentials.

Ganglion Cells and Amacrine Cells

The bipolar cells send outputs to a bewildering array of amacrine cells and ganglion cells, and the amacrine cells in turn connect back with the ganglion cells, both synaptically and electrically. Over 20 different kinds of retinal ganglion cells have been identified in humans and primates, and many of these obtain their properties (both static and dynamic) from the amacrine cells. Each different combination of ganglion and amacrine cells performs a local filtering operation on the input from bipolar cells. The filter acts in both space and time, as many of the inhibitory effects from amacrine cells are synaptic and have time constants on the order of milliseconds.

The figure shows some of the varieties of amacrine cells. Particular varieties can use GABA, glycine, acetylcholine, dopamine and other neurotransmitters. One of the well studied amacrine cells in the inner nuclear layer of the retina is the A-II cell, which gets inputs from rod bipolars and distributes them to cone bipolars (Kolb, 1997). The figure shows the stratification of some of the 40+ different kinds of retinal amacrine cells. Even though this figure is from a rabbit, you can appreciate the specificity of the wiring in the inner plexiform layer.

Amacrine cells participate in retinal gain control, and they help report the direction of movement to ganglion cells. In addition, the rapid response properties of certain kinds of amacrine cells throttle the visual output during ballistic eye movements, helping to avoid blurred images.

The stratification of the inner plexiform layer is relevant for the behavior of the ganglion cells. Most of the synapses between bipolar cells, amacrine cells, and ganglion cells are carefully organized with this layer. The IPL is further subdivided into at least 5 distinct strata, each of which contains a particular type of connection, or multiple types. Kolb reviews at least 40 different kinds of amacrine cells here. As an example of IPL organization, the high-level stratification at the level of on and off responses is shown in the figure.

The information from the photoreceptors is combined in different ways into the retinal ganglion cells. Some examples are shown in the figure. There are over 20 different kinds of retinal ganglion cells that pass filtered versions of the retinal image into the brain. Many of these pathways are kept segregated in the central areas, and it is not entirely clear whether and how they may be eventually combined into a unified visual perception. Most likely, the human brain performs correlations between the information in the various retinal channels, to extract the statistical properties of the visual input.

There are only a very few centrifugal fibers that enter the retina from the brainstem and the central nervous system. Most of the ganglion cells send projections into the optic chiasm, with a few of them traveling to the superior colliculus. The ganglion cells are organized into "channels" that convey different kinds of information. Many are static color sensitive or color insensitive cells that respond mainly to luminance and luminance contrast. Others are highly dynamic and may be both motion and direction sensitive.

The richness of dynamic behavior in the retina is important to understand. It is reflected not only in the dynamic behaviors of individual neurons, but also in the form of traveling waves that augment precision in the population and become very important during development. Many of the dynamics are provided by the amacrine cell layer(s).

Finally, there is a small subset of intrinsically photoreceptive retinal ganglion cells, that contain the photopigment melanopsin.

Retinal ganglion cells in primates are broadly classified into three categories: P cells, M cells, and K cells. These correspond approximately, to the X, Y, and W cells in cats. The P (parvocellular) cells exhibit mostly tonic responses to light, whereas the M (magnocellular) neurons respond transiently to onset and offset. The K (koniocellular) neurons have distinct targets, they project to different areas of the LGN and superior colliculus, and the LGN pathways have different endpoints in the primary visual cortex.

Center-Surround Organization

With respect to light stimuli, the ganglion cells in the retina act like filters. The filters have a "Mexican hat" shape, they have a tall central area that responds to input with excitation, and smaller peripheral lobes that respond to input with inhibition. These filter shapes translate directly to response shapes in the visual field. This relationship is called the "receptive field" of the neuron.

When studying the center-surround organization of retinal receptive fields, it is important to understand that they are not static entities. Stimulation of a single spot on the retina creates a multiplicity of responses in retinal ganglion cells, some when the light turns on, some when it turns off, and some during the entire duration of the stimulus. Additionally, some retinal ganglion cells are direction sensitive, and some of the surrounds are too. Both the center and the surround take time to develop, and generally speaking the surrounds involve more synapses so they take longer (Turner et al 2018, Wienbar & Schwartz 2018).

The filters in the retina pre-process the visual signal before it ever gets to the brain. The output of the retina is already gain-adapted, and subject to the centrally controlled effects of pupil constriction and lens accommodation. Generally speaking the filters cause ganglion cells to respond well to edges and transients, as depicted in these drawings.

Any or all of the P/M/K ganglion cells can be color sensitive and direction sensitive. Color sensitivity is conferred by a particular combination of retinal cones, and direction sensitivity can be generated by geometric specificity of both axon and dendrite branching patterns.

In addition to the time-varying elements of receptive fields, there is also local and global adaptation mediated at the synaptic level.

By the time the visual signal leaves the retina, it has been considerably modified, although its original character is still recognizable. Filtering by the retinal circuitry guarantees that the image leaving the retinal ganglion cells is not the same as the pattern of light that created it. So how then, does the brain reconstruct the original visual input from multiple channels that distort its original character?

To answer this question, we have to keep going and look at what happens to the visual information as it gets transmitted centrally from the eye. The first stop along the way to the cerebral cortex is the lateral geniculate nucleus of the thalamus, which receives inputs from retinal ganglion cells that feed the optic nerve.