Visual coding

General Principles of Perception

Perceive an object when it emits/reflects energy that stimulates receptors that transmit information to your brain
Rene Descartes  believed that brain-s representation of a stimulus resembles the stimulus

 Nerves from the eye would project pattern of impulses arranged like a picture, right side up

Brain codes information in terms of which neurons respond, their amount of response, and timing of their response
One aspect of coding is which neurons are active
- Johannes Muller  described this insight as the law of specific nerve energies
- Whatever excites a particular nerve establishes a special kind of energy unique to that nerve
Another aspect of coding is amount of response  how many action potentials a neuron sends per unit of time (depends on frequency

of firing)

The Eye and Its Connections to the Brain

Light enters the eye through an opening in the center of the iris  pupil
Focused by the lens (adjustable) and cornea (not adjustable) and projected onto retina (rear surface of eye) which is lined with visual receptors
Light from left side of the world strikes right half of the retina and vice versa
Light from above strikes bottom half of retina and vice versa
Inversion of image poses no problem for nervous system
Visual system codes image by various kinds of neuronal activity

Route within the Retina

In vertebrate retina, messages go from receptors at back of the eye to bipolar cells (located closer to the center of the eye)  send messages to ganglion cells (located still closer to center of the eye)  cells- axons join together and travel back to brain
Amacrine cells  additional cells that get information from bipolar cells and send it to other bipolar, amacrine, and ganglion

cells

Cells are transparent, and light passes through them without distortion
Blindspot  ganglion cell axons form optic nerve (exits through back of the eye) and point of which it leaves (also where blood vessels enter/leave) is blind spot because it has no receptors
Never notice blind spot for two reasons:

o Brain fills in the gap o Anything in the blind spot of one eye is visible to the other eye

Fovea and Periphery of the Retina

When looking at details (letters on page), you fixate them on central portion of your retina, especially fovea (“pit”) which is a tiny area specialized for acute, detailed vision
Because blood vessels and ganglion cell axons almost absent near fovea, nearly unimpeded vision
Each receptor in fovea connects to single bipolar cell  connects to single ganglion cell  has an axon to the brain
Midget ganglion cells  ganglion cells in the fovea of humans and other primates
Result  each cone in the fovea is connected to the brain with a direct route that registers the exact location of input

(vision dominated by what we see in fovea)

Hawks and other birds have greater density of visual receptors on top half of retinas than on bottom half  arrangement is adaptive because predatory birds spend more of day soaring high in air looking down
Towards periphery of retina, more and more receptors converge onto bipolar and ganglion cells
Summation enables perception of fainter lights in periphery  foveal vision has better acuity (sensitivity to detail), and peripheral vision has better sensitivity to dim light
In periphery, ability to detect detail is limited by interference from other nearby objects

Visual Receptors: Rods and Cones

Vertebrate retina contains two types of receptors
1. The Rods  abundant in periphery of human retina, respond to faint light but are not useful in daylight because bright light bleaches them
2. Cones  abundant in and near the fovea, less active in dim light, more useful in bright light, and essential for color vision

o Good color vision in fovea but not in periphery (contains mostly rods) o Rods outnumber cones, but cones provides ~90% of brains input o 20:1 ratio of rods to cones  much higher in species active at night

Heightened visual responses are valuable in many activities (sports required to aim)
Part of difference among individuals in visual sensitivity is due to number of axons from retina (probably established early in life)
Photopigments  both rods and cones contain it (chemicals that release energy when struck by light)

Color Vision

Visible light consists of electromagnetic radiation within range from less than 400nm to more than 700nm; receptors in our eyes tuned to detecting light
Perceive shortest visible wavelengths as violet; longer wavelengths as blue, green, yellow, orange, and red; ultraviolet radiation

(shorter wavelengths)

In some species of birds, male and female look alike to us but different to birds because male reflects more UV light

The Trichromatic (Young-Helmholtz) Theory

Thomas Young  1^st person to advance understanding on question of “how many types of color receptors do we have”?; first to decipher Rosetta stone; founded modern wave theory of light; discovered much of anatomy of eye
Recognized that color required biological explanation; proposed that we perceive color by comparing responses across few types of receptors, each of which was sensitive to a different range of wavelengths
Trichromatic theory (Young-Helmholtz Theory)  modified by Hermann von Helmholtz; we perceive color through relative rates of response by 3 kinds of cones (each maximally sensitive to different set of wavelengths)
More intense light increases activity of all 3 cones without much change in ratio of responses (light appears brighter, but still same color)
Perception depends on frequency of response in one cell relative to frequency of another cell
NS determines color and brightness of light by comparing responses of different types of cones (mice have one kind of cone, therefore they are colorblind)
Long and medium-wavelength cones far more abundant than short-wavelength (blue) cones
In the retina-s periphery, cones are so scarce that you have no useful color vision
The smaller the dot, the farther you have to move it into your visual field before you can identify the color

The Opponent-Process Theory

Negative color after-image  result of staring at colored object for prolonged length of time, then looking at white surface  image seen negatively; with replacement of red with green, green with red, yellow and blue with each other , and black and white with each other
Ewald Hering  proposed opponent-process theory (we perceive color in terms of opposites; the brain has a mechanism that perceives color on continuum from red to green, another from yellow to blue, and another from white to black)
Ex: bipolar cell excited by short-wavelength (blue) cone and inhibited by mixture of long-wavelength and mediumwavelength cones  increase in bipolar cell-s activity produces experience blue, and a decrease produces experience yellow
Increase in response produces one perceptions, and decrease produces different perception
Afterimages depend on the whole context, not just the light on individual receptors; cerebral cortex must be responsible, not bipolar or ganglion cells

The Retinex Theory

Both other theories can-t explain color constancy (ability to recognize colors despite changes in light)  brain compares color of one object with color of another, in effect subtracting certain amounts of color from each
Perception of brightness of an object requires comparing it with other objects
Edwin Land  proposed retinex theory to account for color and brightness (cortex compares information from various parts of retina to determine brightness and color for each area)
Visual perception requires a reasoning process, not just retinal stimulation

Color Vision Deficiency

Known as colorblindness (genetic reasons, some people lack one or two types of cones)
Color depends on what our brains do with incoming light
In red-green color deficiency, people have trouble distinguishing red from green because long and medium-wavelength cones have same photo pigment instead of different ones (gene causing deficiency is on X chromosome  because men have only one X chromosome, they only have one type of long-wavelength receptor; women who have both kinds of long-wavelength genes produce two kinds of long-wavelength cones)
Brain adapts to use the information it receives

HOW THE BRAIN PROCESSES VISUAL INFORMATION

Everything in psychology starts with sensations
Explain mechanisms of vision gives us some idea what it means to explain something in biological terms

An Overview of the Mammalian Visual System

Rods and cones of retina make synapses with horizontal cells (type of cell that receives input from receptors and delivers inhibitory input to bipolar cells) and bipolar cells
Horizontal cells make inhibitory contact onto bipolar cells, which in turn make synapses onto amacrine cells and ganglion cells (all cells within eyeball)
Axons of ganglion cells form optic nerve, which leaves the retina and travels alon lower surface of the brain  optic nerves from two eyes meet at optic chiasm, where half the axons from each eye cross to opposite side of the brain
Information from nasal half of each eye crosses to contralateral hemisphere; information from temporal half goes to ipsilateral hemisphere
Most ganglion cell axons go to lateral geniculate nucleus (part of the thalamus)

o Small number of axons go to superior colliculus and other areas (including part of hypothalamus that controls wakingsleeping schedule o Geniculate sends axons to other part of thalamus and occipital cortex  returns many axons to thalamus so thalamus ad cortex constantly feed information back and forth

Processing in the Retina

Rods and cones of two retinas combined send a quarter of a billion messages
Lateral inhibition is retina-s way of sharpening contrasts to emphasize the borders of objects
Receptors send messages to excite closest bipolar cells and also send messages to slightly inhibit them and neighbor-s to their sides
Light striking the rods and cones decreases spontaneous output; they have inhibitory synapses onto bipolar cells  light on rods or cones decreases inhibitory output (means net excitation)
In fovea  each cone attaches to one bipolar cell
Horizontal cell is a local cell (no axon and no action potentials) depolarization decays with distance; bipolar cells to the sides (lateral) get no excitation but some inhibition by horizontal cell
Lateral inhibition  reduction of activity in one neuron by activity in neighboring neurons; when light falls on a surface  bipolar cells just inside border are most excited, and those outside the border respond the least

Further Processing

Each cell in visual system has receptive field (area in visual space that excites/inhibits it); receptive field of receptor is simply point in space from which light strikes the cell
Other visual cells derive their receptive fields from connections they receive
Rod/cone has tiny receptive field in space to which it-s sensitive  small group of rods/cones connect to bipolar cell, with receptive field that is the sum of those of the cells connected to it  several bipolar cells report to a ganglion cell (has larger receptive field); receptive field of several ganglion cells converge to form receptive field at next level
To find a cell-s receptive field, record cell while shining light in various locations; if light from particular spot excites the neuron, that location is part of the neuron-s excitatory receptive field; if it inhibits activity, the location is in inhibitory receptive field
Receptive field of ganglion cell has circular center with antagonistic doughnut-shaped surround (light in center of receptive field might be excitatory with surround inhibitory, or the opposite)
Parocellular neurons  small cell bodies and receptive fields (mostly in/near fovea  has many cones); suited to detect visual details; respond to color (each neuron being excited by some wavelengths and inhibited by others)
Magnocellular neurons  larger cell bodies and receptive fields (distributed evenly throughout retina, including periphery  sensitive to movement but not color/details); respond strongly to movement and large overall patterns (don-t respond to color/fine detail)
Koniocellular neurons  small cell bodies (similar to parvocellular neurons, but they occur throughout retina); have several functions and axons terminate in several locations
Axons from ganglion cells form optic nerve  proceeds to optic chiasm  half of axons cross to opposite hemisphere
Most axons go to lateral geniculate nucleus of the thalamus; cells of lateral geniculate have receptive fields that resemble those of ganglion cells (excitatory/inhibitory central portion and surrounding ring with opposite effect); after information reaches cerebral cortex, receptive fields become more complicated

The Primary Visual Cortex

Most visual information from lateral geniculate nucleus of thalamus goes to primary visual cortex in occipital cortex (also known as area

V1 or striate cortex)

People with damage to area report no conscious vision, no visual imagery, and no visual images in their dreams; adults who lose vision because of eye damage continue to have visual imagery and visual dreams
Some people with damage experience blindsight  ability to respond in limited ways to visual information without perceiving it consciously; within damaged part of visual field they have no awareness of visual input (not even to distinguish between bright sunshine and utter darkness)
Two explanations for this:
1. Small amount of healthy tissue remain within damaged visual cortex, not large enough to provide conscious perception but enough to support limited visual functions
2. Thalamus sends visual input to several other brain areas besides VI (including parts of temporal cortex); after damage, connections to other areas strengthen enough to produce certain kinds of experience despite lack of conscious visual perception; impairing input from thalamus to other cortical areas abolishes blindsight

Simple and Complex Receptive Fields

David Hubel and Torsten Wiesel  recorded from cells in cats- and monkeys- occipital cortex while shining light patterns on retina; distinguished several types of cells in visual cortex
Simple cell  has receptive field with fixed excitatory and inhibitory zones; more light shines in excitatory zone, more the cell responds; more light shines in inhibitory zones, less cell responds; most have bar-shaped/edge-shaped receptive fields (more respond to horizontal/vertical orientations than diagonals)
Complex cells  located in areas V1 and V2 (don-t respond to exact location of stimulus); responds to pattern of light in particular orientation (vertical bar); without excitatory/inhibitory zones
Best way to classify cell as simple or complex is present stimulus in several locations; cell that responds to stimulus in only one location is simple cell; one that responds equally throughout large area is complex cell
Ended-stopped (hypercomplex) cells  resemble complex cells with one exception: end-stopped cell has a strong inhibitory area at one end of bar-shaped receptive field (largest of 3 cells)

The Columnar Organization of the Visual Cortex

Cells having similar properties are grouped together in visual cortex in columns perpendicular to the surface (cells in specific column may respond to only one eye or both equally); don-t fire at the same time
Cells within given column process similar information

Are Visual Cortex Cells Feature Detectors?

Given that neurons in area V1 respond strongly to bar or edge-shaped patterns, might suppose that activity of such a cell is perception of a bar, line, or edge; such cells might be feature detectors (neurons whose responses indicate presence of particular feature/object  shape or direction of movement) o Ex: waterfall illusion  suggests you fatigued neurons that detect downward motion, leaving unopposed the detectors for the opposite motion
Believe that neurons in area V1 detect spatial frequencies rather than bars/edges

o Fourier analysis demonstrates that combination of sine waves can produce unlimited variety of other patterns (sine wave frequencies) o Series of spatial frequency detectors, some sensitive to horizontal patterns and others to vertical patterns, cold represent any possible display (perceive world as objects, not sine waves)

Visual cortex perceives objects because of interactions between primary visual cortex and other brain areas
Even at early stage of perception, brain-s activity corresponds to what person perceives and not just physical pattern of light and dark

Development of the Visual Cortex

In newborn, normal properties of visual system develop first even without visual experience; brain needs visual experience to maintain and fine-tune its connections

Deprived Experience in One Eye

For cats/primates, most neurons in visual cortex receive binocular input (stimulation from both eyes)
Most cells in visual cortex respond to both eyes (better to one eye than the other)
Innate mechanisms can-t make connections exactly right because exact distance between eyes varies

Deprived Experience in Both Eyes

When one eye is open, synapses from open eye inhibit the synapse from the closed eye; if neither eye is active, no axon outcompetes any other
If the eyes remain shut longer, cortical responses start to become sluggish and lose well-defined receptive fields
Visual cortex eventually starts responding to auditory and touch stimuli
Sensitive period  when experiences have particularly strong and enduring influence; it ends with onset of certain chemicals that stabilize synapses and inhibit axonal sprouting; even after long sensitive period, prolonged experience (full week without visual stimulation to one eye) produces measurable effect on visual cortex

Uncorrelated Stimulation in the Two Eyes

Stereoscopic depth perception requires brain to detect retinal disparity (discrepancy between what the left and right eye see); experience fine-tunes binocular vision, and abnormal experience disrupts it

o Both eyes are active but no cortical neuron consistently receives messages from one eye that match messages from the other eye; each neuron in the visual cortex becomes responsive to one eye of the other and few neurons respond to both  poor depth perception

Strabismus (strabismic amblyopia)  “lazy eye”; condition in which eyes do not point in same direction; attend to one eye and not the other (treatment is putting a patch over active eye, forcing attention to other eye)

Early Exposure to a Limited Array of Patterns

Astigmatism  blurring of vision for lines in one direction (vertical, horizontal, diagonal) caused by an asymmetric curvature of the eyes

(when the eyeball is not quite spherical) which person can see one direction of lines more clearly than other

Impaired Infant Vision and Long-Term Consequences

Existence of a sensitive period for visual cortex means that after you pass that period, your visual cortex won-t change as much or as fast; if an infant has a problem early, we need to fix it early (ex: cataracts)

PARALLEL PROCESSING IN THE VISUAL CORTEX

Different parts of the brain-s visual system get information on a need-to-know basis
Person as whole sees all aspects of objects (color, shape, location, movement) but each individual area of visual cortex doesn-t
When you see something, one part of your brain sees its shape, another sees color, another detects location, and another perceives movement
Neuroscientists identified ~80 brain areas that contribute to vision in different ways (after brain damage possible to see certain aspects of object and not others)

The “What” and “Where” Paths

Primary visual cortex (V1) sends information to secondary visual cortex (V2), which processes information further and transmits it to additional areas
V1 sends information to V2  V2 sends information to V1
From V2, information branches out in several directions for specialized processing
Ventral stream  through the temporal cortex is called the “what” pathway because it-s specialized for identifying and recognizing objects o People with damage to area see where but not what; man had stroke that damaged temporal cortex  couldn-t read, recognize faces, or identify objects by sight (could identify/recognize through other senses); could take a walk accurately and go around objects without bumping into them
Dorsal stream  through parietal cortex is the “where” pathway because it helps the motor system locate objects

o People with damage to area seem to have normal vision (can read, recognize faces, describe objects in detail)  they know what things are but not where things are (describe what they see but bump into objects)

Two streams communicate and each participates to some extent in perceiving both shape and location (damaging produces different

deficits)

Detailed Analysis of Shape

As visual information goes from simple cells to complex cells and then to other brain areas, the receptive fields become more specialized
In secondary visual cortex (V2), may cells responds best to lines, edges, and sine wave gratings, but some cells respond selectively to circles, lines at right angles, or other complex patterns; in later parts of visual system, receptive properties become still more complex

The Inferior Temporal Cortex

Cells in inferior temporal cortex respond to identifiable objects
A cell that responded to a particular stimulus would respond almost equally to its negative image/mirror image but not to physically similar stimulus in which the “figure” now appeared to be part of the “background” (respond according to what viewer perceives, not what stimulus is physically; all different views seen as one object)
Visual agnosia  “visual lack of knowledge”; inability to recognize objects despite otherwise satisfactory vision; results from damage in the temporal cortex (someone might be able to point to visual objects and slowly describe them but fail to recognize what they are)
Researchers used fMRI to record brain activity as people viewed pictures of many objects; of various objects, most didn-t activate one brain area more than another; brain doesn-t have specialized area for seeing flowers, fish, birds, etc., but 3 types of objects produce specific responses:

o One part of parahippocampal cortex responds strongly to pictures of places and not so strongly to anything else o Part of fusiform gyrus of inferior temporal cortex (especially right hemisphere) responds strongly to faces, much more than anything else o Area close to this face area responds more strongly to bodies than anything else

Brain is adept at detecting biological motion (kinds of motion produced by people/animals)

Recognizing Faces

Newborns predisposed to pay more attention to faces than other stationary displays; supports idea of built-in face recognition module

(concept of “face” requires eyes to be on top, but face doesn-t have to be realistic)

Most people recognize faces of own ethnic group better than those of other people
Face recognition depends on several brain areas  part of occipital cortex; anterior temporal cortex; prefrontal cortex; and the fusiform gyrus of inferior temporal cortex (right hemisphere) o Damage to any of these areas leads to prosopagnosia (inability to recognize faces); some people are poor throughout life at recognizing faces because they were born with shortage of connections to/from fusiform gyrus o People with prosopagnosia can read and recognize voices (visual activity and memory not a problem); only something related to faces; can-t identify the face but can tell if male/female, old/young

Color Perception

Neurons in many parts of the visual system show some response to changes in color, one brain area is particularly important  V4
Apparent color of an object depends not only on light reflected from that object, but also how it compares with objects around it; responses of cells in V4 correspond to apparent or perceived color of an object (depends on total context) o After damage to area, people don-t become colorblind, but lose color constancy (ability to recognize something as being the same color despite changes in lighting)

Motion Perception

The Middle Temporal Cortex

Viewing a complex moving pattern activates many brain areas spread among all 4 lobes of cerebral cortex; two areas especially activated by motion are:
1. Area MT (area V5)  middle temporal cortex (color-insensitive)

o Most cells respond selectively when something moves at particular speed in particular direction; detect acceleration/deceleration as well as absolute speed, and respond to motion in all 3 dimensions; also responds to photographs that imply movement

Area MST  adjacent to area MT; medial superior temporal cortex
- Both areas receive input mostly from Magnocellular path, which detects overall patterns (color-insensitive) o Cells in dorsal part of area respond best to more complex stimuli (expansion, contraction, rotation) of a large visual scene (when you move forward/backward/tilt head) o Most areas in MST are silent during eye movements; MST neurons enable you to distinguish between result of eye movements and result of object movements
- These two kinds of cells (one that records movement of single objects and ones that record movement of entire background) converge messages onto neurons in ventral part of area MST, where ells respond to an object that moves relative to its background

Motion Blindness

Motion blindness  damage to areas MT and MST; ability to see objects but impairment at seeing whether they are moving or, if so, which direction and how fast
People with motion blindness somewhat better at reaching for moving object than at describing its motion (all aspects far behind other people)
People with full color vision can imagine what it would be like to be color deficient (more difficult to imagine being motion blind); individuals reporting that people were suddenly here or there but didn-t see them moving
You don-t see your own eyes move because several of visual areas of your brain decreases their activity during voluntary eye movements  saccades
Suppression is particularly strong in area MT (motion detection) and the “where” path of parietal cortex
Areas responsible for shape and color detection remain at nearly normal activity
During a voluntary eye movement, you become temporarily motion blind
Opposite for motion blindness also occurs: some people are blind except for the ability to detect which direction something is moving  area MT gets some input directly from the lateral geniculate nucleus of the thalamus  therefore, even after extensive damage to area

V1, area MT still has enough input to permit motion detection

Different areas of brain process different kinds of visual information and possible to develop many kinds of disability