Professional Documents
Culture Documents
A Century of Gestalt Psychology in Visual Perception
A Century of Gestalt Psychology in Visual Perception
James H. Elder
York University
Michael Kubovy
Stephen E. Palmer
University of Virginia
Mary A. Peterson
Manish Singh
University of Arizona
General Introduction
Exactly 100 years ago, Wertheimer (1912) published his paper
on phi motionperception of pure motion, without object motionwhich many consider to be the beginning of Gestalt psychology as an important school of thought. The present status of
Gestalt psychology is ambiguous. On the one hand, many psychologists believe that the Gestalt school died with its founding fathers
in the 1940s or after some devastating empirical findings regarding
electrical field theory in the 1950s, or that it declined because of
fundamental limitations that blocked further progress, while stronger theoretical and experimental frameworks arose in the 1960s
and 1970s that have dominated the field ever since (e.g., cognitive
science, neuroscience). On the other hand, almost all psychology
textbooks still contain a Gestalt-like chapter on perceptual organization (although often poorly connected to the other chapters),
and new empirical papers on Gestalt phenomena appear with
increasing frequency.
We are convinced that Gestalt psychology is still relevant to
current psychology in several ways. First, questions regarding the
emergence of structure in perceptual experience and the subjective
nature of phenomenal awareness (e.g., visual illusions, perceptual
switching, context effects) continue to inspire contemporary scientific research, using methods and tools that were not at the
Gestaltists disposal. Second, the revolutionary ideas of the Gestalt
movement continue to challenge some of the fundamental assumptions of mainstream vision science and cognitive neuroscience
(e.g., elementary building blocks, channels, modules, informationprocessing stages). Much progress has been made in the field of
nonlinear dynamical systems, both theoretically and empirically
(e.g., techniques to measure and analyze cortical dynamics), progress that allows modern vision scientists to surpass some of the
limitations in old-school Gestalt psychology as well as those in
mainstream vision research.
The centennial anniversary of Gestalt psychology is therefore an
excellent opportunity to take stock of what we have discovered
about core Gestalt phenomena of perceptual organization and how
our understanding of the underlying mechanisms has evolved since
Wertheimers seminal contribution. Due to this reviews scope, we
divide it in two parts: This article deals with perceptual grouping
and figure ground organization, whereas the second covers modern developments regarding the general conceptual and theoretical
frameworks that underlie Gestalt ideas (e.g., holism, emergence,
dynamics, simplicity). In Table 1, we provide an overview of the
topics covered in the first review article, together with the section
headings, the questions or issues being raised, and some of the
answers provided. One of the aims of our review is to remove the
many misunderstandings surrounding Gestalt psychology, which
are listed separately in Table 2, along with a more balanced view
on the actual state of affairs.
To put these two reviews in perspective, we first introduce the
key findings and ideas of the founders of Gestalt psychology,
along with a brief sketch of its further development, rise, and fall
(for an extensive treatment, see Ash, 1995). The historical section
ends with a discussion of Gestalt psychologys empirical and
conceptual problems and an indication of how these limitations are
being addressed in current research. We then review current research on perceptual grouping and figure ground organization in
more detail in the remaining sections. We focus on these two
1173
topics because they were the most important ones in the Gestalt
tradition of perceptual organization and still are today, even for
vision in general.
WAGEMANS ET AL.
1174
Table 1
Overview of the Article With Section Numbers and Headings, Questions and Issues Raised, and Answers Provided
Section number
1
Section title
General Introduction
3.1
3.2
3.3
3.4
3.5
4.1
4.2
2.1
2.2
2.3
2.4
3
4.3
4.3.1
4.3.2
4.3.3
4.3.4
4.4
4.4.1
4.4.2
4.4.3
4.5
5
5.1
Questions/issues/answers
We explain why an extensive review of 100 year of research on
perceptual organization is valuable.
We address four questions regarding Gestalt psychology:
(1) How did it start?
(2) What does it stand for?
(3) How did it evolve?
(4) Where does it stand now?
We distinguish grouping and figureground organization.
We enumerate the classic grouping principles: proximity, similarity,
common fate, symmetry, parallelism, continuity, closure.
We review progress in our understanding of perceptual grouping since
the early days of Gestalt psychology; specifically:
(1) We discuss a number of additional principles that have been
discovered since the initial set was described: generalized common
fate, synchrony, common region, element connectedness, uniform
connectedness.
(2) We demonstrate how at least some grouping principles can be
measured experimentally and expressed in quantitative laws:
(a) When several orientations can be perceived based on grouping by
proximity in a particular dot lattice, the outcome is determined by the
relative distance alone, not by the angle between the competing
organizations (affecting the global symmetry of the lattice and how it
looks).
(b) When grouping by proximity and grouping by similarity are
concurrently applied to the same pattern, the two principles are
combined additively.
(3) We review a century of research on grouping in dynamic patterns,
including Kortes laws, element and group motion in Ternus displays,
space-time coupling versus space-time tradeoff.
(4) We demonstrate that grouping principles operate at multiple levels:
Provisional grouping takes place at each stage of processing, possibly
with feedback from higher levels to lower ones, until a final, conscious
experience arises of a grouping that is consistent with the perceived
structure of the 3-D environment.
1175
Table 1 (continued)
Section number
5.2
5.3
5.4
Section title
Classic Image-Based Configural Principles of Figure
Ground Organization
New Image-Based Principles of FigureGround
Organization
6.1
6.2
6.3
5.5
5.6
6
6.4
6.5
7
7.1
7.2
7.3
7.4
Questions/issues/answers
We discuss the classic configural principles of figureground
organization: convexity, symmetry, small region, surroundedness.
We discuss new image-based principles of figureground organization:
lower region, top-bottom polarity, extremal edges and gradient cuts,
edge-region grouping, articulating motion, advancing region motion,
contour entropy as a ground cue ( part salience, axiality).
We discuss the evidence for nonimage-based influences on figureground
organization: past experience, attention and perceptual set.
We discuss how figureground organization relates to shape and depth
perception.
Conclusion
General Discussion and Conclusion
The Swinging Pendulum of Gestalt History
Gestalt Research Anno 2012
Limitations and Challenges to Contemporary Research
on Perceptual Organization
Conclusion
WAGEMANS ET AL.
1176
Table 2
Common Misunderstandings About Gestalt Psychology
Common assumption
1177
Table 3
Key Claims by the Berlin School of Gestalt Psychology in Opposition to Other Schools
Berlin school of Gestalt psychology
(Wertheimer, Khler, Koffka)
Structured wholes or Gestalten are the primary units of mental
life.
Experimental phenomenology: Perceptual experience must be
described in terms of the units people naturally perceive.
Percepts arise on the basis of continuous whole-processes in
the brain; percepts organize themselves by mutual
interactions in the brain.
Perceptual organization is based on innate, intrinsic,
autonomous laws.
Simplicity or minimum principle.
Gestalten (structured experiences, wholes) are different from
the sum of the parts.
Two-sided or reciprocal dependency between parts and wholes:
There are specifiable functional relations that decide what
will appear or function as a whole and what as parts.
Often the whole is grasped even before the individual parts
enter consciousness.
Perception emerges through self-organization; perception
arises nonmechanistically through an autonomous process in
the brain.
No analysis into stages, but functional relations in the
emergence of Gestalts can be specified by Gestalt laws of
perceptual organization.
Holism integrated with natural science (physical Gestalten,
isomorphism, minimum principle).
Opposing schools
Structuralism/associationism/empiricism (von Helmholtz, Wundt)
Sensations are the primary units of mental life.
Introspection: Perceptual experience must be analyzed as combinations of
elementary sensations of physical stimuli as their building blocks.
Percepts are associated combinations of elementary excitations.
Perceptual organization is based on perceptual learning, past experience, intentions.
Likelihood principle
Graz school of Gestalt psychology (Meinong; von Ehrenfels, Benussi)
Gestalt qualities are more than the sum of the constituent primary sensations.
One-sided dependency between parts and wholes (the wholes depend on the parts,
but the parts do not depend on the whole).
whole-processes with their whole-properties and laws, characteristic whole-tendencies and whole-determinations of parts
(Wertheimer, 1922/1938a, p. 14). The perceptual field does not
appear to us as a collection of disjointed sensations, but possesses
a particular organization of spontaneously combined and segregated objects.
In 1923, Wertheimer published a follow-up paper, which was an
attempt to elucidate the fundamental principles of that organization. The most general principle was the so-called law of Prgnanz, stating, in its most general sense, that the perceptual field and
objects within it will take on the simplest and most encompassing
(ausgezeichnet) structure permitted by the given conditions. For
Khler (1920), this tendency towards the Prgnanz of the Gestalt
was just another example that phenomenal Gestalten were like
physical Gestalten: As shown by Maxwell and Planck, all processes in physical systems, left to themselves, show a tendency to
achieve the maximal level of stability (homogeneity, simplicity,
symmetry) with the minimum expenditure of energy allowed by
the prevailing conditions. More specific principles that determine
perceptual organization according to Wertheimer were proximity,
similarity, uniform density, common fate, direction, good continuation and whole properties (or Ganzeigenschaften) such as closure, equilibrium, and symmetry.
Empirical work on these principles existed before Wertheimers
landmark paper (for a recent review, see Vezzani, Marino, &
Giora, 2012), but now the general claim that perceptual experience
is organized was turned into a complex open-ended research
1178
WAGEMANS ET AL.
mode of thinking and research style did not sit comfortably in the
intellectual and social climate of the postwar world, and they were
confronted with vehement criticism. Two sets of explanations have
been given for this outcome (Ash, 1995). The first emphasizes
institutional, political, and biographical contingencies. Koffka,
Khler, and Wertheimer all left for the United States and obtained
positions where they could do excellent research but could not
train doctoral students. The Gestalt schools further expansion was
also handicapped by the early deaths of Max Wertheimer in 1943
and Kurt Koffka in 1941, as well as many other Gestalt psychologists of the first and second generations (e.g., Duncker, Gelb,
Lauenstein, Lewin, von Restorff). In Germany, Metzger, Rausch,
and Gottschaldt did have a large number of doctoral students, but
few of them carried on in the Gestalt tradition. A notable exception
is Lothar Spillmann, who obtained his doctorate with Metzger in
Munster in 1964 and who has pioneered the impact of Gestalt ideas
in modern neurophysiology ever since (e.g., Spillmann, 1999,
2009).
The second set of explanations concerns scientific issues of a
methodological and conceptual nature (summarized in the left
column of Table 4). Compared to the rigor of psychophysics and
behaviorism, Gestalt psychology was severely criticized for offering mere demonstrations, using either very simple or confounded
stimuli, formulating laws with little precision, and adding new
laws for every factor shown to have an influence on perceptual
organization. In the 1950s and 1960s, its critics increasingly insisted on causal explanations, by which they meant cognitive
operations in the mind that could be modeled as computer algorithms or neural mechanisms that could be attributed to the properties of single cells that were discovered by Hubel and Wiesel in
that period. In addition, serious conceptual limitations appeared
when Gestalt thinking was extended to other areas such as personality and social psychology (e.g., Richard Crutchfield, Solomon Asch, Fritz Heider, David Krech). The further the metaphors
1179
Table 4
Problems in Old-School Gestalt Psychology and How They Are Solved in Contemporary Research
Problems
Mere demonstrations based on direct (subjective) reports
1 Article 1; 2 Article 2.
Solutions
Real experiments (1)
Also indirect methods (matching, priming, cuing) and performance
measures (accuracy, reaction time)
Also psychophysical techniques (thresholds in
detection/discrimination tasks)
Also neuropsychological studies with brain-damaged patients
Carefully constructed stimuli, sometimes also richer stimuli (1)
Allowing research of everyday tasks
Allowing research of ecological foundations
Also studying relationships with other processes (1), e.g.,
Perceptual grouping in relation to depth perception, lightness
perception
Visual contour completion in relation to surface geometry and
layout
Figureground organization in relation to shape and depth
perception
Quantification, which allows measurement (1, 2)
Unification into stronger, better developed theoretical frameworks (2)
Computational models (1, 2)
Somewhat better understanding of neural basis (1, 2)
WAGEMANS ET AL.
1180
Perceptual Grouping
Introduction
Historically, the visual phenomenon most closely associated
with perceptual organization is grouping: the fact that observers
perceive some elements of the visual field as going together more
strongly than others. Indeed, perceptual grouping and perceptual
organization are sometimes presented as though they were synonymous. They are not. Grouping is one particular kind of organizational phenomenon, albeit a very important one. Another is
figure ground organization. In general, grouping determines what
the qualitative elements of perception are, and figure ground
determines the interpretation of those elements in terms of their
shapes and relative locations in the layout of surfaces in the 3-D
world.
Max Wertheimer first posed the problem of perceptual grouping
in his groundbreaking 1923 paper by asking what stimulus factors
influence the perceived grouping of discrete elements. He first
demonstrated that equally spaced dots do not group together into
larger perceptual units, except as a uniform line (see Figure 1A),
and then noted that when he altered the spacing between adjacent
dots so that some dots were closer than others, the closer ones
grouped together strongly into pairs (see Figure 1B). This factor of
relative distance, which Wertheimer called proximity, was the first
of his famous laws or (more accurately) principles of grouping.
Wertheimer went on to illustrate other grouping principles,
several of which are portrayed in Figure 1. Figures 1C, 1D, and 1E
demonstrate different versions of the general principle of similarity: All else being equal, the most similar elements (in color, size,
and orientation for these examples) tend to be grouped together.
Another powerful grouping factor is common fate: All else being
equal, elements that move in the same way tend to be grouped
together. Notice that both common fate and proximity can actually
be considered special cases of similarity grouping, with velocity
and position as the relevant properties, respectively. Further factors influencing perceptual grouping of more complex elements,
such as lines and curves, include symmetry (see Figure 1G),
parallelism (see Figure 1H), and continuity or good continuation
(see Figure 1I). Continuity is important in Figure 1I because
observers perceive it as containing two continuous intersecting
lines rather than as two angles whose vertices meet at a point.
Figure 1J illustrates the effect of closure: All else being equal,
elements that form a closed figure tend to be grouped together.
This display also shows that closure can dominate continuity, since
Figure 1. Illustration of several grouping principles. Adapted from Perceptual Organization in Vision, by S. E. Palmer, in Stevens Handbook of
Experimental Psychology: Vol. 1. Sensation and Perception (p. 183), ed.
by H. Pashler, 2002, New York: NY, Wiley. Copyright 2002 by John
Wiley and Sons. Adapted with permission.
1181
1182
WAGEMANS ET AL.
and closure were at least as important, if not more so, in the initial
organization that can be tapped by such methods. Nevertheless, the
theoretical rationale for some organizational process like UC to
contribute to creating a set of potential perceptual units on which
further grouping and parsing can operate seems sound. Indeed,
something like it is a standard assumption in most theories of
computational vision (e.g., Marr, 1982).
Whereas the research on grouping principles, as described
above, provides predictions about what elements in a display are
likely to be grouped together, it does not reveal how strong each of
the grouping principles is. This is the focus of the next sections,
covering studies applying static and dynamic stimuli, respectively.
Figure 2. A: Defining features of a dot lattice stimulus. B: Twodimensional space and nomenclature of dot lattices. Adapted from The
Perceptual Organization of Dot Lattices, by M. Kubovy, 1994, Psychonomic Bulletin & Review, 1, pp. 183, 184, 187. Copyright 1994 by
Springer. Adapted with permission.
Figure 3. Two dimotif rectangular dot lattices with |b|/|a| 1.2. A: The
elements are the same in the direction of the shortest distance and different
along the second shortest distance. Grouping by proximity is reinforced by
similarity. B: The elements are different in the direction of the shortest
distance and same along the second shortest distance. Grouping by proximity is opposed by similarity.
1183
1184
WAGEMANS ET AL.
Figure 4. Two grouping indifference curves. The abscissa, a, represents the difference in luminance between
adjacent elements of a. The ordinate, |b|, represents the distance between the dots of b (assuming |a| 1). Only
the equilibrium grouping indifference curve is achievable without independently measuring the strength of
grouping by proximity. The methods to be described later allow us to plot a family of indifference curves. (The
values are different for each of the four dot lattices.)
1185
Figure 6. The conjoined effects of proximity and similarity are additive. The dashed lines in Panel A turn into
grouping indifference curves in Panel B. Adapted from The Whole Is Equal to the Sum of Its Parts: A
Probabilistic Model of Grouping by Proximity and Similarity in Regular Patterns, by M. Kubovy and M. van
den Berg, 2008, Psychological Review, 115, pp. 147150. Copyright 2008 by the American Psychological
Association.
1186
WAGEMANS ET AL.
found that the spacing of the dots in the array strongly influenced
the apparent motion percept in addition to the effects of the
temporal interval between frames and the spatial displacement of
the entire array.
The influence of spatial and temporal factors in apparent motion
was further investigated using the Ternus display (Kramer &
Yantis, 1997; Pantle & Picciano, 1976; Ternus, 1926; Wallace &
Scott-Samuel, 2007), in which an array of three dots is presented
across two frames at different spatial locations. When the two
frames are presented in rapid succession (i.e., with a short interstimulus interval), it appears that the outmost dot is displaced
while the center two dots appear to be stationary: Element motion
occurs. When the temporal interval between the successive frames
is longer, the entire array of dots appears to jump: Group motion
is perceived. The two different types of perceived apparent motion
represent two different solutions to the correspondence problem
(Ullman, 1979), referring to the task of matching the objects in the
first frame to the (possibly displaced) objects in the second frame.
Whether element or group motion was perceived was found to
depend on the properties of the individual stimuli in both frames
such as their features (Dawson, Nevin-Meadows, & Wright, 1994),
their size or the sharpness of their edges (Casco, 1990), as well as
on the presence of contextual elements affecting how they are
grouped (Kramer & Yantis, 1997) or how they are perceived in
3-D space (He & Ooi, 1999).
The interaction between spatial and temporal aspects was further investigated by Gepshtein and Kubovy (2000), who were able
to determine the relationship between spatial grouping (determining which elements in each frame belong together) and temporal
grouping (determining which elements across frames belong together), by using successive presentations of dot latticesmotion
latticeswhich allowed them to independently manipulate the
strength of spatial and temporal groupings. A motion lattice (see
Figure 7) is composed of two identical dot lattices, D1 and D2,
displayed in alternation. Two ratios determine the perceived motion: (a) the motion ratio rm m2/m1, where m1 and m2 are the
shortest and the next shortest spatial distances across which the
apparent motion could occur between the frames; (b) the baseline
ratio rb b/m1, where b is the shortest spatial distance between the
dots within D1 and D2 to which the apparent motion could apply.
The orientation of a virtual line drawn through these dots is called
the baseline orientation.
As in the classic Ternus display, two classes of motion can be
perceived. First, element motion is now apparent motion from
each dot in D1 to a corresponding dot in D2 (and vice versa as
the dot lattices alternate). The log-odds of seeing m2 rather than
m1 as a function of the ratio of the distances is called an affinity
function, by analogy with the concept of an attraction function
for static dot lattices (see Figure 8A). Second, group motion is
now apparent motion orthogonal to the baseline orientation (see
Figure 7). Sequential models predict that if the spatial configuration of a stimulus remains constant, the likelihood of seeing
group motionan indicator of spatial grouping cannot be
affected by manipulations of the temporal configuration of the
stimulus. However, the pattern of interaction in Figure 8B
between rm, the temporal configuration of the stimulus, and rb,
the relative density of the dots along the baseline, clearly
refutes the sequential model.
How spatial and temporal distances interact to determine the
strength of apparent motion has been controversial. Some studies
report space-time coupling: If the spatial or temporal distance
between successive stimuli is increased, the other distance between
them must also be increased to maintain a constant strength of
apparent motion (i.e., Kortes third law of motion). Other studies
report space-time tradeoff: If one of the distances is increased, the
other must be decreased to maintain a constant strength of apparent
motion. To establish what determines whether coupling or tradeoff
occurs, Gepshtein and Kubovy (2007) generalized the motion
lattice of Figure 7, as illustrated in Figure 9, showing a temporal
component of m3, T3, of twice the magnitude of the temporal
component of m1, T1. By manipulating the spatial components of
these motions, S3 and S1 from S3 S1 to S3 S1, an equilibrium
point between the extremes was found at r31 S3/S1, for which the
probability of seeing the two motions was the same. If r31 1,
then space-time coupling holds; if r31 1, then space-time
tradeoff holds. This suggests that previous findings on apparent
motion were special cases and that the allegedly inconsistent
results can be embraced by a simple law in which a smooth
transition from tradeoff to coupling occurs as a function of speed:
Tradeoff holds at low speeds of motion (below 12/s), whereas
coupling (Kortes law) holds at high speeds. The deeper theoretical
implications of these results for the visual systems economy
principles are discussed in the second review article (Wagemans et
al., 2012, Section 4).
The above research on perceptual grouping in static and
dynamic discrete patterns spans a complete century, from Schumann (1900), Wertheimer (1912, 1923), and Korte (1915) up
until today. It mainly used well-controlled, parametrically varied stimuli in order to isolate one factor or another, and trying
to quantify its strength. In addition, it sparked a renewed
interest in understanding the level at which perceptual grouping
operates, which was addressed in studies that used somewhat
richer stimuli with additional variations, more typical for naturally occurring stimulation.
Figure 8.
1187
Figure 9. A six-stroke motion lattice. A: The successive frames are superimposed in space. Gray levels indicate
time. b is the baseline distance. B: The time course of the display. The three most likely motions along m1, m2,
and m3 can occur because dots in frame fi can match dots in either frame fi 1 or frame fi 2. CD: Conditions
in which different motion paths dominate: m1 in Panel C and m3 in Panel D. (The stimuli were designed so that
m2 would never dominate.) Adapted from The Lawful Perception of Apparent Motion, by S. Gepshtein and
M. Kubovy, 2007, Journal of Vision, 7(8), Article 9, p. 7. Copyright 2007 by the Association for Research in
Vision and Ophthalmology. Adapted with permission.
WAGEMANS ET AL.
1188
Figure 10. Object boundaries project to the image as fragmented contours, due to occlusions (dashed cyan line) and low figure/ground contrast
(dashed red line).
1189
the principle of good continuation in cluttered images was developed by Field, Hayes, and Hess (1993). In this method, a contour
formed from localized oriented elements is embedded in a random
field of homogeneously distributed distractor elements, in order to
eliminate the role of proximity (see Figure 11). Aligning the
contour elements tangentially to the contour makes the contour
easily detected, whereas randomizing the orientation of the elements renders the contour invisible, clearly demonstrating the role
of good continuation in isolation from proximity. These findings
led Field et al. to suggest the notion of an association field that
determines the linking of oriented elements within a local visual
neighborhood (see Figure 12), a construct that is closely related to
the machinery of cocircularity support neighborhoods, developed
earlier for the purpose of contour refinement in computer vision
(Parent & Zucker, 1989).
Ecological data on good continuation have also emerged over
the last decade. Kruger (1998) and later Sigman et al. (2001) found
evidence for collinearity, cocircularity, and parallelism in the statistics of natural images. Geisler, Perry, Super, and Gallogly
(2001) found similar results using both labeled and unlabeled
natural image data, in fairly close correspondence with the tuning
of human perception to the good continuation cue. Geisler et al.
treated contours as unordered sets of oriented elements, measuring
the statistics for pairs of contour elements on a common object
boundary, regardless of whether these element pairs were close
together or far apart on the object contour. In contrast, Elder and
Goldberg (2002) modeled contours as ordered sequences of oriented elements, restricting measurements to adjacent pairs of oriented elements along the contours. The likelihood ratios for two
Figure 11. Example of stimuli devised by Field, Hayes, and Hess (1993)
to probe the role of good continuation in contour integration. Adapted from
Contour Integration by the Human Visual System: Evidence for a Local
Association Field, by D. J. Field, A. Hayes, and R. F. Hess, 1993, Vision
Research, 33, p. 177. Copyright 1993 by Elsevier. Adapted with permission.
1190
WAGEMANS ET AL.
Figure 12. Models of good continuation. A: Cocircularity support neighborhood. B: Association field. Panel A adapted from Trace Inference,
Curvature Consistency, and Curve Detection, by P. Parent and S. W.
Zucker, 1989, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, p. 831. Copyright 1989 by IEEE. Adapted with permission.
Panel B adapted from Contour Integration by the Human Visual System:
Evidence for a Local Association Field, by D. J. Field, A. Hayes, and
R. F. Hess, 1993, Vision Research, 33, p. 190. Copyright 1993 by Elsevier.
Adapted with permission.
The original Gestalt claim was thus not that closure is a grouping
cue per se, but rather that it somehow profoundly determines the
final percept of form. In the same spirit, Elder and Zucker (1993,
1994) argued that the most important role for closure was as a
bridge from 1-D contour to 2-D shape, a suggestion that was
supported by the finding that small changes in good continuation
and closure can yield large changes in shape discriminability.
Symmetry and parallelism. The Gestaltists identified symmetry as a factor of good shape (Koffka, 1935), although it seems
to be easily overruled by good continuation and convexity
(Kanizsa, 1979). In the computer vision literature, symmetry has
been used in numerous contour integration algorithms (e.g., Mohan & Nevatia, 1992; Stahl & Wang, 2008). Parallelism has been
identified as a factor determining the perceptual simplicity of line
configurations (Arnheim, 1967), and as a grouping cue in computer vision algorithms (e.g., Jacobs, 2003; Lowe, 1985). In the
ecological statistics literature, parallelism has been studied extensively as a cue for the grouping of oriented edge elements into
contours (Elder & Goldberg, 2002; Geisler et al., 2001; Kruger,
1998). Psychophysical evidence for the role of symmetry and
parallelism in contour integration has been reported by Feldman
(2007), who showed that comparison of features lying on pairs of
line segments is significantly faster if the segments are parallel or
mirror symmetric, suggesting a fast grouping of the segments
based upon these cues. More recently, Machilsen, Pauwels, and
Wagemans (2009) demonstrated enhanced detectability of bilaterally symmetric versus asymmetric closed forms, suggesting a role
for more complex, global symmetry processing in contour integration. Physiologically, it is known that bilaterally symmetric patterns differentially activate human extrastriate visual areas V3, V4,
V7 and LO, and homologous areas in macaque cortex (Sasaki,
2007).
Convexity. Convexity has long been known as a grouping
cue (Rubin, 1927). In the computer vision literature, Jacobs (1996)
also demonstrated its utility for grouping contour fragments that
can then be used as features for object recognition. Liu, Jacobs,
and Basri (1999) subsequently employed a novel psychophysical
Contour Completion
Modal and amodal completion. Whereas contour integration refers to the integration of discrete elements such as dots or
oriented elements, contour completion refers to the integration of
smooth extended contours, separated by gaps due to occlusion or
camouflage. Stimuli used in the study of contour completion often
include cues to relative depth or occlusion, such as T junctions (see
Figure 13). In Figure 13A, for example, the perceived unity of the
black fragments and the perceived shape of the black surface
behind the gray occluder appear to result from visual contour
1191
Figure 13. A: Amodal completion of the black shape behind the gray shape. B: A white shape seen on top of
three black shapes. The perceived contours of this white shape have a sensory quality (hence, modal completion),
although they are completely illusory. Adapted from Bayesian Contour Extrapolation: Geometric Determinants
of Good Continuation, by M. Singh and J. M. Fulvio, 2007, Vision Research, 47, p. 784. Copyright 2007 by
Elsevier. Adapted with permission.
1192
WAGEMANS ET AL.
Figure 14. Two curved fragments are seen to complete amodally behind the gray rectangle in Panel A, not in
Panel B. C: In case of amodal completion, an important issue regards the shape of the completed curve. Adapted
from Bayesian Contour Extrapolation: Geometric Determinants of Good Continuation, by M. Singh and J. M.
Fulvio, 2007, Vision Research, 47, p. 784. Copyright 2007 by Elsevier. Adapted with permission.
contours (Takeichi, Nakazawa, Murakami, & Shimojo, 1995; Ullman, 1976). Other studies have asked observers to mark the
farthest point along the interpolated contours (Fantoni, Bertamini,
& Gerbino, 2005; Guttman & Kellman, 2004; Takeichi et al.,
1995), to match the perceived interpolated shape from a parametric
family (Singh, 2004), or to rate its overall degree of roundedness
(Fantoni & Gerbino, 2003). More recently, Fulvio et al. (2008)
used a method that obtains interpolation settings (position and
orientation) at multiple locations along the length of a partly
occluded contour. An examination of the precision and consistency
of these settings showed that observers can interpolate a single,
stable (self-consistent), smooth contour when the inducing contours are relatable, but not when they require an inflecting interpolating curve.
An empirically successful model that addresses the grouping
problem and the shape problem simultaneously has been proposed
by van Lier (1999; van Lier, van der Helm, & Leeuwenberg, 1994,
1995). In line with the descriptive simplicity principle (see Wagemans et al., 2012, Sections 5 and 6), this model postulates that an
interpretation is judged not only by the complexities of the potentially occluding and occluded shapes but also by the complexity of
the relative positions of these shapes. The combination with the
lowest overall complexity is then taken to predict whether occlusion occurs, and if so, how the occluded shape looks.
Contour interpolation and extrapolation. Good continuation is an important criterion in shape interpolation, but it is not
perfectly clear what is meant by this term. Dating back to
Wertheimer (1923, especially his Figures 16 19), two aspects
have been distinguished: (a) What geometric properties of the
visible contours are used by human vision (e.g., orientation, curvature, rate of change of curvature, higher order derivates), and (b)
how are they combined to define the shape of the contour?
These questions were addressed in studies of contour extrapolation (e.g., Singh & Fulvio, 2005, 2007), in which observers
adjusted the position and orientation of a line probe on the other
side of an occluder (see Figure 15) at multiple distances from the
point of occlusion in order to optimize the percept of smooth
continuation. The findings suggested that observers use contour
orientation and curvature, but not rate of change of curvature. The
shape of visually extrapolated contours was best characterized by
a decaying curvature. The results were modeled by a Bayesian
interaction between a tendency to extend contour curvature and a
prior tendency for straight contours.
Figure 15. A: Measuring extrapolation of curvature by (Panel B) asking observers to position and orient a
small curved line fragment. Adapted from Bayesian Contour Extrapolation: Geometric Determinants of Good
Continuation, by M. Singh and J. M. Fulvio, 2007, Vision Research, 47, pp. 784, 787. Copyright 2007 by
Elsevier. Adapted with permission.
1193
Figure 16. Contour geometry depends on surface geometry. The same curved contour segment (Panel A) can
correspond either to a locally convex (Panel B) or a concave portion of a surface (Panel C). Curvature on an
object boundary can also arise because the axis of the object itself is curved (Panel D). Adapted from Surface
Geometry Influences the Shape of Illusory Contours, by J. M. Fulvio and M. Singh, 2006, Acta Psychologica,
123, p. 23. Copyright 2006 by Elsevier. Adapted with permission.
1194
WAGEMANS ET AL.
suggesting that not all grouping cues are readily available to young
infants.
Furthermore, recent psychophysical studies of older children
suggest that there is a protracted developmental trajectory for some
perceptual organization abilities, even those that appear to emerge
during infancy (e.g., Hadad & Kimchi, 2006; Hadad, Maurer, &
Lewis, 2010a, 2010b; Kimchi, Hadad, Behrmann, & Palmer, 2005;
Kovacs, 2000). For example, the Gestalt principles underlying
contour grouping continue to develop in children through their late
teens. These findings suggest that visual experience plays a role in
at least some aspects of perceptual organization. Further support
for this notion comes from studies demonstrating influence of
perceptual learning on organizational abilities in infancy (e.g.,
Quinn & Bhatt, 2005), and the influence of associative learning
(Vickery & Jiang, 2009) and past experience (Kimchi & Hadad,
2002) on perceptual grouping in adult observers.
Cue combination. One of the central questions in grouping
concerns the way in which the brain combines multiple cues to
yield a unitary organization. Historically, this problem has often
been posed in terms of competitive interactions formulated either
in descriptive terms (usually seeking compliance with the simplicity principle) or in probabilistic terms (usually Bayesian formulations that may or may not seek compliance with the Helmholtzian
likelihood principle). (For more details on this, see Wagemans et
al., 2012, Sections 5 and 6.) However, in natural scenes, disparate
weak cues can often combine synergistically to yield strong evidence for a particular grouping.
To explore this issue, Geisler et al. (2001) used a nonparametric
statistical approach, jointly modeling the ecological statistics of
proximity and good continuation cues as a 3-D histogram, to show
that human observers combine these two classic Gestalt principles
in a roughly optimal way. Elder and Goldberg (2002) demonstrated that the ecological statistics of proximity, good continuation, and similarity are roughly uncorrelated, so that to a first
approximation the Gestalt laws can be factored: The likelihood of
a particular grouping can be computed as the product of the
likelihoods of each grouping factor in turn (or, equivalently, the
log likelihood of the grouping factor is the sum of the log likelihoods of the grouping factors; for further explanation, see Claessens & Wagemans, 2008).
Elder and Goldbergs (2002) approach also allowed quantification of the statistical power of each Gestalt cue, as the reduction
in the entropy of the grouping decision given each cue individually. They found that proximity was by far the most powerful,
reducing the entropy by roughly 75%, whereas good continuation
and similarity reduced entropy by roughly 10% each. The most
accurate grouping decisions could be made by combining all of the
cues optimally according to the probabilistic model, trained on the
ecological statistics of natural images. Such a statistically optimal
combination of grouping cues has also received some psychophysical support (Claessens & Wagemans, 2008).
Computational models.
Numerous computer vision algorithms for grouping make use of the Gestalt factors described
above (e.g., Elder et al., 2003; Estrada & Elder, 2006; Jacobs,
1996; Lowe, 1985; Shaashua & Ullman, 1988; Stahl & Wang,
2008). Most of these use the local Gestalt principles of proximity,
good continuation, and similarity to group long chains under an
explicit or implicit Markov assumption, while additional global
factors of convexity (Jacobs, 1996), closure (Elder et al., 2003),
Conclusion
Research on principles of perceptual grouping has come a long
way since Wertheimers (1923) phenomenological demonstrations. Using well-controlled discrete patterns, grouping by proximity could be quantified as a real law. In the last 2 decades or so,
new grouping principles have been identified, and their functional
role in the processing of more complex images occurring in
real-world situations has become a central focus. In much of this
research, quantitative psychophysical work went hand in hand with
the analysis of ecological statistics, the development of computational models, and neurophysiological research. In Section 5, we
see a similar evolution in research on the closely related problem
of figure ground organization. Section 6 is devoted to an integrated review of the research on neural mechanisms of contour
grouping and figure ground organization.
FigureGround Organization
Introduction
Two adjoining regions of the visual field with a shared border
can lead to a mosaic percept, but more often lead to figure ground
organization. In this percept, the shared border is perceived as the
occluding edge of one of the regions. The occluding region is
perceived as the figure, and it appears shaped by the border. The
adjoining region appears to simply continue behind it as its background, with no shape being imparted by the border. The figure is
said to own the borderline. When figure ground reversals occur,
the border ownership switches as well (e.g., the famous vasefaces
figure by Rubin, 1915). Why switching occurs and how its temporal characteristics inform us about the dynamics of the brain are
discussed extensively in the second article (Wagemans et al., 2012,
Section 3). Here, we focus on the factors or principles that determine what is perceived as figure.
1195
Figure 17. Example of a display used in classic tests of whether or not convexity serves as a configural
figure ground principle. Here the black regions have convex parts, and the white regions have concave parts.
Regions with convex parts were black in half of the test displays and white in the other half. Adapted from
Inhibitory Competition in Figure-Ground Perception: Context and Convexity, by M. A. Peterson and E.
Salvagio, 2008, Journal of Vision, 8(16), Article 4, p. 2. Copyright 2008 by the Association for Research in
Vision and Ophthalmology. Adapted with permission.
1196
WAGEMANS ET AL.
1197
1198
WAGEMANS ET AL.
Figure 18. AB: Sample stimuli used by Peterson, Harvey, and Weidenbacher (1991). The configural factors
of small area, symmetry, and enclosure favor seeing the central, black region as figure. In Panel A, a portion of
a familiar object, a standing woman, is suggested on the outside of the left and right borders of the black region.
Panel B is an inverted version of Panel A. CD: Sample upright (Panel C) and inverted (Panel D) bipartite
displays used by Peterson and Gibson (1994a). EG: Sample stimuli used by Peterson and Skow (2008). The
configural factors of small area, symmetry, and enclosure favor seeing the inside of the black silhouettes as the
figures. Portions of familiar objects are suggested on the outsides of the silhouettes left and right borders (in
Panel E, sea horses; in Panel F, table lamps; in Panel G, pineapples). Panels AB adapted from Shape
Recognition Contributions to FigureGround Reversal: Which Route Counts?, by M. A. Peterson, E. M.
Harvey, and H. J. Weidenbacher, 1991, Journal of Experimental Psychology: Human Perception and Performance, 17, p. 1077. Copyright 1991 by the American Psychological Association. Panels CD adapted from
Must Figure-Ground Organization Precede Object Recognition? An Assumption in Peril, by M. A. Peterson
and B. S. Gibson, 1994, Psychological Science, 5, p. 254. Copyright 1994 by Sage Publications. Adapted with
permission. Panels EG adapted from Inhibitory Competition Between Shape Properties in FigureGround
Perception, by M. A. Peterson and E. Skow, 2008, Journal of Experimental Psychology: Human Perception and
Performance, 34, p. 254. Copyright 2008 by the American Psychological Association.
the familiar object that lost the competition were suppressed for a
brief period of time after figure ground perception was achieved,
providing evidence that past experience plays a role in figure
ground segregation even when it does not dominate (see also
Trujillo, Allen, Schnyer, & Peterson, 2010, for a study measuring
the associated event related potentials). Recently, Navon (2011)
conducted a series of experiments that he took as evidence that the
familiarity of objects may exert an influence even earlier, when
prefigure assignment parsing occurs.
Taken together, these findings from studies that satisfy both of
Wertheimers (1923) criteria show that past experience can exert
an influence on several aspects of figure ground perception.
Attention and perceptual set. Before any behavioral evidence was obtained that attention affects which region will be
perceived as figure, Kienker, Sejnowski, Hinton, and Schumacher
1199
Figure 19. Sample displays used by Driver and Baylis (1996). A: Study display. BC: Figure and ground
probes, respectively. In both Panel B and Panel C, the top probe has the same border as the study display.
Adapted from Edge-Assignment and FigureGround Segmentation in Short-Term Visual Masking, by J.
Driver and G. C. Baylis, 1996, Cognitive Psychology, 31, p. 254. Copyright 1996 by Elsevier. Adapted with
permission.
(1986; Sejnowski & Hinton, 1987) modeled figure ground segregation as cross-border inhibitory competition that could be biased
by attention. A decade later, Baylis and Driver (1995) presented
empirical evidence for the role of attention. They found that when
observers allocated their attention endogenously to one of two
regions sharing a border, they were more likely to perceive the
attended region as figure than the unattended region. Similar but
weaker effects of exogenous attention were later obtained (Vecera
et al., 2004). In addition, it has been shown that the viewers
intention, or perceptual set, to perceive one of two regions sharing
a border as figure can influence figure assignment (Peterson et al.,
1991; Peterson & Gibson, 1994b). Thus, there is ample evidence
that observer-dependent factors can influence figure assignment.
The effects of attention and perceptual set contradict claims that
figure ground perception always occurs early in the visual system.
Indeed, although figure ground perception can be affected by
focused attention, there is evidence that it can also occur preattentively. Preattentive processes are considered early in that they
are carried out before attention is focused on individual regions in
the visual field, that is, when attention is either distributed broadly
or allocated to a different task. For example, in a case study of a
patient with left neglect (a failure to attend to the left side of
objects and spaces), it was found that judgments of the symmetry
of individual shapes in the left hemisphere were severely impaired
(Driver et al., 1992). Nevertheless, when asked to make figure
ground judgments regarding six-region displays with green and red
alternating symmetric and asymmetric regions, the patient, like
controls, showed a bias to see symmetric regions as figures,
suggesting that the patient could use symmetry preattentively for
figure ground segregation even though he could not detect it
postattentively. Convexity-based figure ground segregation can
also occur preattentively. Kimchi and Peterson (2008) asked participants to perform a demanding change detection task on a
stimulus presented on a task-irrelevant background of alternating
regions of figures and grounds by convexity. They found that
changes in the organization of the background produced congruency effects on target-change judgments, even though participants
were unable to report the figure ground status of the background.
Taken together, these findings are consistent with the claim that
figure ground segregation can occur before focal attention is
directed to making a figure ground judgment. Thus, figure
ground segregation can occur preattentively, but it can also be
affected by attention.
1200
WAGEMANS ET AL.
Figure 20. When a wiggly curved line is drawn on a circular disc, the two halves arising from this divide
appear to have a bounding contour with a different shape. Adapted from Multistability in Perception, by F.
Attneave, 1971, Scientific American, 225(6), p. 68. Copyright 1971 by Scientific American.
Building on this account, Hoffman and Singh (1997) proposed that the relative salience of the two sets of parts, one at
each side of the borderline, plays an important role in determining figure and ground. (In this respect, it could be considered one of the new image-based principles of figure ground
organization reviewed before.) The perceptual salience of a
part, or part salience for short how much it visually stands out
as a separate partis determined by a number of geometric
factors, including the curvature (sharpness) of its part boundaries, and its protrusion (the degree to which it sticks out,
measured as perimeter/cut length). Typically, the side with the
more salient parts is assigned figural status, as illustrated in
Figure 21. In this figure, Panel a shows the original contour,
Panel b shows the parts obtained from the two sides, and Panel
c shows the side with more salient parts, which tends to be
perceived as figure. Because part salience is based on relatively
local computations, figure and ground can perceptually reverse
along a continuous contour (see Figure 21d). This suggests that
shape analysis occurs not only on the figure sideafter figure
ground assignment has taken place but actually precedes it,
and it plays a crucial role in making the figure ground assign-
Figure 21. The role of part salience in figure ground organization. Adapted from Salience of Visual Parts,
by D. D. Hoffman and M. Singh, 1997, Cognition, 63, p. 47. Copyright 1997 by Elsevier. Adapted with
permission.
Conclusion
Research on figure ground organization has also come a long
way since Wertheimers (1923) paper on the Gestalt laws. First,
when studied in more controlled experiments, the classic configural principles turn out weaker than previously supposed. Second,
using richer displays, new figure ground principles have been
discovered with a plausible ecological foundation. Third, the role
of past experience and attention has now been clearly demonstrated in experiments that satisfy Wertheimers own criteria.
Finally, figure ground organization is no longer studied in isolation but turns out to be intimately related to other processes such
as shape and depth perception. These connections are currently
also the focus of interesting research on the neural mechanisms of
contour grouping and figure ground organization, as reviewed
next.
1201
WAGEMANS ET AL.
1202
ler, & Kubler, 1998; Peterhans & von der Heydt, 1989). Under
natural conditions, objects often occlude one another, and detecting the borders between image regions corresponding to foreground and background objects is a basic task of vision. V2
mechanisms might combine two sources of evidence for occluding
contours: the presence of a luminance/color edge and the presence
of occlusion features along the contour. Luminance/color differences can be detected by simple and complex cells and occlusion
features (i.e., terminations of background structures) by endstopped cells (Heitger, Rosenthaler, von der Heydt, Peterhans, &
Kubler, 1992). By integrating inputs from simple/complex cells
and appropriately selected end-stopped cells, V2 neurons can
signal occluding contours more reliably than mechanisms based on
edge detection alone (Heitger et al., 1998).
This model can be considered as a simple implementation of the
Gestalt completion principle according to which illusory contours
are formed in situations in which unbalanced shapes tend to
completion (Kanizsa, 1979; Michotte et al., 1964). For example,
the contours of an illusory white square in a configuration with
four black pacmen are thought to emerge because the disc sectors
tend to complete to circular discs (see Figure 22A). In the model,
the terminations of the circular edges activate end-stopped cells.
This mechanism allows for the possibility that the terminations are
caused by occlusion and that the edges might continue in the
background. While the principle of combining edge evidence with
evidence from occlusion features for the definition of contours is
valid, the model might be too simplistic because it fails to account
for illusory contour perception in stereoscopic displays (Gillam &
Nakayama, 2002). Also, models of illusory contours in Kanizsa
figures should be able to explain why the illusion disappears in
other configurations (e.g., four crosses instead of four pacmen; see
Figure 22B), which is a clear indication of the important role of the
whole stimulus configuration, in line with the basic tenets of
Gestalt psychology (see Kogo, Strecha, Van Gool, & Wagemans,
2010, discussed further in the next section).
Figure 22. In a configuration with four black pacmen, an illusory white square emerges in the center, which
does not happen when the same local edges occur in a configuration with four black crosses.
degrees of visual angle (Peterhans & von der Heydt, 1989). However, the spatial range at which figure ground organization occurs
is impressive, given the small receptive fields in the visual cortex.
Even the extent of the nonclassic surrounds, which is typically
about twice the size of the classic receptive field (see Cavanaugh,
Bair, & Movshon, 2002; Levitt & Lund, 2002), is insufficient to
explain the spatial range of figure ground organization. It was
therefore a surprise when Lamme (1995) discovered that neurons
in monkey V1 show enhanced responses in figure regions compared to ground regions (see also T. S. Lee, Mumford, Romero, &
Lamme, 1998; Zipser, Lamme, & Schiller, 1996). The figures in
these studies were defined by a difference in texture, which could
be based on orientation, motion, depth, or color, suggesting a
general mechanism. Although the figure ground modulation decreased with figure size and disappeared for figures larger than 8,
the range of context integration was clearly much larger than the
typical size of the receptive fields (but see Rossi, Desimone, &
Ungerleider, 2001).
While Lammes (1995) finding suggests coding in terms of
regions, perceptual studies also emphasize the distinctive role of
the borders between regions. Bregman (1981) has published a
striking demonstration of this: Figure 23a shows a number of
apparently meaningless shapes, while Figure 23b shows exactly
the same shape fragments, but with a blotch of ink present which
is removed in Figure 23a. Because the blotch is seen as a foreground object, it takes ownership of some of the borders of the
original shapes, which now organize to letters due to amodal
completion of the shape fragments behind the blotch. The crucial
role of border ownership in image interpretation was rediscovered
by Nakayama, Shimojo, and Silverman (1989), and it was also
recognized in computational modeling studies (Finkel & Sajda,
1992), but how border ownership might be coded by neurons was
less clear until recordings from monkey visual cortex by H. Zhou,
Friedman, and von der Heydt (2000) revealed that the firing rate of
orientation-selective neurons that respond to the contours of figures depended on where the figure is located relative to the
receptive field of the neurons. Moreover, each neuron was found to
1203
have a fixed preference for direction of figure, while across neurons all directions in the visual field were represented equally. As
a consequence, each border is represented by two groups of neurons, one for each side of ownership. Zhou et al. suggested that the
differential activity between the two represents the borderownership assignment. In area V2, about half of the oriented
neurons are border-ownership selective. Such neurons can also be
found in V1 and V4, but less frequently.
Two characteristics of border-ownership coding are remarkable:
the large range of context integration and the short latency of the
differential response. The context influence extends far beyond the
classic receptive field, and it appears less than 30 ms after
the earliest responses in V1 or V2 (60 70 ms after stimulus onset),
which is earlier than the figure ground enhancement in V1
(Lamme, 1995). Tests with fragmented figures showed that most
or all segments of the figure contours contribute (Zhang & von der
Heydt, 2010). These findings have strong implications for possible
neural mechanisms, in particular for the role of horizontal interactions, which is often postulated in models of figure ground
organization (e.g., Finkel & Sajda, 1992; Grossberg, 1994; Kogo et
al., 2010; Sajda & Finkel, 1995; Zhaoping, 2005). Close examination of the conduction delays shows that the horizontal propagation scheme is incompatible with the large range of context
integration and the short latency of the context influence (Zhang &
von der Heydt, 2010; for a discussion, see Craft, Schutze, Niebur,
& von der Heydt, 2007). There is little evidence for propagation
delays in context integration for border ownership. Instead, Zhang
and von der Heydt (2010) found that the influence of the most
distant contour segments arrived earlier than that of the segments
closest to the receptive field. Hence, it is more likely that context
integration involves feedback from higher level visual areas (e.g.,
Craft et al., 2007; Jehee, Lamme, & Roelfsema, 2007; Roelfsema,
Lamme, Spekreijse, & Bosch, 2002): Neurons in these areas have
larger receptive fields and signals between areas travel through
white matter fibers, which are much faster than the intracortical
horizontal fibers. These models are in agreement with studies
Figure 23. A: A quasi-random collection of quasi-random shapes. B: The same shapes as in Panel a with a
black ink blotch, which is seen to occlude five letters B. Adapted from Asking the What for Question in
Auditory Perception, by A. S. Bregman, in Perceptual Organization (pp. 106 107), ed. by M. Kubovy and J. R.
Pomerantz, 1981, Hillsdale, NJ: Erlbaum. Copyright 1981 by Lawrence Erlbaum Associates. Adapted with
permission.
1204
WAGEMANS ET AL.
showing that the far surround of receptive fields in V1 is contributed by feedback (Angelucci et al., 2002).
The involvement of border ownership and feedback in figure
ground organization have much to say about its relevance in
everyday vision. Figure ground organization is obviously related
to the task of interpreting 2-D images in terms of a 3-D world,
which is fundamental to vision. What Gestalt psychology has
revealed is the compulsion of the system to seek a 3-D interpretation, even under the impoverished conditions of simple drawings
that do not provide information about depth. When explicit depth
information is available, for example, in stereoscopic vision, it has
a profound influence on image segmentation and perceptual organization (Gregory & Harris, 1974; Nakayama et al., 1989; Shimojo, Silverman, & Nakayama, 1989). The influence of stereoscopic depth is an important criterion for the validity of the
interpretation of the neural signals. If response enhancement and
border-ownership modulation indeed participate in the attempt to
fit a 3-D interpretation, then these signals should respond to border
ownership given by disparity as well as by 2-D cues, but if they
reflect general receptive field mechanisms (e.g., centersurround
antagonism), there would be no reason for this parallelism. Several
neurophysiological studies have shown that stereoscopic depth can
indeed dramatically influence the cortical responses to a visual
stimulus, even in early visual regions (Bakin, Nakayama, & Gilbert, 2000; Duncan, Albright, & Stoner, 2000; Qiu & von der
Heydt, 2005; Zipser et al., 1996).
For instance, applying Lammes figure ground paradigm to
random-dot stereograms, Zipser et al. (1996) showed that the
enhancement is not simply the result of surroundedness, but depends on the conditions of figure perception. When a surface in the
fixation plane (zero disparity) is surrounded by a region of far
disparity, it assumes figure status and produces enhancement of
responses; it appears as an object floating in front of a background.
In contrast, when it is surrounded by near disparity, it assumes
ground status and no enhancement occurs; the surrounded region
now appears as a surface in the back that is seen through a window.
Stereoscopic depth also has a strong influence on the coding of
border ownership. Qiu and von der Heydt (2005) found that many
V2 cells combines border-ownership selectivity (for contrastdefined figures without disparity) with stereo-edge selectivity in
random-dot stereograms, and the preferred side of the figure was
generally the same as the preferred foreground side in the stereograms. This means that the cortex interprets a contrast-defined
figure as an object occluding a background, in line with the Gestalt
view.
The above findings have recently been integrated in a neurocomputational model (Kogo et al., 2010) in which border ownership and occlusion play a central role in the emergence of illusory
surfaces and contours in Kanizsa-type displays (see Figure 22
again). Specifically, local occlusion cues (L and T junctions)
trigger a perceived depth difference and border-ownership signals
at the visible edges, which then spread and combine with borderownership signals in the open space between the pacmen. This
spreading and enhancement of border ownership only occurs with
particular global configurations of the visible elements (the pacmen) that are consistent with a central surface (e.g., not in the
four-crosses configuration). In contrast with many previous models, this model is not aimed at contour completion based on the
collinearity of the local image fragments. Instead, it achieves
Figure 24. The isolated square (Panel a) and the squares with rounded
corners (Panel b) appear as distinct objects, but when the squares come in
contact at the corners, two bars in transparent overlay are perceived (Panel
c). The small alteration of contours results in a perceptual reorganization.
Note the reversal of border ownership at the marked edge. The population
border ownership signal in the visual cortex (area V2) also shows this
reversal (curves below). The dashed ellipses mark the receptive field
positions.
1205
from the vase region to a face region and vice versa (although
spontaneous, automatic switches occur as well).
The interactions between attention and border ownership may
seem complicated, but there is an additional observation that
suggests that the underlying mechanism is simple: The side of
attentive enhancement and the preferred side of border ownership
of a neuron tend to be the same (Qiu et al., 2007). This indicates
that the same neural circuits that make a neuron border-ownership
selective also produce the modulation by attention. One can explain all the above results by assuming that the edge signals
produced by a figure are summed by a common grouping cell that,
by feedback, sets the gain of the corresponding edge neurons
(Craft et al., 2007). Each border-ownership selective V2 neuron is
connected to grouping cells on one side of its receptive field, and
therefore shows enhanced responses when a figure is present on
that side. The effects of attention are explained by assuming that
volitional attention activates the grouping cells corresponding to
the object to be attended (Mihalas, Dong, von der Heydt, &
Niebur, 2011). This raises the gain of the connected edge neurons.
The one-sided connectivity of the edge neurons accounts for the
asymmetric attention effect, and, because the same grouping cells
produce border ownership and attentive modulation, the side of
attentive enhancement is also the preferred side of border ownership.
Conclusion
In sum, the neurophysiological evidence from the last 2 decades
seems to converge on the idea that the responses of cortical
neurons depend on the properties of the overall configuration in
the visual field as well as on the parameters of the stimulus in its
receptive field. The connectivity and rules of the visual cortex
allow illusory contours to be formed and figure ground segmentation to be performed by autonomous processes that are at the
same time also context sensitive. The segregation of automatic and
volitional processing closely resembles what the pioneers of Gestalt psychology envisioned, and many of the details currently
discovered show parallels to the perceptual phenomena that they
pointed out. As indicated by Westheimer (1999), neurophysiology
has come a long way since Hubel and Wiesels (1968) atomistic
approach to orientation selectivity of single cells in cat and monkey cortex, which were taken as prototypical feature detectors. The
literature reviewed above matches well with the premature physiological theory postulated by Wertheimer (1922/1938a, p. 15):
The cells of an organism are parts of the whole and excitations
occurring in them are thus to be viewed as part-processes functionally related to whole-processes of the entire organism. Indeed,
it gives a concrete meaning to it, by emphasizing the role of
context-sensitive, autonomous processes within recurrent networks.
1206
WAGEMANS ET AL.
1207
Table 5
Current More Synthetic Positions
Some Gestalten are strong Gestalten, others are weak Gestalten.
Some Gestalten arise suddenly and as immediately organized wholes, but sometimes it is useful to examine gradual emergence or microgenesis.
Grouping principles operate at multiple levels: Grouping occurs both pre- and postconstancy.
Grouping principles sometimes combine additively, sometimes nonadditively.
Historically, Gestalt psychology has emphasized the degree to which the Gestalt laws are innate or intrinsic to the brain rather than learned from past
experience, but there is now also a lot of attention to the development of perceptual organization and to the role of past experience. For instance,
we now know that not all grouping cues are readily available to young infants and that there is a protracted developmental trajectory for some
perceptual organization abilities, even those that appear to emerge during infancy.
How the brain combines multiple cues to yield a unitary organization has often been posed in terms of competitive interactions formulated either in
descriptive terms (usually seeking compliance with the simplicity principle) or in probabilistic terms (mostly Bayesian formulations which may or
may not seek compliance with the Helmholtzian likelihood principle). In natural scenes, however, disparate weak cues can often combine
synergistically to yield strong evidence for a particular grouping.
Figureground organization is driven by image-based cues as well as by subjective factors such as past experience (familiarity), attention, and
perceptual set.
Figureground organization does not always occur early in the visual system; it can be affected by focused attention, but it can also occur
preattentively.
Neurophysiological studies investigating whether perceptual organization processes are preattentive, are influenced by attention, or take place only
under attention have produced mixed results. In some situations, attention initiates a process of organization that reflects the intrinsic connectivity of
the cortex. In other situations, organization emerges independently of attention, creating a structure for selective attention.
The neurophysiological evidence from the last 2 decades seems to converge on the idea that the responses of cortical neurons depend on the properties
of the overall configuration in the visual field as well as on the parameters of the stimulus in its receptive field. The connectivity and rules of the
visual cortex allow illusory contours to be formed and figureground segmentation to be performed by autonomous processes that are at the same
time also context sensitive.
1208
WAGEMANS ET AL.
Table 6
Limitations and Challenges to Contemporary Research on Perceptual Organization
There is a clear need for a systematic analysis of the factors that are common to both perceptual grouping and figureground organization, and of the
factors that are specific to one of them.
A thorough examination of the specific task requirements induced by the stimulus and imposed by the instructions is needed to be able to determine
the processes involved and the potential generalization beyond the test conditions.
Further progress with respect to theoretical integration between different processes of perceptual organization will depend on experiments that bridge
the gaps between different experimental paradigms.
Progress regarding figureground organization could profit from a more fine-grained analysis of the different components involved.
Linking experiments are needed to facilitate an integration of grouping and segregation processes into the figureground organization literature.
A fundamental limitation of current research on perceptual grouping as well as figureground organization is the shortage of computational process
models.
Progress in developing and testing neurocomputational models, which are supposed to rely on solid computational principles that are compatible with
known neurophysiology and human psychophysics, requires painstaking bridging efforts by multidisciplinary teams (e.g., psychophysicists,
modelers, neurophysiologists, and neuroanatomists).
1209
Low-level vision can also relate more directly to neurophysiological findings than mid-level vision. Although we believe in the
value of a thorough phenomenological analysis of interesting
perceptual phenomena (e.g., Sinico, 2008), we are also convinced
that explanations at a neural level provide considerable added
value (e.g., Spillmann, 2009). Single-cell responses of V1 are very
well characterized in relation to well-controlled low-level stimulus
characteristics, and their response properties and tuning are relatively well understood (for a review, see Carandini et al., 2005).
Where context effects or more realistic configurations are concerned, the combinatorial space of stimulus attributes becomes
prohibitive for similar parametric studies (but see Brincat & Connor, 2004, 2006, for a most interesting attempt).When true Gestalt
experiences are at stake, either fundamental limitations of the
research methods (fMRI or EEG) or problems with cross-species
comparisons (from monkey to human) are unavoidable. As a
result, success stories regarding direct neural correlates of interesting Gestalt phenomena are relatively rare. Similar progress
regarding neurocomputational models, which are supposed to rely
on solid computational principles that are compatible with known
neurophysiology and human psychophysics, requires painstaking
bridging efforts and also raises many open questions with respect
to specific details that can be addressed only by multidisciplinary
teams (e.g., psychophysicists, modelers, neurophysiologists, and
neuroanatomists).
Perceptual grouping and figure ground organization are two
important aspects of perceptual organization, but they are not the
only ones. Also interesting are, for instance, the representation of
partwhole relationships in a hierarchical structure, texture segregation, geometric distortions due to context or field effects, embedded figures, and holistic processing of faces. Many of the
above lessons we derived from this review also apply to these
other areas of mid-level vision.
Conclusion
We hope this review has demonstrated that rumors about the
death of Gestalt psychology were greatly exaggerated (Epstein,
1988). On the contrary, the field of research on perceptual grouping and figure ground organization is thriving, and progress has
been tremendous compared to the situation of 100 years ago.
However, significant challenges remain. The above discussion has
outlined an open-ended research program, like Wertheimer and his
fellow-pioneers did. However, we are now able to build on a
research tradition of more than a century. We are convinced that
we can reconsider some of the old puzzles at a much more
advanced scientific level now. The most important challenge will
be to integrate better this research tradition with the rest of vision
science. Such an integration will strongly depend on progress
regarding the conceptual and theoretical foundations of the Gestalt
approach. This will be the focus of the second of this twin set of
review articles on the occasion of the centennial anniversary of
Gestalt psychology (Wagemans et al., 2012).
References
Alais, D., Blake, R., & Lee, S. H. (1998). Visual features that vary together
over time group together over space. Nature Neuroscience, 1, 160 164.
doi:10.1038/1151
1210
WAGEMANS ET AL.
Bhatt, R. S., & Quinn, P. C. (2011). How does learning impact development in infancy? The case of perceptual organization. Infancy, 16, 238.
doi:10.1111/j.1532-7078.2010.00048.x
Biederman, I. (1987). Recognition-by-components: A theory of human
image understanding. Psychological Review, 94, 115147. doi:10.1037/
0033-295X.94.2.115
Binford, T. (1981). Inferring surfaces from images. Artificial Intelligence,
17, 205244. doi:10.1016/0004-3702(81)90025-4
Blum, H. (1973). Biological shape and visual science (Part I). Journal of
Theoretical Biology, 38, 205287. doi:10.1016/0022-5193(73)90175-6
Bravais, A. (1949). Crystallographic studies: On the systems formed by
points regularly distributed on a plane or in space (A. J. Shaler, Trans.).
New York, NY: Crystallographic Society of America. (Original work
published 1850)
Bregman, A. S. (1981). Asking the what for question in auditory perception. In M. Kubovy & J. R. Pomerantz (Eds.), Perceptual organization (pp. 99 118). Hillsdale, NJ: Erlbaum.
Bregman, A. S. (1990). Auditory scene analysis: The perceptual organization of sound. Cambridge, MA: Bradford/MIT Press.
Brincat, S. L., & Connor, C. E. (2004). Underlying principles of visual
shape selectivity in posterior inferotemporal cortex. Nature Neuroscience, 7, 880 886. doi:10.1038/nn1278
Brincat, S. L., & Connor, C. E. (2006). Dynamic shape synthesis in
posterior inferotemporal cortex. Neuron, 49, 1724. doi:10.1016/
j.neuron.2005.11.026
Brown, J. F., & Voth, A. C. (1937). The path of seen movement as a
function of the vector-field. American Journal of Psychology, 49, 543
563. doi:10.2307/1416380
Burge, J., Fowlkes, C., & Banks, M. S. (2010). Natural-scene statistics
predict how the figure ground cue of convexity affects human depth
perception. Journal of Neuroscience, 30, 7269 7280. doi:10.1523/
JNEUROSCI.5551-09.2010
Burge, J., Peterson, M. A., & Palmer, S. E. (2005). Ordinal configural cues
combine with metric disparity in depth perception. Journal of Vision,
5(6), Article 5. doi:10.1167/5.6.5
Burt, P., & Sperling, G. (1981). Time, distance, and feature trade-offs in
visual apparent motion. Psychological Review, 88, 171195. doi:
10.1037/0033-295X.88.2.171
Canny, J. F. (1983). Finding edges and lines in images (Unpublished
masters thesis). MIT Artificial Intelligence Laboratory, Cambridge,
MA.
Carandini, M., Demb, J. B., Mante, V., Tolhurst, D. J., Dan, Y., Olshausen,
B. A., . . . Rust, N. C. (2005). Do we know what the early visual system
does? Journal of Neuroscience, 25, 1057710597. doi:10.1523/
JNEUROSCI.3726-05.2005
Casco, C. (1990). The relationship between visual persistence and event
perception in bistable motion display. Perception, 19, 437 445. doi:
10.1068/p190437
Cavanaugh, J. R., Bair, W., & Movshon, J. A. (2002). Nature and interaction of signals from the receptive field center and surround in macaque
V1 neurons. Journal of Neurophysiology, 88, 2530 2546. doi:10.1152/
jn.00692.2001
Claessens, P. M. E., & Wagemans, J. (2005). Perceptual grouping in Gabor
lattices: Proximity and alignment. Perception & Psychophysics, 67,
1446 1459. doi:10.3758/BF03193649
Claessens, P. M. E., & Wagemans, J. (2008). A Bayesian framework for
cue integration in multistable grouping: Proximity, collinearity, and
orientation priors in zigzag lattices. Journal of Vision, 8(7), Article 33.
doi:10.1167/8.7.33
Cohen, E. H., & Singh, M. (2007). Geometric determinants of shape
segmentation: Tests using segment identification. Vision Research, 47,
28252840. doi:10.1016/j.visres.2007.06.021
Coren, S. (1972). Subjective contours and apparent depth. Psychological
Review, 79, 359 367. doi:10.1037/h0032940
1211
1212
WAGEMANS ET AL.
1213
Kovacs, I., & Julesz, B. (1993). A closed curve is much more than an
incomplete one: Effect of closure in figure-ground segmentation. PNAS:
Proceedings of the National Academy of Sciences, USA, 90, 74957497.
doi:10.1073/pnas.90.16.7495
Kramer, P., & Yantis, S. (1997). Perceptual grouping in space and time:
Evidence from the Ternus display. Perception & Psychophysics, 59,
8799. doi:10.3758/BF03206851
Krantz, D. H., Luce, R. D., Suppes, P., & Tversky, A. (1971). Foundations
of measurement: Vol. I. Additive and polynomial representations. New
York, NY: Academic Press.
Kruger, N. (1998). Collinearity and parallelism are statistically significant
second order relations of complex cell responses. Neural Processing
Letters, 8, 117129. doi:10.1023/A:1009688428205
Kubovy, M. (1994). The perceptual organization of dot lattices. Psychonomic Bulletin & Review, 1, 182190. doi:10.3758/BF03200772
Kubovy, M., Holcombe, A. O., & Wagemans, J. (1998). On the lawfulness
of grouping by proximity. Cognitive Psychology, 35, 7198. doi:
10.1006/cogp.1997.0673
Kubovy, M., & Pomerantz, J. R. (Eds.). (1981). Perceptual organization.
Hillsdale, NJ: Erlbaum.
Kubovy, M., & van den Berg, M. (2008). The whole is equal to the sum of
its parts: A probabilistic model of grouping by proximity and similarity
in regular patterns. Psychological Review, 115, 131154. doi:10.1037/
0033-295X.115.1.131
Kubovy, M., & Wagemans, J. (1995). Grouping by proximity and multistability in dot lattices: A quantitative Gestalt theory. Psychological
Science, 6, 225234. doi:10.1111/j.1467-9280.1995.tb00597.x
Lamme, V. A. F. (1995). The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience, 15, 16051615.
Lashley, K. S., Chow, K. L., & Semmes, J. (1951). An examination of the
electrical field theory of cerebral integration. Psychological Review, 58,
123136. doi:10.1037/h0056603
Lee, S.-H., & Blake, R. (1999, May 14). Visual form created solely from
temporal structure. Science, 284, 11651168. doi:10.1126/science
.284.5417.1165
Lee, T. S., Mumford, D., Romero, R., & Lamme, V. A. F. (1998). The role
of the primary visual cortex in higher level vision. Vision Research, 38,
2429 2454. doi:10.1016/S0042-6989(97)00464-1
Leeuwenberg, E. L. J. (1969). Quantitative specification of information in
sequential patterns. Psychological Review, 76, 216 220. doi:10.1037/
h0027285
Leeuwenberg, E. L. J. (1971). A perceptual coding language for visual and
auditory patterns. American Journal of Psychology, 84, 307349. doi:
10.2307/1420464
Levitt, J. B., & Lund, J. S. (2002). The spatial extent over which neurons
in macaque striate cortex pool visual signals. Visual Neuroscience, 19,
439 452. doi:10.1017/S0952523802194065
Leyton, M. (1989). Inferring causal history from shape. Cognitive Science,
13, 357387.
Li, Z. (1998). A neural model of contour integration in the primary visual cortex.
Neural Computation, 10, 903940. doi:10.1162/089976698300017557
Liu, Z., Jacobs, D. W., & Basri, R. (1999). The role of convexity in
perceptual completion: Beyond good continuation. Vision Research, 39,
4244 4257. doi:10.1016/S0042-6989(99)00141-8
Lowe, D. G. (1985). Perceptual organization and visual recognition.
Boston, MA: Kluwer. doi:10.1007/978-1-4613-2551-2
Machilsen, B., Pauwels, M., & Wagemans, J. (2009). The role of vertical
mirror symmetry in visual shape detection. Journal of Vision, 9(12),
Article 11. doi:10.1167/9.12.11
Machilsen, B., & Wagemans, J. (2011). Integration of contour and surface
information in shape detection. Vision Research, 51, 179 186. doi:
10.1016/j.visres.2010.11.005
Marr, D. (1982). Vision: A computational investigation into the human
1214
WAGEMANS ET AL.
1215
1216
WAGEMANS ET AL.
theory of the curvature-constraint line for amodal completion. Perception, 24, 373389. doi:10.1068/p240373
Ternus, J. (1926). Experimentelle Untersuchungen uber phnomenale Identitt [The problem of phenomenal identity]. Psychologische Forschung,
7, 81136.
Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of
attention. Cognitive Psychology, 12, 97136. doi:10.1016/00100285(80)90005-5
Trujillo, L. T., Allen, J. J. B., Schnyer, D. M., & Peterson, M. A. (2010).
Neurophysiological evidence for the influence of past experience on
figure ground perception. Journal of Vision, 10(2), Article 5. doi:
10.1167/10.2.5
Tse, P. U. (1999). Complete mergeability and amodal completion. Acta
Psychologica, 102, 165201. doi:10.1016/S0001-6918(99)00027-X
Tversky, T., Geisler, W. S., & Perry, J. S. (2004). Contour grouping:
Closure effects are explained by good continuation and proximity.
Vision Research, 44, 2769 2777. doi:10.1016/j.visres.2004.06.011
Ullman, S. (1976). Filling in the gaps: The shape of subjective contours and
a model for their generation. Biological Cybernetics, 25, 1 6.
Ullman, S. (1979). The interpretation of visual motion. Cambridge, MA:
MIT Press.
van den Berg, M., Kubovy, M., & Schirillo, J. A. (2011). Grouping by
regularity and the perception of illumination. Vision Research, 51,
1360 1371. doi:10.1016/j.visres.2011.04.013
van Lier, R. (1999). Investigating global effects in visual occlusion: From
a partly occluded square to a tree-trunks rear. Acta Psychologica, 102,
203220. doi:10.1016/S0001-6918(98)00055-9
van Lier, R., van der Helm, P. A., & Leeuwenberg, E. L. J. (1994).
Integrating global and local aspects of visual occlusion. Perception, 23,
883903. doi:10.1068/p230883
van Lier, R. J., van der Helm, P. A., & Leeuwenberg, E. L. J. (1995).
Competing global and local completions in visual occlusion. Journal of
Experimental Psychology: Human Perception and Performance, 21,
571583. doi:10.1037/0096-1523.21.3.571
Vecera, S. P. (2004). The reference frame of figure-ground assignment.
Psychonomic Bulletin & Review, 11, 909 915. doi:10.3758/
BF03196720
Vecera, S. P., & Farah, M. J. (1997). Is visual image segmentation a
bottom-up or an interactive process? Perception & Psychophysics, 59,
1280 1296. doi:10.3758/BF03214214
Vecera, S. P., Flevaris, A. V., & Filapek, J. C. (2004). Exogenous spatial
attention influences figure-ground assignment. Psychological Science,
15, 20 26. doi:10.1111/j.0963-7214.2004.01501004.x
Vecera, S. P., & Palmer, S. E. (2006). Grounding the figure: Contextual
effects of depth planes on figure-ground organization. Psychonomic
Bulletin & Review, 13, 563569. doi:10.3758/BF03193963
Vecera, S. P., Vogel, E. K., & Woodman, G. F. (2002). Lower region: A
new cue for figure ground assignment. Journal of Experimental Psychology: General, 131, 194 205. doi:10.1037/0096-3445.131.2.194
Vezzani, S., Marino, B. F. M., & Giora, E. (2012). An early history of the
Gestalt factors of organization. Perception, 41, 148 167. doi:10.1068/
p7122
Vickery, T. J., & Jiang, Y. V. (2009). Associative grouping: Perceptual
grouping of shapes by association. Attention, Perception, & Psychophysics, 71, 896 909. doi:10.3758/APP.71.4.896
von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984, June 15).
Illusory contours and cortical neuron responses. Science, 224, 1260
1262. doi:10.1126/science.6539501
von der Malsburg, C. (1981). The correlation theory of brain function
(Departmental Technical Report No. 812). Gttingen, Germany: MaxPlanck-Institut fur Biophysical Chemistry.
von Ehrenfels, C. (1988). On Gestalt qualities. In B. Smith (Ed. &
Trans.), Foundations of Gestalt theory (pp. 82117). Munich, Germany:
Philosophia Verlag. (Original work published 1890)
1217