Sixth Edition

AN INTRODUCTION
TO THEORIES OF LEARNING

Chapter 5
Burrhus Frederic Skinner

Major Theoretical Notions

B. R. HERGENHAHN
Professor Emeritus
Hamline University

MATTHEW H. OLSON
Hamline University

~Dol

Radical Behaviorism
Respondent and Operant Behavior
TypeS and Type R Conditioning
Operant Conditioning Principles
The Skinner Box
The Cumulative Recording
Conditioning the Lever-Pressing Response
Shaping
Extinction
Spontaneous Recovery
Superstitious Behavior
Discriminative Operant
Secondary Reinforcement
Generalized Reinforcers
Chaining
Positive and Negative Reinforcers
Punishment
Alternatives to Punishment
Comparison of Skinner and Thorndike
Schedules of Reinforcement
Verbal Behavior
Prognimmed Learning
Contingency Contracting
Skinner's Attitude toward Learning Theory
The Need for a Technology of Behavior

The Relativity of Reinforcement

David Premack
Revisions of the Premack Principle
William Timberlake

Upper Saddle River, New Jersey 07458

73

74

CHAPTERS

The Misbehavior of Organisms

Evaluation of Skinner's Theory
Contributions
Criticisms

Skinner (1904--1990) was born in Susquehanna, Pennsylvania, and his ideas remain highly influential in contemporary psychology. He received his master's degree in 1930 and his Ph.D. in 1931 from Harvard University. His B.A. degree was
obtained from Hamilton College in New York, where he majored in English. While
at Hamilton, . Skinner had lunch with Robert Frost, the great American poet, who
encouraged Skinner to send him a sample of his writing. Frost favorably reviewed
the three short stmies that Skinner sent, and Skinner decided definitely to become
a writer. This decision was a great disappointment to his father, who was a lawyer
and wanted his son to become a lawyer.
Skinner' s early efforts to write were so frustrating that he thought of seeing
a psychiatrist. He eventually went to work for the coal industry summarizing legal
documents. In fact, his first book, coauthored by his father, concerned those
legal documents and was entitled A Digest ofDecisions of the Anthracite Bomd of Conciliation. After finishing this 'book, Skinner
moved to Greenvvich Village in New York
City, where h e lived like a Bohemian for six
months before going to Harvard to study psychology. By tl1at time he had developed a distaste for most literary pursuits. In his
autobiography (1967) , he said , "I had failed
as a writer because I had nothing important
to say, but I could not accept that explanation. It was literature which must be at fault"
(p. 395). When he failed in desc ribing
human behavior through li teratllle, Skinner
attempted to describe human behavior
through science. Clearly, he was much more
successful at the latter pursuit.
Skinner taught psychology at tl1e University of Minnesota between 1936 and 1945, during which time he wrote his highly influential
text, The Behavior of 01ganisms (1938). One of
Skinner's students at the University of Minnesota was W. K Estes, whose work has had a
considerable impact on psychology (see ChapBurrhus Frederic Skinner. (Courtesy of Otter 9) . In 1945, Skinner went to Indiana Unitawa Karsh.)
versity as chair of the psychology department,

BURRHUS FREDERIC SKINNER

75

and in 1948, he returned to Harvard, to which he remained affiliated until his

death in 1990.
In a survey taken just prior to Skinner's deatl1 (Kom , Davis, & Davis, 1991),
historians of psychology and chairpersons of graduate departments of psychology
were asked to rank tl1e ten most eminent (contemporary and all-time) psychologists. Skinner was ranked eightl1 on the all-time list but first among contemporary
psychologists by histmians of psychology; department chairs ranked Skinner first
on both lists.

Skinner's position was similar to Thorndike's position after 1930 in that it

emph asized the effects of a response on tl1e response itself. Moreover, like
Thorndike, Skinner concluded that the effects of reinforcement and punishment
are not symmetrical; that is, reinforcement changes tl1e probability of a response's
recurring, but punishment does not.
Through the years, Skinner was a highly prolific WJiter. One of his main concerns was to relate his laboratory findings to the solution of human problems. His
work led to the development of programmed learning and teaching machines.
Two representative articles in this area are "The Science of Learning and tl1e Art of
Teaching" (1954) and "Teaching Machines" (1958). Following his own ideas on
this topic, he and his coauthor, Holland, produced a programmed text on his theoretical notions entitled The Analysis of Behavior (Holland & Skinner, 1961). In
1948 he wrote a Utopian novel called Walden Two. The title paid tribute to
Thoreau's Walden. In Walden Two (1948), which Skinner Wlote in only seven weeks,
he attempted to utilize his principles of! earning in tl1e building of a model society.
More recently Skinner wrote Beyond Freedom and Dignit)' (1971), in which he
showed how a technology of behavior could be used in designing a culture. In Beyond Freedom and Dignity, he discussed why tl1e idea of cultural enginee1ing is met
witl1 so much opposition. Skinner's writings have been extended into the area of
child development tlll'ough tl1e efforts of Bijou and Baer (1961 , 1965). His
thoughts have been related to tl1e area of personality through the writings of
Lundin (1974), who wrote Personality: A Behavioral Analysis, and to child rea1ing by
Hergenhahn (1972), who wrote ShajJing Yow Child's Personality.
Most students of psychology are well aware of the widespread utilization of
Skinnerian notions in the area of psychotherapy. For example, Lovaas's early work
with autistic children relied heavily on Skinner's ideas. Today the beha,~or modification approach, based on Skinner's ideas, has become the most widely used and
most effective way of working with autistic or re tarded children. Beha,~oral engineeing, however, is by no means limited to children. The technique has been successfully applied to the all eviation of a number of adult problems such as
stuttering, phobias, eatin g disorders, and psychotic behavior.
During v\Torld War II, while at the Un iversity of Minnesota, Skinner attempted to apply his theory to the problem of national defense. He uained pigeons to peck at discs on which moving pictures of enemy tagets were being
shown. The discs and the motion pictures were ultimately to be contained in .a
glider loaded witl1 high explosives. The glider was called the Pelican, thus, the
name of the article desoibing these events, "Pigeons in a Pelican" (1960). The

CHAPTERS

76

.pecking of the pigeons closed various electronic circuits and thereby kept the vehicle on target. This American version of kamikaze fighter planes would involve no
loss of human lives. Although Skinner demonstrated to a group of America's top scientists that he and his coworkers had perfected a homing device that was almost
immune to el ectronic jamming, was capable of reacting to a wide variety of e nemy
targets, and was simple to build, their proposed project was turned down. Skinner
speculated that the whole idea was simply too fantastic for th e committee to cope
with .

Major Theoretical Notions

Radical Behaviorism
Skinner adopted and developed the scientific philosophy known as radical behaviorism. This sc1entific orientation rejects scientific language and interpretations
that refer to -mentalistic events. As we discuss, some behavioristic lean1ing themists
use term's like drive, motivation, and purpose in order to explain certain aspects of
human and n onhuman behavior. Skinner rejected these klllds of"terms because
they refer to private, mental experience and represented, in his view, a return to
nonscientific psychology. For Skinner, observable and measurable aspects of the
environme nt, of an organism's behavior, and of the consequences of that behavior
were the critical material for scientific scrutiny. Ringen (1999) writes that

BURRHUS FREDERIC SKINNER

77

arms, or legs arbitrarily. Most of our everyday acti,~ties are operant behaviors. Note
that Skinn er did not say tint operant beh avior occurs independen tly of stimulation; rather, h e said that the stimulus causing such behavior is unknown and that it
is not important to know its cause. Unlike respondent behavior, which is dep endent on th e stimulus tl1at preceded it, operant b ehavior is controlled by its consequences.

Type S and Type ~ Conditioning

Along witll tile two kinds of behavior described previously, tl1ere are two kinds of
conditioning. Type S conditioning is also called respondent conditioning and is
identical to classical conditioning. It is called Type S conditioning to emphasize
tl1e importan ce of the stimulus in eliciting tl1 e desired response. The type of conditioning tlla t involves operant behavior is called Type R because of the emph asis on
tile response. Type R conditioning is also calle d operant conditioning.
It is important to n o te tllat in Type R conditioning the stren gth of conditioning is shown by response mte, whereas in Type S conditioning the strengtll of conditioning is usually d e termin ed by tile magnitude of th e con dition e d response. We
see, tllen-, tllat Skinner's Type R conditioning very closely resembles Thorndike's
instrumental conditioning, and Skinner's Typ e S conditi oning is id entical to
Pavlov's classical conditioning. Mter making tile distinction between Type S and
Type R conditioning, Skinner 's r esearch was concerned almost entirely with Type
R, or operant, conditioning.

Operant Conditioning Principles

Skinner holds that science is the search for causes, that the indentification of causes
enables prediction and control, and that experimental investigation, properly conducted, enables indentification of causes. This much of Skinner's radical behaviorism
is a rather traditional and unremarkable view of science.... What is unique, challenging and largely misunderstood about Skinner's radical behaviorism is Skinner's argument that this view of science provides a basis for skepticism about mentalism, in
particular, and about various influential approaches to the development of theories of
learning and intelligent action, in general. (p. 161)

Two general prin ciples are associa ted with TypeR conditioning: (1) any response
tlla t is followed by a reinforcing stimulus tends to be repeated; and (2) a reinforcing stimulus is anytl1ing that incr eases the rate with which an operant resp o nse occurs. Or as we saw earlier, we can say tha t a reinforcer is anything that increases tile
probability of a response's recuning.
Skinner (1 953) did not provide a rule that one would follow in discovering
wh a t would be an effective reinforcer. Ratller, he said that whetller sometlling is reinforcing can only be ascertained b y its effect on behavior:

Respondent and Operant Behavior

Skinner distinguished two kinds of behavior: respondent behavior, which is elicited
by a known stimulus, and operant behavior, which is not elicited by a known stimulus but is simply emitted by the organism. Unconditioned responses would be examples of respondent behavior because they are elicited by unconditioned stimuli.
Examples of respondent behavior would include all reflexes, such as j e rking one's
hand when jabbed with a pin , the constriction of the pupil of the eye when it is exposed to bright light, and salivation in the presence of food. Because operant behavior is not initially correlated with known stimuli, it seems to appear
spontaneously. Examples include beginning to whistle, standing up and walking
about, a child abandoning one toy in favor of another, and moving one's hands,

In dealing with our fellow men in everyday life and in the clinic and laboratory, we
may need to k.J1o'v just h ow reinforcing a specific event js. V\Te often begin by n oting

the extent to which our own behavior is reinforced by the same event. This practice
frequently miscarries; yet it is still commonly believed that reinforcers can be identifi ed apart from their effects upon a particular organism. As the tenn is used here,
however, the only defining characteristic of a reinforcing stimulus is that it reinfo rces.
(p. 72 )

In op erant conditioning, tile emphasis is on b eha,~or and its consequen ces;

witll operant condi tioning, tl1e organism must respond in such a way as to produce
tile reinforcing stimulus. This p rocess also exemplifies contingent reinfonement, be-

78

CHAPTERS

cause getting the reinforcer is contingent (dependent) on the organism's emitting

a certain response. Vve have more to say about contingent reinforcement in our
discussion of superstitious behavior.
The principles of operant conditioning can apply to a variety of situations. To
modify behavior, one merely has to find something that is reinforcing for the organism whose beha\~or one wishes to modify, wait until the desired behavior occurs, and then immediately reinforce the organism. When this is done, the rate
with which the desired response occurs goes up. When the behavior next occurs, it
is again reinforced, and the rate of responding goes up further. Any behavior that
the organism is capable of performing can be manipulated in this manner.
The same principles are th ought to apply to the development of human personality. According to Skinner, we are what we have been reinforced for being.
What we cill personality is nothing more than consistent behavior patterns that
summarize our reinforcement history. We learn to speak English, for example, becau se we have been reinforced for approximating the sounds of the English language in our early home environment. If we happened to be brought up in a
J apanese or Russian home, we would learn to speak japanese or Russian because
when we approximated sounds in that language, we would have been attended to
or reinforced in some other way. Skinner (1971) said,
The evidence for a crude environmentalism is clear enough. People are extraordinarily different in different places, and possibly just because of the places. The nomad on
horseback in Outer Mongolia and the astronaut in outer space are different people,
but, as far as we know, if they had been exchanged at birth, they would have taken
each other's place. (The expression "change places" shows how closely we identify a
person's behavior with the environment in which it occurs.) But we need to know a
great deal more before that fact becomes useful. What is it about the environment
that produces a Hottentot? And what would need to be changed to produce an English conservative instead? (p. 185)

Skinner defined culture as a set of reinforcement contingencies. His answers

to these questions were that a parti cular set of reinforcement contingencies produces a Hottentot and another set produces the English conservative. Different
cul tures reinforce different behavior patten1s. This fact must be clearly understood
before an adequate technology of behavior can be developed. Skinner said (1971) ,
The environment is obviously important, but its role has remained obscure. It does
not push or pull, it selects, and this function is difficult to discover and analyze. The
role of natural selection in evolution was formulated only a little more than a hundred years ago, and the selective role of the em~ronment in shaping and maintaining
the behavior of the individual is only beginning to be recognized and studied. As the
interaction between organism and environment has come to be understood, however,

effects once assigned to states of mind, feelings, and traits are beginning to be traced
to accessible conclitions, and a technology of behavior may therefore become available. It will not solve our problems, however, until it replaces traditional prescientific
views, and these are strongly entrenched. (p. 25)

BURRHUS FREDERIC SKINNER

79

In Skinner's attempt to understand the causes of be havior, and thus to predict and control beha\~Or, the analogy between operant conditioning and na tural
selection is an important one. Ringen (1999) writes,
His main thesis is that the causal processes producing the behavior traditionally called
purposive and intentional are instances of selection by consequences, a causal mode
exhibited in the analogous processes of operant conclitioning (the contingencies of
reinforcement) and natural selection (the con tingencies of survival) .... He suggests
that just as we learned that design could be produced withouta designer we are learning that intelligence (and purpose) can be produced without mind. (p. 168)
If one controls reinforcement, one can also control be h a,~or. However, this
need not be looked on as a negative statement because beha\~or is constantly
being influenced by reinforcement whether we are aware of that fact. It is never a
question of whether behavior is going to be controlled but wh o or what is going to
control it. Parents, for example, can decide to give direction to their child's emerging personality by reinforcing certain behavior, or they can let society rear their
child by le tting television , peers, school , books, and baby-sitters do the reinforcing.
Giving direction to a child's life is difficult, however, and any parents wishing to do
so must take at least the following steps (Hergenhahn, 1972):

1. Decide the major personality characteristics you want your child to possess as
an adult. Let's say, for example, you want the child to grow up to be a creative
adult.
2. Define these goals in b ehavioral terms. In this case you ask, "What is a child
doing wh en he or she is being creative?"
3. Reward behavior that is in accordance with these goals. With this example,
you would reward instances nf creativity as they occurred.
4. Provide consistency by arranging the major aspects of the child 's environment so that they too reward the behavior you have deemed important. (pp.
152-153)
Without knowledge of these principles, a parent could easily misapply them
without knowing it. Skinner (1951) said,
The mod1er may unwillingly promote d1e very behavior she does not want. For example, when she is busy she is likely not to respond to a call or request made in a quiet
tone of voice. She may answer tl1e child only when it raises its voice. The average intensity of the child 's vocal behavior therefore moves up to anotl1er levei. .. . Evenntally
the mother gets used to this level and again reinforces on ly louder instances. This~
cious circle brings about louder and louder behavior. .. . The mother behaves, in fact,
as if she has been given the assignment of teaching th e child to be annoying. (p. 29)
According to Skinner, living organisms are constantly being conditioned by
th eir environment. We can eith er allow the principles of learning to operate capri-

80

CHAPTERS

BURRHUS FREDERIC SKINNER

81

ciously on our children, or by systematically applying those principles, we can give

some direction to their development.

ily be transformed into the total number of responses. Sample cumu lative r ecordings are shown in Figure 5-2.

The Skinner Box

Conditioning the Lever-Pressing Response

Most of Skinner's early animal work was done in a small test chamber that has
come to be called the Skinner box. It is a direct d escendant of the puzzle box used
by Thorndike. The S)<inner box usually has a grid floor, light, lever, a nd food cup.
It is arranged so that when the animal depresses the lever, the feed er mechanism is
activated, and a small pellet offood is r eleased into the food cup. A typical Skinner
box is sh own in Figure 5-1.

Typically, conditioning the lever-pressing response involves the following steps:

The Cumulative Recording

Skinner used a cumulative recording to keep track of an animal's-behavior in th e
Skinner box. A cumulative recording is quite different from other ways of graphing
data in learning experiments. Time is rec8rded on the x-axis and total number of
responses.is recorded on the y-axis . Th e cumulative recording never goes downthe line either climbs or remains parallel to the x-axis. Let's say we are interested in
how often the animal presses the
lever. When the cumulative recording
shows a lin e parallel to the x-axis, it indicates no responding; that is , the animal is n o t pressing th e lever. When
the animal makes a lever-pressing response, the pen goes up a notch a nd
remains at that level until the animal
makes another respon se. If, for example, th e animal presses the lever when
it is first placed in th e Skinner box,
the pen will go up a notch and remain
there until the animal responds again,
a t which time the pen will go up another notch, and so on. If the animal
responds very rapidly, the lin e will rise
very rapidly. The rate with which the
line ascends indicates the rate of responding; a very steep lin e indi cates
very rapid responding, and a lin e parallel to the x-axis indicates no responding. If at any time you want to
know th e total number of responses
made by the animal, you just measure
the distance between the lin e of the
FIGURE 5-1 A typical Skinner box.
graph and the x-axis , and this can eas(Courtesy ofTime Life Syndication.)

1. Deprivation The experimental animal is put on a deprivation schedule. If

food is to be used as the reinforcer, the animal is d eprived of foo d for a twentythree-hour period for a number of days prior to the experiment, or it is m aintained a,t 80 percent of its free-feeding body weight. If water is to be used as the
reinforcer, the animal is deprived of water for a twenty-three-hour period for a
number of days prior to the experiment. (Some Skinner boxes are designed to deliver small pellets of food and others small drops of water.) Skinner does not say
that these procedures "motivate" the animal; he even hesitates to say that they produce a drive state. Deprivation is simply a set of procedures that is related to how
an organism performs on a certain task; nothing more needs to be said.
2. Magazine Training After being on a deprivation schedule for a number of
days, the animal is placed into the Skinner box. Using an external hand switch, the
experimenter p eriodically triggers the feeder mechanism (also called the magazine) , making sure the animal is not in the vicinity of the food cup when he or she
does so (otherwise the animal would learn to remain near the food cup). When
the feeder mechanism is activated by the hand switch, it produces a fairly loud

No Responding
A Series of litelat1vely\-Fast Responses

A Senes of Relatively
Slow Responses

~
~

z ~
Iii <I>

;~

Another
Response

Q)

<I)

<I)

;;c.

One
Response

Q)

QlCI:

~0

'0

No
Responding

J
-

Paper Moves in this Direction

FIGURE 5-2 A cumulative recording. Note that the steeper the line, the faster the
rate of responding. A line parallel to the baseline indicates no response.

CHAPTER 5

82

BURRHUS FREDERIC SKINNER

clicking sound before delivering a pellet of food into the food cup. Gradually the
animal associates the click of the magazine 'Acith the presence of a food pellet. At
that point the click has become a secondary reinforcer through its association 'Acith
a primary reinforcement (food ). (We discuss secondary reinforcement in a later
section.) The click also acts as a cue or signal indi cating to the animal that if it responds by going to the food cup, it will be reinforced.
3. Lever Pressing Now the animal can be left in the Skinner box on its own.
Eventually, it will press the lever, which mil fire the food magazine, producing a
click that reinforces the bar press and also signals the animal to go to the food cup,
where it is reinforced by food. According to operant conditioning principles, the
lever-pressing response , having been reinforced, mil tend to be repeated, and wh en
it is repeated, it is again reinforced, whi ch further increases the probability that the
lever-pressing response mil be repeated, and so on. A typical cumulative recording
generated by an animal placed in a Skinner box after magazine training is shown in
Figure 5-3.
;

Shaping
The process of operant conditioning we have described so far takes considerable
time . As discussed earli er, one way to train the lever-pressing response is to place
the deprived animal in the Skinner box and simply leave it there. The experimenter merely checks the cumulative recording periodically to see if the response
has been learned. Under these conditions the animal either learns or dies.
There is another approach to operant conditioning that does not take as long
as the procedure previously described. Again, the animal is placed on a deprivation
schedule and is magazine trained, and again the experimenter uses the hand s'Acitch
to trigger the feeder mechanism externally. This time, however, the experimenter
decides to fire the feeder mechanism only when the animal is in the half of the
Skinner box containing the lever. vVhen the animal is reinforced for being near the
lever, it will tend to remain in th at part of the test chamber. Now that the animal re-

FIGURE 5-3 A typical

cumulative recording that
reflects the acquisition of a
lever-pressing response .

"'
"'c:0
Ql

c.

"'

a:"

'

83

mains in the vicinity of the lever, the experimenter begins to reinforce it only when
it is still closer to the lever. Next it is reinforced only when it touches the lever, then
only when it is putting pressure on it, and finally only when it is pressing it by itself.
The process is similar to a childhood 'game called You 're Hot, You're Cold, in
which a child hides something and the child's playmates try to find it. As they get
closer to the hidden object, the child who hid the object says, ''You' re getting
warm , you 're warmer, you're boiling hot, you ' re on fire." As they get farther from
the object, the child says, ''You're getting cold , colder, very cold, you're fre e zing."
When this game is played in the laboratory, it is called shaping. In the shaping procedure described, the lever-pressing response was shaped rather tl1an waited for.
Shaping has two components: differential reinforcement, which simply
means some responses are reinforced and others are not, and successive approximation, which refers to the fact that only those responses that become increasingly
similar to the one tl1e experimenter wants are reinforced. In our example, only
those responses tl1at successively approximated the lever-pressing response were
differentially reinforced.
Recently it has been found that under certain circumstances, preexisting or
even accidental contingencies between events in the emcironment and an animal's
response automatically shape behavior. This phenomenon is called autoshaping,
which we discuss later in this chapter.

Extinction
As mth classical conditioning, when we remove the reinforcer from the operant
conditioning situation, we produce extinction. During acquisition the an imal gets
a pellet of food whenever it presses the lever. Under these circumstances the animall earns to press the lever and persists in doing so until it is satiated with food. If
the feeder mechanism was suddenly disconnected, thus preventing a lever pressing
from producing a pellet of food, we would note that t11e cumulative recording
would gradually become shallower and would eventually become parallel to the xaxis, indicating tl1at no lever-pressing responses are being made. At that point we
say that extinction has occuned.
We are being somewhat inaccurate when we say that after extinction a response is no longer made; it is more accurate to say that after extinction, the response rate goes back to where it was before reinforcement was introduced. Tlus
baseline rate is the frequency mth which the response occurs naturally in the life
of the animal "cithout tl1e introduction of reinforcement. This is called tl1e operant
level for that response . VVhen we remove reinforcement from the experimen tal
arrangement, as in extin ction, the response tends to go back to its operant level.

Q)

"1ii

Spontaneous Recovery

'S

u"

Time-+-

After extinction, if the animal is returned to its home cage for a period of time and
then brought back into the experimental situation, it mll again begin to press the
lever for a short period of time mthout any additional training. This is refened to

CHAPTERS

84

BURRHUS FREDERIC SKINNER

as spontaneous recovery. A cumulative recording showing both extinction and

spontaneous recovery is shown in Figure 5-4.

85

Discriminative Operant
Now we return to the Skinner box and discuss th e light that we referred to earlier.

Mter we have conditioned the animal to press the lever, we can make the situation

Superstitious Behavior
In our earlier discussion of operant conclitioning, we briefly mentioned contingent
reinforcement. Reinforcement following the lever-pressing response is an example
of contingentreinforcement-because the reinforcer is dependent on the response.
What would happen , however, if the situation was arranged so that the feeder
mechanism would fire every now and then, independently of the animal's behavior? In other words, we are now going to arrange .t he situation so that the feeder
mechanism randomly delivers a pellet of .food regardless of what the animal is doing.
According to the principles of operant conditioning, we can predict that
whatever the animal is doing when the feeder mechanism is activated will be reinforced, and the animal will tend to repeat the reinforced behavior. Mter a period
of time, the reinforced behavior will reoccur when the feeder mechanism fires
again, and the response will be strengthened. Thus the animal is apt to develop
strange ritualistic responses; it may bob its head, tum in a circle, stand up on its
back legs , or pelform a se1ies of actions according to what it was doing when the
feeder mechanism fired. This ritualistic behavior is referred to as superstitious because the animal looks as if it believes that what it is doing is causing a pellet of
food to appear. Because the reinforcer in this situation is independent of the animal's behavior, it is referred to as noncontingent reinforcement.
One can think of numerous examples of superstitious behavior on the part of
humans. Organized sports, for instance, are filled with many examples. Imagine what
happens to the baseball player who, after stepping to the plate, adjusts his hat in a certain way and hits the very next pitch out of the ball park. There will be a strong tendency on his part to adjust his hat in a similar way the next time he is at bat.

FIGURE S-4 A cumulative

recording that depicts the
extinction and spontaneous
recovery of a lever-pressing
response.

"'
"'"
""'
0::
Ql

Spontaneous Recovery

Ql

~
'S

::>

Extinction

.,

more complex. We can arrange the situation so that the animal receives a pellet of
food when the light in tl1e Skinner box is on but not when tl1e light is off. Under
these conditions, we refer to the light as S0 , or a cliscriminative stimulus. The light
being on defines the S0 condition, and the light being off d efines the S" condition
(ll=delta). With this arrangement, tl1e animal learns to press the lever when the
light is on and not to press when the light is off. The light, therefore , has become a
signal (cue) for the lever-pressing response. We have developed a cliscrin1inative
operant, which is an operant response given to one set of circumstances but not to
another. The arrangement can be symbolized as follows: S0 -7R-7SR where R is the
operant response and SR is the reinforcing stimulus.
The concept of the discriminative stimulus allows for a more detailed statement about which stimulus-response relationship is of interest in operant conditioning. For Thorndike, the association of interest was between a general
environmental situation and a response effective in solving a problem. For Skinner, the relationship of interest can be diagrammed as follows:
Discriminative stimulus
(S 0 )

--7

operant response
(R)

--7

reinforcing stimulus
(SR)

+
association of interest

Except for slight differences in terminology, Skinner's views of learning are

quite similar to those of Thorndike after 1930. In fact, except for the way each researcher measured the dependent variable, which was pointed out earlier in this
chapter, instrumental conditioning and operant conditioning can be considered
the same procedure.
There is some slight similarity between the discriminative operant and respondent conditioning. You will recall that respondent behavior is elicited by a
known stimulus. The beha\~or occurs because of its association "~th the stimulus.
Such behavior, as we have seen, is not under the control of its consequences. In
the case of the discriminative operant, the light becomes a signal associated with a
certain response tl1at the organism has learned will be followed by reinforcement. .
Operant behavior is emitted beha,~or, but Skinner (1953) said,
Most operant behavior ... acquires important connections with the surrounding world.
'\e may show how it does so .in our pigeon experi1nent by reinforcing neck-stretching

when a signal light is on and allowing it to be extinguished when the light is off Eventually stretchin g occurs only when the light is on. We can then demonstrate a stimulus-

Time--

response connection which is roughly comparable to a conditioned or unconditioned

reflex: the appearance of the light will be quickly followed by an upward movement of
the head. But the relation is fundamentally quite different. It has a different history and
different current properties. We describe the contingency by saying that a stimulus (the

86

CHAPTERS

light) is the occasion upon which a response (stretching the neck) is followed by reinforcement (with food)_ We must specify all three terms. The effect upon the pigeon is that
eventually the response is more likely to occur when the light is on. The process through
which this comes about is called discrimination. Its importance in a theoreti cal analysis,
as well as in the practical control of behavior, is obvious: when a discrimination has been
established, we may alter the probability of a response instantly by presenting or removing the discriminative stimulus. (pp. 107-108)
Thus, the discriminative operant involves a signal that leads to a response,
which in tum leads to reinforcement.
There are numerous examples of discriminative operants in everyday life. A
certain time of the day (S 0 ) indicates that you must be in a certain place (R) to
transact some business (SR) . As you're driving down the street, you encounter a red
light (S0 ) , which causes you to stop (R) , thereby avoiding a ticket or an acciden~
(SR). You see someon e you don 't care for (S0 ), causing you to change the direction
you are waiking in (R), thereby avoiding the person (SR) .

Secondary Reinforcement
Any neutral stimulus paired with a primary reinforcer (e.g., food or water) takes on
reinforcing properties of its own; this is the principle of secondary reinforcement.
It follows then that every S0 must be a secondary reinforcer because it consistently
precedes primary reinforcement.

One way to demonstrate the reinforcing properties of a previously n eutral

stimulus is to wire the Skinner box so that a light comes on before the animal receives food for making a lever-pressing r esponse. According to the principle of secondary reinforcement, the pairing of th e light with fo od should cause the light to
take on reinforcing properties of its owri. One way to test this notion is to extinguish the lever-pressing response so that the animal presses the lever, and n either
light nor food is produced. When the response rate d ecreases to its-operant level ,
we arrange for the lever pressing to tum on ,the light but not deliver a pellet of
food. We note that the response rate goes way up. Because the light alone h as increased the response rate and thereby prolonged extincti on , we say it has developed secondary reinforcing characteristics through its association with food dming
acquisition (training) . A light not associated with a primary reinforcer will not produce a similar effect during extinction.
In addition to maintaining th e lever-pressing resp onse, we can now use the
light to condition other responses. Once a previously neutral stimulus takes on reinfor cing properties throu gh its associa tion with primary reinforcement, it can be
used to reinforce any number of responses.
Keller and Schoenfeld (1950) provid e an excellent summary of secondary reinforceme nt:

BURRHUS FREDERIC SKINNER

87

when repeatedly applied to a respon se for which there is n o ultimate primary

reinforcem ent.
2. A secondary reinforcement is positive wh en the reinforcement with which it
is correlated is positive, and negative when the latter is negative .
3. Once established, a secondary reinforcement is independent and nonspecific; it will n ot only strengthen the same response which produced th e origimil reinforcement, but it will also condition a new and unrelated response.
Moreover it will do so even in the presence of a different motive.
4. Through ~eneralization, many stimuli besides th e one correlated with reinforcement acquire reinforcing value-positive or negative . (p. 260)

Generalized Reinforcers
A generalized reinforcer is a secondary reinforcer that has been paired with mor~
th an one primary reinforcer. Money is a generalized reinforcer because It IS ulum ately associated with any number of primary reinforcers. The _m am a~vantage of
the generalized reinfo rcer is that it does not depend on a certam condiUoi: of deprivation to be effective. Food, for exampl.e, is only reinforcing for an orgamsm d eprived of fo od, but m oney can be used as a reinforcer wheth er or not someone IS
deprived of food . Moreover, the very activities that on ce led to reinforcement may
themselves become reinforcing. Skinner (1953) said,
Even tuall'y generalized reinforcers are effective even though tl1e primary reinf~ rcers
upon which tl1ey are based no longer accompany them. We play g~mes of skill for
tlleir own s~ke . We get attention or approval for its own sak~. A.ffecnon IS not always
followed by a more explicit sexual reinforcement. The submiSSiveness of others IS reinforcing even though we make no use of it. A miser may be so remforced by money
that he will stan>e rather than give it up . (p. 81)
With these comments, Skinner came very close to Gordon Allport's concept of
functional autonomy. Allport (1961) maintained that although a~ activity '?ay once
have been engaged in because it led to reinforcement, after a while the acUVJty Itself
may become r einfo rcing. In other words, the activity may become mdependent of
th e reinforcer on which it was miginally dependent. For example, a person might
originally join the merchant marines in order to m ake a living but later in life go sailing because it is enjoyable to do so even though sa1lmg no longer prm'!des an mcome. In this case we say that sailing has become funcu onally autonomous; th at IS,_ It
continues in the absence of the original motive . Skinner said that such an acUVIty
must u ltimately result in primary reinforcement or it would extinguish. Allport, however, would say tha t the activity no longer d epends on primary re inforce m en t.

Chaining
0

1. A stimulus that occasions or accompanies a reinforcement acquires thereby

reinforcing value of its own, and. may be called a conditioned, secondary, or

derived reinforcement. A secondary reinforcem ent may be extinguish ed

One response can bring the organism into contact with stimu li that act as an S for
anoth er response , which in tum causes it to experience stimuli that cause a third
response , and so on. This process is referred to as chaining. In fact, most be ha\'1 0r

88

CHAPTERS

can be shown to involve some form of chaining. For example, even the lever pressing in the Skinner box is ncit an isolated response. The stimuli in the Skinner box
act as S0 s, causing the animal to tun1 toward the lever. The sight of the lever causes
the animal to approach it and press it. The firing of the feeder mechanism acts as
an additional S0 , which elicits the_response of going to the food cup. Consuming
the food pellet acts as an S0 , causing the animal to return to the lever and again
press it. This sequence of events (chain) is held together by the food pellet, which
of course is a priinary positive reinforcer. It can be said that various elements of a
behavioral chain are held together by secondary reinforcers but that the entire
chain depends on a primary reinforcer.
To explain how chaining comes about from Skinner's point of view, one must
utilize the concepts of secondary reinforcement and associative shifting. Because
of their association with the primary reinforcer, the e vents prior to the delivery of
the food pellet take on secondary r einforcing properties. Thus the sight of the
lever itself becomes a secondary reinforcer, and the response of looking at the
lever is reinforced by the sight of the lever. Now, through a process similar to associative shifting (or higher-order conditioning, which we discuss in Chapter 7),
other. stimuli mort remote from the lever develop reinforcing properties. Thus,
after considerable training when the animal is placed in i:he Skinner box, the initial stimuli it encounters will act as an S0 , causing the animal to orient toward the
lever. The sight of the lever at this point acts both as a reinforcer and an S0 , eliciting the next response in the chain. The situation is diagrammed in Figure 5-5.
It is important to note that the development of a chained response always
acts from the primary reinforcer backward. As more and more related stimuli take
on reinforcing properties, the chain is extended. It is possible, for example, for the
chain to extend gradually all the way back to the animal's home cage.
Occasionally rats have been trained to perform complex chained responses
such as climbing a staircase, riding in a cart, crossing a bridge, playing a note on a
toy piano, entering a small elevator, pulling a chain, riding the elevator down, and
receiving a small pellet, of food. This chain, too, is developed backward so that the
events that precede the primary reinforcer gradually become secondary reinforcers. VVhen they do, they reinforce the responses prior to them, and so on along
the chain of behaviors.

so
General
Stim"Uii in

the Test
Chamber

--7

--7

so
s

--7

Orient
toward

Sight of .

Approach

Lever Reinforces

Lever

Lever

Response of

--7

Turning toward
It and Acts as

a Cue for the

Next Response

FIGURE 5-S

An example of chained behavior.

so
s
Contact with
Lever Reinforces

Approaching
and Acts as
Cue to Press

--7

R
Press
Lever

--7

s
Food
Pellet

89

BURRHUS FREDERIC SKINNER

Chained responses can also occur between two people. For example, seeing
someone you know acts as an S0 to say "hello." Your hello acts as an S0 for your
friend to say "hi." The response of"hi" acts not only as a reinforcer for your "hello"
but also as an S0 for you to say "How are you?" This two-person chain can be diagrammed as follows:
You:

so
Seeing
friend

--7

R
H ello

.L

You1-jriend:

so

--7

so
sR

--7

--7R --7
Hi

R
--7
How are
you?

.L

so
sR

--7

so
sR

--7 R --7etc.

R
Fine

Not only do the consequences of certain responses act as cues for other responses, but certain thoughts can act as S0 s for other_thoughts. Skinner (1953)
said,
A response may produce or alter some of the variables which control another response. The result is a "chain." It may have little or no organization. When we go for a
walk, roaming the countryside or wandering idly tl1rough a museum or store, one
episode in our behavior generates conditions responsible for anoth er. We look to one
side and are stimulated by an object which causes us to move in its direction . In the
course of this movement, we receive aversive stimulation from which we beat a hasty

retreat. This generates a condition of satiation or fatigue in which, once free of aversive stimulation, we sit down to rest. And so on. Chaining need not be the result of
movement in space. VVe wander or roam verbally, for example, in a casual conversa-

tion or when we "speak our thoughts" in free association. (p. 224)

Positive and Negative Reinforcers

marize Skinner's position on reinforcement, we have first of all

positive r forcement. This is something that is naturally reinforc ~;;:~-;;;_
ganism and is 1 ted to survival , such as food or water. Any neu
stimulus associ. ate d with primary
itive reinforcement takes on positi
econdary reinforcing
characteristics. A positive n orcer, either primary or se
ary, is something that, when
e probability of that 1esponse 's recuradded to the situation by a certaz

90

spouses produce pleasant or desirable outcomes. Similarly, negative reinforc ment

es not earn its name because a response produces nasty or unpleasant o comes.
In ddition, negative reinforcement should not be confused with p ishment
(Skit 1er, 1953):
Eve ts which are found to be reinforcing are of two sorts. Some rei orcements consist o resenting stimuli, of adding something-for exan1ple, foo water, or sexual
con tac to the situation. These we call positive reinforcers. Other consist of removing

somethin -for example, a loud noise, a very bright light, extr me cold or heat, or
electric sh k-from the situation. These we call negative rei orcers. In both cases
einforcement is the same-the probability of r ponse is increased. We
cannot avoid s dis.tinction by arguing that what is reinfor ng in the negative case is
the absence of tli bright light, loud noise, and so on; for is absence after presence
which is effective, nd this is only another way of saying at the stimulus is removed.
The difference be
en the tvvo cases vvill be clearer wl n we consider the presentation
of a negative reinforc or the removal of a positive. T1 se are the consequences which

we call punishment. (p. 3)

made during the second extinction session was about the same for both grou
with the non punished group making slightly more responses. From the data o
rst two sessions, one can conclude that punishment was effective becau
ber of responses to extinction was much lower for tl1e punished gro . Dure third extinction session, however, the previously punished gr p made
many ore responses than did the non punished group. Thus, in the I g run the
origina " punished group caught up in tl1e total number of respo es to extinction to th non punished group. The conclusion was that simple no reinforcement
(extinction is as effective in extinguishing a habit as nonreinfo ement plus punishment. The esults of the Estes study are summarized in Figu 5-6.
Skinner's ain argument against the use of punishm t is that it is inef(ective in t11e long
. It appears tl1at punishment simply s
when the threat of unishment is removed, tl1e rate wi
curs retun1s to its on al level. Thus, punishment o en appears to be very successful when, in fact, it as produced only a temp ary effect. Other arguments
against the use of punish nt follow.

Punishment
be-

1.

400

Punishment is designed t remove awkward, dan

rou.s, or otherwise unwanted be-

""'

havior from a repertoire n the assumption that a p rson who has been punished is
less likely to behave in

"' 300

e same way again. Unfortuna ly, the matter is not that sim-

rF

ple. Reward and puni ment do not differ merely in the irection of the changes they
induce. A child who as been severely punished for sex p
is not necessarily less inclined to continue nd a n1an who has been imprisoned fo

"

cr:

.~

riolent assault is not nec-

1ii
'S
E

essarily less incli d toward violence. Punished behavior is lik ' to reappear after the
punitive con tin ncies are "~thdrawn. (pp. 51-52)

::l

()

A typical e periment that Jed Skinner to this conclusion was lon e by one of
his students, tes (1944). Two groups of eight rats each were traine to press the
lever in aS ner box. After training, both groups were placed on exti ction. One
group was xtinguished in the regular way; that is, food was withheld allowing
lever pr sing. Rats in the second group, in addition to not receiving od , received shock when they pressed the lever. Rats in this group were shocked n average f nine times. There were three extinction sessions, and the rats were nly
shoJ ed during the first of the three sessions. The second and third sessions we e
t~~ same for both groups . The punished group made fewer responses during th
fiTst extinction session than did the nonpunished group. The number of responses

o Rats Experienced Extinction

Procedures Without Punishment

Extinction Sessions

The results of Estes's research showing that the effect of punishmen

is to suppress the rate of responding only temporarily. (From W.K. Estes, An
experimental study of punishment, Psychological Monographs, 57, Whole No. 263,
1944, 5.)
FIGURE S-6

BUR,RHUS FREDERIC SKINNER

2. It indicates what the organism should not do, not what it should do Com-
pared to reinforcement, punishment conveys '~rtually no i.nformati
to the
organism. Reinforcement indicates that what was done is effective the situnishment
ation; therefore, no additional learning is required. Very often
informs the organism only that the punished response is one hat will not
ark to b1ing reinforcement in a given situation, and additi a! learning is
re uired to hit on a response that will work.
3. It j tifies inflicting pain on others. This, of course, applies o the use of punishm t in child rearing. When children are spanked, 1e only thing they
may be earning is that under some circumstances it justifiable to inflict
pain on o ers.
4. Being in a 'tuation where previously punished beha or could be engaged in
without bein unished may excuse a child to do so hus, in the absence of a
punishing age , children may swear, break win ows, be disrespectful to elderly people, pu smaller children around, a
so on. These children have
learned to suppres these behaviors when they, auld lead to punishment, but
in the absence of a
nishing agent, there i no reason to avoid engaging in
these activities.
5. Punishment elicits aggr sion toward th punishing agent and others. Punishment. causes the pun hed organi7
to become aggressive, and this
aggression may cause add! anal pr 6 lems. For example, our penal institutions, which use punishme t as eir major means of control, are filled
with highly aggressive indiVid a! who will continue to be aggressive as
long as punishment or the thr t of punishment is used to control their
behavior.
6.

In their study of how. 79 New England subur n mothers brought up their

children from birth to dergarten age, Sears, Mace by, and Levin (1957) concluded the following co cerning the relative effects of e
hasizing reinforcement
as opposed to punish1 ent in child rearing:
In our discuss' n of the training process, we have contrasted pu 'shment with reward.

Both are techniques used for changing the child's habitual ways o cting. Do they work
equally weiJf The answer is unequivocally '"no"; but to be tmly uneq "vocal, the answer
must be urlderstood as referring to the kind of punishment we were a e to m easure by
our inte ew method. '1\Te could not, as one can with laboratory experi ents on white
rats or igeons, examine the effects of punishment on isolated bits of havior. Our
measures of punishment, whether of the object-oriented or love-oriente variety, refen: d to Levels ofPunitiveness in the mothers. Punitiveness, in contrast to rewa

w a quite ineffectual quality for a mother to inject into her child training.
The evidence for this conclusion is ove1whelming. The unhappy effects of p mishment have run like a dismal thread through our findings. Mothers who punishe toi-

Jet accidents severely ended up with bedwetting children. Mothers who punishe
pendency to get rid of it had more dependent children than mothers who d. not
punish. Mothers who punished aggressive behavior severely had more aggress e children than mothers who punished lightly. They also had more dependent ildren.
arsh physical punishment was associated with high childhood aggressi ness and
"th the development of feeding problems.
ur evaluation of punishment is tl1at it is ineffectual over the long t
fore ninating the kind of behavior taward which it is directed. (p. 484)
Why, th , is punishment so widely used? Because, said S
reinforcing to t e punisher:

,,

Severe punishm nt unquestionably has an immediate effe in reducing a tendency to

act in a given wa . This result is no d()ubt responsible fo its widespread use. We "instinctively" attack a one whose behavior displeases us- erhaps not in physical assault,
but wit.Ii criticism , dis proval, blame, or ridicule. Wh
er or not there is an inherited
tendency to do this, th immediate effect of the pr tice is reinforcing enough to explain.its currency. In the ng run, however, punis ent does not actually eliminate behavior frOm a repertoire,

;;; '

_ its tetnpori:.uy achie

mentis o~?tained at.tren1endous cost

in reducing the over-all effiCl ncy and happine of the group. (p. 190)
It is interesting to note that S "hrier h ' self was never physically punished by
his father and only once by his mot
o washed his mouth out with soap for
swearing (Skinner, 1967, p. 390).

Altematives to Punishment
Skinner lists a number of altern ves to the us of punishment.. The circumstances
. causing the undesirable beZ?.hvi can be changed, ere by changing the behavior. For
example, removing fine chin from the lh~ng roo will eliminate the problem of a
child's breaking fine china. e undesirable _r~s~m~se a~ be satiated by lettin~ the organism perform the unde red response until It 1s s1ck o t, such as letnng a chtld continue to light matches o eat candy (advice similar to that ven by Guthrie, as we see
in Chapter 8). If the
desirable behavior is a function of e child's developmental
stage, it can be elimi ated by simplywaii:ingforthe child to ou row it. Skinner (1953)
said about the latt . approach, "It is not always easy to put up {h the behavior tmtil
this happens, estfecially under the conditions of the average ho,1~hold, but there is
some cont ola n if we know that by carrying the child through a \ Cial!y unacceptab-l e stage w pare him the _later complic~tions arisin.g from pu.nishm nt" (p. 192).
Ano er method ts simply to let IJme pass, bu.t th1s approach ay take too
long. HaBits are not soon forgotten . For example, in his "Pigeons in a P ican" project m /n tioned earlier, Skinner (1960) found that his trained animals 'immediately, and correctly" performed their task after six years of inactivity. Still
other
alt; native to punishment is to reinforce behavior incompatible with the un esir\16Je behavior (e.g., a child is reinforced for reading in the presence of mat es
rather than stiiking them) . The best way to discourage an undesirable habit, however, is to ignore it (Skinner 1953):

94

CHAPTERS

BURRHUS FREDERIC SKINNER

95

Schedules of Reinforcement

General.Iy spea~ , ehavior persists beca

it is being reinforced ; this is
true ofundestr ~ well as destrable behaVlor. To ehm
e objectionable behavtor one
s to find the source of reinforcement and remo . Behavior th a t
s not lead to reinforcement extinguishes.

Comparison of Skinner and Thorndike

Although Skinner and Thorndike were in close agreement on a number of important Issues such as control of behavior by stimuli in the environment and the in effectiveness of punishment, there are important differences between them. For
example, the dependent variable in Thorndike 's leaming exp eriments (his measure of the extent to which learning took place) was time to solution. Thomdike was
interested in measuring how long it took an animal to perform whatever task was
~ecessary to release it from confinement. Skinner, in contrast, used rate of respondmg~s Ius dependent vanable. Other differences between Skinner's operant condil:lonmg and Thorndike's instrumental conditioning illustrate that the two
approaches are quite distinct and that the terms operant and instrumental are not to
be us~d int~rchangeably.. In th e history of learning theory, Skinner's operant conditlonmg differed so radtcally fiom Thorndike's instrumental conditioning that it
was co~stdered revolunonary. The differences between operant and instrumen tal
condttlonmg are summaiized in Table 5-1.
TABLE 5-1

Differences between Arrangements for Instrumental Conditioning

and for Operant Conditioning

CHARACTERISTIC

INSTRUMENTAL

Location of behavior
Methodology

Maze, runway, puzzle box

Procedure

Subject is replaced in
apparatus to begin each trial in

Display
Data display

Discrete trials

a sess ron
Learning curve
On-trial performance against trials

D ata source

Average of performance of group

of subjects

Statistics?
Is a control used?

Yes: significan ce test

Yes: not administered the treatment
variable or factor

OPERANT

Operant c hamber
Free responding
Subject is placed in the apparatus only to
begin a session
Cumulative record

Cumulative freq uency against time

.I

Although Pavlov (1927, pp. 384-386) had done some work with partial reinforcement, using classical conditioning, it was Skinner who thoroughly investigated
the topic. Skinner h ad already published data on the effects of partial reinforcement when Humphreys (1939a, 1939b) startled the psychological world by showing that the extinction process was more rapid following 100 percent
reinforcement than after parti al reinforcement. That is, if an orga nism receives a
reinforcer every time it makes an appropriate response during learning and th en
is placed on extinction , it will ex tinguish faster than an organism who had only a
certain percentage of its correct responses reinforced during acquisition. In
other words, partial reinforcement leads to greater resistance to extinction than
continuous, or 100 pe rcent, reinforcement, and this fact is called the partial reinforcement effect (PRE).
Skinner studied the partial reinforcement effect extensively and eventually
wrote a book with Ferster called Schedules of Reinforcement (Ferster & Skinner, 1957).
This book summarized years of research on various types of partial reinforcement.
Seven schedules of reinfo rcement have become th e most common and they are described below.
1. Continuous Reinforcement Schedule Using a continuous reinforcement
schedule (CRF), every correct response during acquisition is reinforced. Usually in
a partial reinforcem ent study, the animal is first trained on a 100 percent reinforcement schedule and then switched to a partial reinforcement sch edule . It is
difficult to bting about the acquisition of any response when partial reinforcement
is used during th e initial uaining period.
2. Fixed Interval Reinforcement Schedule Using a rLXed interval reinforcement
schedcle (FI), the animal is reinforced for a response made only after a set interval
of time. For example, only a response following a three-minute interval is reinforced. At tl1e beginning of the fixed time interval, the animal responds slowly or
not at all. As the end of the time interval approaches, the animal gradually increases its speed of responding, apparently anticipating the moment of reinforceme nt. This kind of responding produces a pattern on the cumulative recording
referred to as the fixed-interval scallop. Such a pattem is shown in Figure 5-7.
The beha,~or of an animal under this schedule is somewhat similar to tl1e way
a person behaves as a deadline approaches. After putting off a certain task as long
as possible, the due date is rapidly approaching and activity increases accordingly.
Often a stude nt preparing a term paper will act in this manner.

Individual-subject performance
No

Subject's pretreatment baseline serves as

a comparison value

(Bringman , W.G., LOck, H.E., Miller, R. & Early, C. E. [Ed s. )[ 1997). A pictoro/ history o{psycholo!')'. Carol Stream, Illin ois.
Quintessence Publishing Co.)

3. Fixed Ratio Reinforcement Schedule A rLXed ratio reinforcement schedule

(FR) occurs when every nth response that tl1 e animal makes is reinforced. FR5, for
example, means that th e animal will be reinforced at every ftfth response. Here the
important factor in d etermining when a response is reinforced is the number of responses made. Theoretically, an animal on a fixed interval schedule could make

96

CHAPTERS

just one response at the end of th e interval and be reinforced each time it responds. With a fixed ratio schedule, this is not possible; the animal must respond a
fixed number of times before it is reinforced.

For both the FI and FR reinforcement schedules, a reinforced response is followed by a depression in the rate of responding. This is called the postreinforcement
pause. There is considerable speculation about why such a pause exists. Perhaps the
animal learns that the responses immediately following a reinforced response are
never reinforced. However, the scallop on the cumulative recording of an FI schedule is usually not found on that of an FR schedule. The FR schedule usually generates a steplike cumulative recording, indicating that the animal temporarily ~tops
responding after a reinforced response and then, at some point, resumes responding at a rapid rate: Such behavior has been characterized as "break and run." A cum~.tlative recording generated by an animal under an FR schedule is shown in
Figure 5- 7.

4 . . Variable Interval Reinforcement Schedule With the variable interval reinforcement schedule (VI), the animal is reinforced for responses made at the end
of time intervals of variable durations. That is, rather than having a fixed time interval, as with FI schedules, the animal is reinforced on the average of, say, ' every

Fixed Variable
Ratio Ratio

Fixed
Interval

Variable
Interval

BURRHUS FREDERIC SKINNER

three minutes, but it m ay be reinforced immediately after a prior reinforcement,

or it may be reinforced after thirty seconds or after seven minutes. This schedule
eliminates the scalloping effect found in FI schedules and produces a steady, moderately high response rate . A typical cumulative recording generated by an animal
on a VI schedule is shov.m in Figure 5-7.

5. Variable Ratio Reinforcement Schedule The variable ratio reinforcement

schedule (VR) eliminates the steplike cumulative recording found with the FR
schedule and produces the highest response rate of the five schedules considered thus far. With the FR schedule, an animal is reinforced after making a specific number of responses , say, five. With the VR5 schedule, the animal is
reinforced on the average of every five responses; thus, it might receive two reinforcers in a row or may make ten or fifte e n responses without being reinforced .
A cumulative recording produced by an animal under a VR schedule is shown in
Figure 5- 7.
The VR reinforcement schedule is the one governing the behavior of gamblers at a place like Las Vegas. The faster one pulls the handle of a slot machin e,
for example, the more frequently one is reinforced.
To summarize, continuous reinforcement yields the least resistance to extinction and the lowest response rate during training. All partial reinforcement schedules produce greater resistance to extinction and higher response rates during
training than continuous reinforcement. Generally speaking, the VR schedule produces the
highest response rate , FR produces the next
highest rate, then VI, followed by FI, and finallyCRF.
6. Concurrent Schedules and th e Matchi (
Skinner (1950) trained pigeons to eck
o . erant keys that were availabl at the
but that delivered rein~ cements
rocedure is

"'c
Q)

"'
0

"-

a:"'
Q)

Responses per Min

-~

97

1ii

:;
E

"

(.)

Time

Time

FIGU_RE 5-7 Typical cumulative recordings generated by fixed ratio, variable ratio,
fixed onterval, and variable interval reinforcement schedules. The slash marks in the
recordings indicate a reinforced response.

Richard J. Herrn stein. (Courtesy of Richard

J. Herrnstein.)

Herrn-1994) quan tified the r tionship

betwe
reinforcement and perfo ance
un r concurrent schedules and provid " dir tion for operant research for about ~

98

CHAPTERS

~ext

thirty years . He refined Skinner's earlier observation by noting that un d

c ncurrent schedules, the relative frequency of behavior matches the relativ
equ 1cy of reinforcement. This relationship is referred to as Herrnstein 's atching 1 . The equation expressing matching is written as follows:

B1+B2+B,=k. It is possible, then , to write an equation that expresses the respon e

rate for any single behavior:

where B1 is the equency of pecking at key 1 and R 1 is the fre uency of reinforcement for that be a\~ Or, and so on. Matching is illustrated in lgure 5-8.
In two subse uent papers, Hennstein (1970, 1974. extended the implications of the matchi
law. First, he noted that even i a testing situation where
there are two keys for igeons to peck, the pigeons ei;gage in behaviors other than
pecking. He included ill se extraneous behaviors (JY,) and the reinforcements that
maintained them (R,) in e matching equation:

~
B1 +

R,
R1 +R,+R,

+B,

'.

Furthennore, he made the ass

sum of the rates of all behavior

,;

BURRHUS FREDERIC SKINNER

p tion that in an y given testing situation , the

en together is a constant (k). That is,

where _r:-e_ is the sumii_~ation of the frequencies of reinforcement

occurnng\ mder the circumstances.
This exRression is called Herrnstein's equation or Herrn,2 in's hyperbola, in
reference to tl\ e function it generates for different values of(< kl and R,. AE shown
in Figure 5-9, H ~nstein 's hyperbola IS a form of the learnmg curve-a m~themat
ical statement of ~mdike:s law of effect.Figure 5-9 als6 ~eveals the !ogre of the
matl1emaucal express~on as It applies to a Simple Situatid'i:1 wrtl1 one operant behavior (B1), two different lues of extraneous behaviOr;(B,), and assooated extraneous reinforcements (R.;;) l;mt only one value of k. ~/the case illustrated on the left,
there is little extraneous b ehaVIor and remforc<:lment (R,=5). In tl1e case on the
right there are increased effects of extraneous leinforcement (R,=20) .
'Recall that the sum of rates,_ of all possib{e behaviors is a constant (h). Therefore, the asymptote, or maximum_rate, o~t>he operan: behavior in either case is k.
In each instance, as the rate of remfqrcement for B1 mcreases, more and more of
the total behavioral output is in the r m of B, and B, necessarily decreases toward
zero (0) . Notice that the effect of uamwus reinforcement is distraction from B1
VVhen values of extraneous reinforcemen and the accompan)~ng extraneous behavior are high , the teaming cur<:e for B1 rises more slowly and asymptotic performance is delayed. Similarly, when more extran ous reinforcements are available,

RGURE 5-8

cr---o

Results of two pigeons pecking

on concurrent VI VI schedules.
The pecking of key A is plotted
rl(lative to the pecking of key B.
Tl't~ total possible number of reinf?!(ers from pecking both keys
is 40~ r hour; therefore, if pecking key ~ generated 1 0 percent
of the reuiJ[orcers (4), pecking
key B gene ~ted 90 percent (36).
Notice that b~e relative rate of
responding vel> nearly equaled
the relative rate'lof reinforcement. (From R/'Herrnstein,
"Relative and Absolute Strength
of Response as a Fu"l1ction offreq uency of Reinforcem'ent, "jour-

Pigeon #055

---- .. Pigeon #231

<(
>,

"'c

0
0>

-~

"0

Q.

a:"'
"'
0

..-~

.5

"'

1ii

a:

.~

.-~

.(ij

Qi

a:

,'

.1

.2

nal of the Experimental Anqlysis of

Behavior, 1961, 4, 267-272 .

.1

Q)

1ii

.3

.4

.5

.6

.7

.8

.9

1.0

a:

Copyright 1961 by the Sodety

for the Experimental Analysi of
Behavior, Inc.)

'

/;

;,

s,

s,

FIGURE S-9

Rate of Reinforcement

Rate of Reinforcement

~~

~~

High levels of extraneous reinforcement (on the right) interfere with

performance of a target operant behavior and prolong extraneous behaviors .

100

1/1

BURRHUS FREDERIC SKINNER

extraneous behavior (B,)

for BI.
As an example of the effects of extraneous behavior and rein rcement on
human performance, imagine two piano students trying to master aldifficult musi~1 passage. One student is alone with the instructor. Contrary t~the instructor's
ge\_leral policy, the other student has invited several friends to thl- Iesson. The first
stud\nt will be reinforced only by the instructor and will eam einforcement only
for co~ect musical performance. Therefore, extraneous beh,aviors and their reinforceme~ are minimized (as in Fi~ure 5-9, where R,;5) . l7he second student has
avarlable ~ore sources and potential types of reinforcerfient (as in Figure 5-9,
where R,;2Q) and can be reinforced for behaviors otl{er than musical performance. That ~tudent may engage in a number of nonu<usical activities to earn approval, attentw , and respect from the observing ends. Herrnstein 's equation
accounts for the bservation that the first student 'II improve faster and master
the musical passag sooner than the second.
Development o \ the matching law con~n s, and the current body of literature evolving ~om ~nstein's early mate ng observation is extensive. The
matchmg .equauo~ has o~en used to expla1 effects of delay, magnitude, quality,
and duratJ.on ofremforcements, as well as t ose of punishment (see Davison & McCarthy, 1988). However, th mechanism underlying matching phenomena continue to be debated (Herrnsteil!, 1990, 91 ; Staddon, 1991).
7. Concurrent Chain Reinforc
t Schedule VVhereas a concurrent schedule
of reinforcement is used to investi e simple-choice behavior, a concurrent chain
reinforcement schedule is used t inv~ti.gate complex-choice behavior. With concurrent chain schedules an ani l's belia,_vior during the initial phase of the experiment determines what sche ule of rei forcement it experiences during the
second, or terminal, phase.
One interesting findi g using concurren~hain schedules is in the area of selfcontro.l. Under ordina.J')' 'rcumstances, organis~ will clearly prefer small, immediate reinforcers over I ge, delayed reinforc~rs. ~~ing concurrent sche?ules, if
response A IS remforce by small, Immediate reinforC(S and response B IS remforced
b~ lar~e rei~orcert llowing a delay, response A will b,e overwhelmingly preferred.
LikeWise, usmg a co current ~hain s~hedule_ , if responding, to alternative A switches in
a schedule proVI ng small, Immediate reinforcers ar1d re'sponding to alternative B
switches in a sch dule producing large, delayed reinforcers, a~als will show a strong
preference fo ternati.ve A. Rachlin and Green (1972) , however, showed that under
certain eire stances, large, delayed reinforcers are preferred ov r small, immediate
reinforc~rs Ra~hlin =d Green ~ed a conc~rrent chain reinfo_rcel{lent schedule in
which pi ons uuually had a chmce of pecking one of two white dis\ s. Pecking the
whited' c ~n the left fifteen ti.mes (FR 15) was followed by a ten-secondl(llackout and
then choice between a red a.Jld a green disc. Pecking the red disc prod~ ed two second offood (a relatively small reinforcer) immediately, a.Jld pecking the een disc
pr duced four seconds offood (a relatively larger reinforcer) following a deJa offour
s conds. In the original choice phase of the experiment, if the white disc on th right

was responded to fifteen times (FR 15) , there was also a ten-second blackout a.Jld the
the opportunity to peck a green disc. Pecking the green disc produced a four-seco a
delay followed by four seconds of food. Under these expe1imental conditions, i~ as
ound that pigeons chose the white disc on the right 65 percent of the time, :{;s revesing the preference for small, immediate reinforcers. The experimental rrangemei used by Rachlin and Green is shown in Figure 5-10.
/

produce
10-second
blackout,

15 pecks
at left key

A peck at
red key (
immediat<;)l
producis
2 seyO'nds
o food,

followed by
a choice
between
red and
green keys.

~
~
/ :

<{:. -

f>,pck at
g(een key
/ produces
a further
blackout of
4 seconds
followed by
4 seconds
of food.

-------------.

:----------------- _________ /~-~ ~ @I

Seconds

10-Second Blaskoy(

~ ~--8 1
:

/'

I
/

10-Second Blackout

1/ 15 pecks
at nght key

Followed
by return
toinitial
conditions.

A~

fl:
~

produce
10-second
blackout,

~~

;--------------

~Sec~nds'

----1~
followed by
presentation
of green
key only
(other key IS
dark and

A peck at
reen key
p duces
a f ~er
blacko t of
4 secon

meffect1ve).

followed b\

::f::::l~n

FIGURE 5-10 The concurrent-chain reinforcement schedule used

and
Green. (From H. Rachlin, Behavior and Learning, p. 584. Copynght 1976 W. H .
Freeman and Company. Reprinted with permissio n.)

102

CHAPTERS

- '"Qlat is it that changed the preference for small, immediate rein

ed remforcers? The answer seems to be time . It has been sb~ that rer
111forcers lose the1r re111forcement value over time Thus an organ
ll _
.
. - .
_

may opt 10r a

sma
I emfm cer U: 1t 1s tmmedlatel)' available but will not plan t
ave one somettme
.
111 the future. If a dela'K!s involved, organisms tend to c~ ~&rger reinforcers over
smaller ones. Schwartz a'hd Robbins (1995) generaliz;;n:ese findings to humans:

~arge, dela

th~hoice betweel -~ to the movies and studying on a

Con:ider, for example,

parucular evenmg. We could Im

me j)>~~~;\o the movies involves a -small but

;mme~I~te reinf~rcement (an even I :(s entertainment) whereas studying i~volves a

arge e ayed ;emforcement (a
b
and an
8
thetween11stud)~ng
.
.
.
.
e sma

tmmed~ate rei

od e:xam grade). Given the choice at 7:45 PM

- h"'- - d
1

'
- .,

mo\Je, tu ~ tu ent, tke the ptgeon, might choose

rcement. But if the choice were required at 9 AM

so

that .now both remfo ers were going to be del ~d, the student might cho~s~ to
study. (p. 283)

~~~2

is beyo~d the scope of this text to explain how, it is interesting to

~ate ~~~r switchover 111 preference from small, immediate to I ge, delayed re:nfo_;;.y, s fmmd b~ Rachlin and Green (1972) is predicted by He ~stein's matchg'J.a" (see Rachlm, 1991, pp. 585-586, for an explanation).
It

Skin
bdiev~d that verbal behavior (language) can be explained "~thin the con-_/'
text of 1
fotcement theory. Talking and listening are responses that are in ~
enced by re forcementjust as is any other response. Any utterance, therefo , will
tend to be rep ted tf tt 1s remforced . Skinner classified verbal respons 111 terms
of how they were
ated to remforcement, that is, in terms of what
being done
111
order to be remfo ed. These classifications are discussed b1;
below.

l.Mand

The word mand mes from the fact that a deman s being made. VVhen the
demand IS met,
utterance (mand) is reinforced, and n t time the need arises
the person ts ely to repeat the mand.

BURRHUS FREDERIC SKINNER

10/

picture thereof, is the occasion upon wruch tile student: of zoology is reinforced when he
says teleost fish. There is no suitable tenn for tl1is type of operant. "Sign," "symbol," an
more technical terms from logic and semantics commit us to special schemes of ref; -

ence and stress the verbal response itself rather than the controlling relationship. he
1 vented tenn "tact" will be used here. The term carries a mnemonic suggestion f beha or which "makes contact witl1" the physical world. A tact may be defined as verbal
ope nt in which a response of given form is evoked (or at least strengthene by a particula bject or event or property of an object or event. We account forth trength by
showin hat in the presence of the object or event, a response of that f; nn is characteristicall einforced in a given verbal community. (pp. 81-82)
Generally sp king the tact involves naming objects or eve ts in the em~ron
ment appropriately, nd its reinforcement comes from othe people 's reinforcement of the match be een the environment and the verbal ehavior.

3. &Jwic Behavior E
someone else's verbal respon is repeated verbatim.
oic beha,~or is often a prerequisite to a more complicated rbal behavior; fore
pie, first a child must imitate
a word before the child can lea how that word - related to other words or other
events. Thus, repeating something s
as said is reinforced, and when this
response is learned, it permits the spe
more complex verbal relationships.
4. Autoclitic Behavior According to kinner (1957) , "The term 'autoclitic' is
intended to suggest behavior which is ase upon or depends upon other verbal
behavior" (p. 315) . The main functio of aut litic behavior is to qualify responses,
express relations, and provide a gr matical fr ework for verbal behavior.
The most severe critic of S 1mer's explana -on of verbal behavior has been
Noam Chomsky (1959) . Cho sky contends that nguage is too complex for a
child to have learned. Some rocess other than lear
g must explain all the verbal utterances that, say, at ee-year-old is capable of rna ng. G. A. Miller (1965) in
fact, points out that the e are 10 20 possible twenty-word ntences in the English
language, and it waul ake one thousand times the estimat d age of the earth just
to listen to them all bviously, says Chomsky, operant condi 'oningjust does not
explain the camp xity of our language capabilities. Chomsky's xplanation oflanguage develop
ntis that our brain is structured to generate Ia uage. The underlying gra
atical structure of all human languages reflects
underlying
brain struct re. That is, we are "wired" to produce grammatical utter 1ces,just as
Skinner
a com put can be wired to produce moves in a chess game . Chomsky a
seem to e continuing the nature-nurture debate launched by Plato and stotle:
Cho ky's deep-brain structures theory oflanguage acquisition represents e natur , or Platonic, side, and Skinner's \~ew that verbal behavior is shaped by
ment represents the nurture, or Aristotelian, side .

About tact, Skinner (1957) said,

Programmed Learning
This type of operant is exemplified when, in the presence of a doll a child fr ue 1Jtl
achieves some sort of generalized reinforcement by saying doU; or when a teleost fi 1 0 ~

Skinner, like Thorndike, was very interested in applying his theory of learning to
the process of education. To Skinner, learning proceeds most effectively if (1) the

CHAPTERS

104

information to be leanted is presented in small steps, (2) the learners are given
rapid feedback concerning the accuracy of their learning (i .e., they are shown immediately after a learning expetience whether they have learned the information
cotTectly or incorrectly), and (3) the learners are able to learn at their own pace, .
Skinner learned firsthand that these p1inciples were not being used in the classroom. He reflected on a visit he made in 1953 to one of his daughter's classes (Skinner, 1967): "On November 11, as a visitin g fatl1er, I was sitting in the back of the room
in an a~ithmetic class. Suddenly the situation seemed perfectly absurd. Here were
twenty extremely valuable organisms. Through no fault of her own, tl1e teacher was
violating a\most everything we knew about the learning process" (p . 406).
It is interesting to note that tl1e most common teaching technique is the lectme, and tl1e lecture technique violates all three of the above principles. Skinner
proposed an alten1ative teaching technique, programmed learning, which does incorporate all three principles. A de,~ce invented to present programmed material
has been called a teaching machine. The advantages of using a teaching machine
were outlined by Skinner (1958) as follows:
The machine itself, of course, does not teach. It simply brings the student into contact
with the person who composed the material it presents. It is a labor-saving device because it can bring one programmer into contact with an indefin ite number of stu-

dents: They may suggest mass production, but the effect upon each smdent "is
surprisingly like that of a private Ultor. The comparison holds in several respects. (i)
There is a constant interchange between program and st1.1dent. Un like lecmres, textbooks, and the usual audio-visual aids, the machine induces sustained activity. The student is always alert and busy. (ii) Like a good tutor, the machine insists that a given
point be thoroughly understood, either frame-by-frame or set-by-set, before the student moves on. Lectures, textbooks, and their mechanized equivalents, on the other

hand, proceed without making sure that the student w1derstands and easily leave him
behind. (iii) Like a good tutor, the machine presents just that mateJ;al for which the
student is ready. It asks rum to take only that step which he is at the moment best
equipped and most likely to take . (iv) Like a skillful tutor, the machine helps the student to come up with the right answer. It does this in part through the orderly construction of the program and in part with techniques of hinting, prompting,
suggesting, and so on , derived from an analysis of verbal behavior .. . . (v) Lastly, of
course, tl1e machine, like the pt;vate tutor, reinforces the student for every correct response, using this immediate feedback not only to shape his behavior most efficiently
but to maintain it in strengtl1 in a manner which the laymen would desCI;be as "holding tl1e student's interest." (p. 971)

BURRHUS FREDERIC SKINNER

105

time . For example, if a person has a weight problem and has difficulty losin_g
weight, that person may wish to arrange the em~ronment so tl1at losmg we1ght IS
reinforced. The person may, for example, sign over to another person somethmg
personally important such as money, a CD collection, a stamp coll ection , or favorite clotl1es. Taking money as an example, the person trying to lose weight may
put up, say, 100 dollars and draw up an agreement whereby the other person gives
back 10 dollars each week if tl1ree pounds are lost. Each week that at least three
pounds are not lost, tl1e person loses 10 dollars. The same kind of arrangement
can be made by utilizing anytl1ing important to the person, and the behav10r mvolved could as easily be smoking as losing weight.
The tenn contingency contmctingcomes from tl1e fact that an agreement (contract) is made tl1at certain activities will be reinforced that otherwise may not have
been. In other words, the contract rearranges the reinforcement contingencies in
the environment, causing t!1em to be responsive to behavior patten1S tl1at one
hopes to modify in some way.
Many beha,~or problems occur because our behavior is influenced more by
immediate reinforcers than by distant ones. For example, for some the taste of
food in the present is more reinforcing than the distant promise of a longer life if
one eats in moderation. Likewise, th e immediate effect of nicotine is more reinforcing than me promise of a longer life without smoking. Contingency contracting is a way of modifying behavior through current reinforcing contingen~1 es
ratl1er than distant ones. It is h oped that as desirable behavior is shaped by usmg
this procedure, the desirable behavior itself will be functional in obtaining rein.forcers from the social em~ronment. Not being overweigh t and not smoking both
can be very reinforcing, but the problem is switching the overweight person and
the smoker to another class of reinforcing expe1iences. Contingency contracting
can be a very effective tool in accomplishing this switchover. Once the switch in reinforcement systems has been made , however, the desired behavior is usually sustained by the social environment, md therefore the artificial reinforcement
contingencies are no longer needed.
Contingency contracting need not involve a second person; one cm follow
these procedures alone by gi'~ng oneself a "treat" of some kind each day one goes
without smoking, drinking, or overeating. For a more detailed discussion of contingency contracting, see Homme, Csanyi , Gonzales, and Rechs (1970).

Skinner's Attitude toward Learning Theory

We have more to say about programmed learning in Ch apter 16.

Contingency Contracting
Contingency contracting is an exten sion of Skinnerian thinking. Briefly, it involves
making arrangements so that a person gets sometl1ing wanted when that pe1~son
acts in a certain way. Some arrangements can be simple and cover simple behavior,
such as when a teacher says to a child, "If you sit quietly for five minutes, you can
go out and play." Other arrangements can extend over a much longer period of

Skinner believed that it is unnecessary to formulate complicated theories to study

human behavior, and he believed it is unnecessary to know the physiological correlates of behavior. He believed that beha,~oral events must be desoibed in terms of
ti1ings that directly affect behavior and that it is logically inconsistent to attempt to
explain behavior in terms of physiological events. For this reason, Skinner's
method of research has been called "the empty organism approach."
Skinner also mought that complex theories oflearning, such as Hull's (Chapter 6), are time-consuming and wasteful. Some day such theories may be useful m

106

CHAPTERS

BURRHUS FREDERIC SKINNER

psychology, but not until we have collected much more basic data. Our main concern at this time should be, Skinner believed, to discover basic relationships between classes of stimuli and classes of responses. Therefore, the use of theory in
studying the learning process cannot be justified (Skinner, 1950) :

The Need for a Technology of Behavior

i<

Research designed with respect to theory is also likely to be wasteful. That a theory
generates research does_not prove its value unless the research is valuable. Much
useless experimentation results from theories, and much energy and skil1 are ab-

sorbed by them. Most theories are eventually overthrown, and the greater part of
the assoCiated research 1s d1scarded. This could be justified if it were true that productJve research requires a tl~eory-as is , of course, often claimed. It is argued that
research would be aitnless and disorganized without a theory to guide it. The vie\v is

suppo;ted by psychological texts which take their cue from the logicians rather than
e.mpmcal soence and describe thinking as necessarily involving stages of hypotheSIS, _deduction, expenmen~ test, and confirn1ation. But this is not the way most sci-

entiSts actually work. It ts possible to design significant experiments for other

reasons, and the possibility to be examined is that such research will lead more
directly to tl1e kind of information wh ich a science usually accumulates.
(pp. 194-195)

Skinner's (1953) approach to research was to do a functional analysis between stimulating events and measurable behavior:

107

Skinner felt very strongly that a carefully worked-out behavior technology could
solve many human problems, yet many people would oppose such a technology because it seems to challenge a nwnber of our cherished beliefs about ourselves, especially that human beings are rational, free, and dignified. Skinner (1971)
believed that these beliefs were interfering with the solution of our major problems and also preventing the development of the very tool that could solve them:
vVhat we need is a technology of behavior. We could solve our problems quickly
enough if we could adjust tl1e growth of the world 's population as precisely as we adjust the course of a spaceship, or improve agriculture and industry with some of tl1e
confidence with which we accelerate high-energy particles, or move toward a peaceful

world with sometl1ing like the steady progress 1\;th which physics has approached absolute zero (even tl1ough both remain presumably out of reach). But a beha,;oral
technology comparable in power and precision to physical and biological technology
is lacking, and tl1ose who do not find the very possibility ridiculous are more likely to
be frightened by it than reassured. That is how far we are from "understanding
human issues" in the sense in which physics and biology understand their fields, and
how far we are from preventing the catastrophe toward which the world seems to be
inexorably mm;ng. (p. 5)
Elsewhere, Skinner (1953) said,

The external variables of which behavior is a function provide for what may be called
a causal or functwnal analys1s. We undertake to predict and control the behavior of
the individual organism. This is our "dependent variable"-the effect for which we are
to find the cause. Our "independent variables"-the causes of behavior-are the ex-

ternal conditions of which behavior is a function. Relations between the two-tl1e

"cause-and-effect relationships" in behavior-are the laws of a science. A synthesis of

The traditional view of human nature in V\Testern culture is well known . The concep-

tion of a free, responsible individual is embedded in our language and pervades owpractices, codes, and beliefs. Given an example of human behavior, most people can
. describe it immediately in terms of such a conception. The practice is so natural that
it is seldom exatnined. A scientific formulation, on the other hand, is new and

these laws expressed in quantitative terms yields a comprehensive picture of the organism as a behaving system. (p. 35)

strange. Very few people have any notion of the extent to which a science of human
behavior is indeed possible. In what way can the behavior of the indh;dual or of
groups of indi,;duals be predicted and controlled? \~That are laws of behavior like?
What over-all conception of the human organism as a behaving system emerges? It is

Thus, Skinner manipulated hours of food or water deprivation and noted the
effect on the rate ~~~th which the lever-pressing response was l earned; or he observed the effect of schedu les of reinforcement on response rate or resistance to
extinction . In interpreting the results of his research, Skinner stayed very close to
the data; that is, if partial reinforcement produces greater resistance to extinction
than does 100 percent reinforcement, that is a fact and that is all that can be said.
In other words, Skinner did not attempt to explain why this is the case.
Even in deciding what to inves tigate, Skinner claimed he was not guided by
theoretical notions but rather used a hit-and-miss process. He tried first one thing
and then another. If he saw that one line of research was not producing anything
worthwhile, he would shift to something that looked more fruitfu l, and he would
continue in this trial-and-etTor fashion until he hit on something of value. This
rather liberal attitude toward scientific investigation was summarized in Skinner's
(1956) article "A Case History in Scientific Method."

only when we have answered these questions, at least in a preliminary fashion, that we
may consider the implications of a science of human behavior with respect to either a
theory of human nature or the management of human affairs. (pp. 9-10)

In an article entitled "Wbat Is Wrong with Daily Life in the Western World? "
Skinner (1986) renewed his plea for the utilization of behavior technology in solving human problems. In this article , Skinner argued that five cultural practices are
eroding the strengthening effects of reinforcement contingencies. These cultural
practices are : "(a) Alienating workers from the consequences of their work;
(b) helping those who could help themselves; (c) guiding behm~or \\~th rules
rather than supplying reinforcing consequences; (d) maintaining aversive sanctions of government and religions with long-deferred benefits for the individual;
and (e) reinforcing looking, listening, reading, gambling, and so on, while
strengthening few other behaviors" (p. 568) .

108

CHAPTER 5

According to Sldnner, the many problems caused by these cultural practices

could be solved by strengthening desirable behavior by using the principles derived from an experimental analysis ofbeha,~or, that is, by usi n g tl1e principles described in tl1is chapter.
Sldnner's theory of learning has had, and is having, a profound influence on
psychology. No matter what area of psychology one studies, one is apt to find reference to some aspect of Sldnner's work. As we noted in Chapter 2, a characteristic
of any good theory is that it generates research, and Sldnner's theory h as certainly
done that. vVe now review the work of an important research er who has been influenced by Sldnner's work.

The Relativity of Reinforcement

David Premack
Traditionally, reinforcers have been thought of as stimuli. A primary reinforcer is
usually thought of as being related to an organism's survival, and a secondary reinforcer is a stimulus that has been consistently paired "~th a primary r einforcer.
Premack, however, has suggested iliat all responses should be thought of as potential
reinforcers. Specifically, he suggests that any response that occurs Mth a fairly high
frequency can be used to reinforce a response
that occurs with a relative ly lower frequency.
Using Premack's notion of reinforcement, one
would allow an organism to engage freely in
whatever activities it wanted to and carefully
record what acti,~ties were engaged in and
Mth what frequen cy. Afterward, the various acti,~ties that the organism engaged in would be
arranged in a hierarchy. The activity iliat was
engaged in most frequently would be listed
first, the next frequently engaged in acti\~t:y
would be listed next, and so on. By referring to
this list, the experimenter would know exactly
what could and could not be used to reinforce
that particular organisiTL Say, for example, it
was found that in a twenty-four-hour period,
the activity engaged in most frequently by a rat
was eating, then drinldng, then running in an
activity wheel, ilien grooming, and finally gazing out of the cage. According to Premack, allomng the animal to eat could be used to
reinforce any of the other acti\~ties . For examDavid Premack. (Courtesy of David
Premack. )
ple, if the animal was allowed to eat each time

BURRHUS FREDERIC SKINNER

109

it groomed itself, grooming would increase in frequency. Likewise, allowing the animal to groom itself could be used to reinforce tl1e animal for looking outside ilie
cage. The opportnnity to look outside the cage, however, could not be used to reinforce any of ilie other acti,~ ties because they all occuned with a greater frequency than th e response oflooldng outside the cage.
According to Premack, tl1e way to find out what can be used as a reinforcer is
to observe the organism's beh avior while it has the opportunity to e ngage in any
number of acti,~ties, and th e acti,~ti es that it engages in most often can be used to
reinforce the activities th at it engages in less often.
In summary we can say tl1at if one acti\~ty occurs more often tl1an another, it
can be used to reinforce the acti,~ty that occurs less often. This is calle d ilie
Premack principle, and it seems to hold for humans as well as for lower o rganisms.
To test his theory, Premack (1959) allowed iliirty-one first-grade children eitl1er to play a pinball machine o r to operate a candy dispenser as often as they
wanted. Some of ilie childJ:en played mainly "~th the pinball m achine, and they
were called manipulators. The childre n who were primarily interested in tl1e candy
dispenser were called eaters. The first phase of ilie srudy merely determined the
children's preferences for these two events.
In the second phase of the study, the groups of manipulators and eaters were
each subdivided into two gJoups . One group was placed on manipulate-eat contingencies, in which the children had to play the pinball machine before tl1ey were
allowed to operate the candy dispenser. The other group was pl aced on eatmanipulate contingencies, in which they had to operate the candy dispenser before they could play th e pinball machine. It was found that for the manipulators
th e manipulate-eat anangement made little difference in their behavior. They simply went right on playing th e pinball machine as before. Under the eat-manipulate
conditions, however, the frequ ency of eating went way up for ilie manipulators because tl1ey now had to eat in order to play the pinball machine. Like"~se , for the
eaters the eat-manipulate condition made litde differen ce. They simply went on
eating candy as before. But under the manipulate-eat conditions, their frequency
of playing the pinball machine went way up. Thus, Prem ack found support for his
contention that a less frequently e n gaged-in acti,~ty can be reinforced by the opportunity to engage in a more frequently engaged-in activity.
W11en preferences change, the reinforcers also change. For example, as long
as an animal is hungry, it mil eat frequently, and therefore th e opporrunity to eat
can be used to reinforce any number of activities. W11en the animal is satiated,
however, d1e frequency of its eating Mil decrease, and d1 e opportunity to eat mil
become ineffective as a reinforcer. Premack (1962) demonstrated d1e reversibility
of reinforcement in a study involving a running response and a drinldng response.
It was found d1at if animals were deprived of water for a considerable length of
time , tl1ey would rum an acti,~ ty wheel in order to gain access to water. But iliey
would not increase their drinldng to run in the activity wheel. That is, d1inking reinforced running, but running did not reinforce drinking. This is what one would
predict from traditional reinforcement theory. Premack also found that if an animal was allowed to drink all the water it wanted but was prevented from running in

CHAPTERS

110

the

acth~ty

wheel, the situation was reversed. Under these circumstances, drinking

increased if it resulted in ha~ng the opportunity to run, but running did
not increase if it allowed the animal to drink. That is, now running could reinforce
drinking but not \~ce versa.
The implications ofPremack's research are far-reaching. For one thing, what
can act as a reinforcer becomes a very personal and continuously changing thing.
The teacher can apply this knowledge by noticing indi~dual children's preferences in a free-choice situation and determining their reinforcers accordingly. For
one child, the opportunity to run and play may be a reinforcer; for another child,
pla)~ng with clay may be a reinforcer. The idea of recess as a way to improve the
performance of the class as a whole will need to be looked at more carefully. For
examples of how the Premack principle can be used to control the beha~or of
schoolchildren, see Homme, DeBaca, Di~e. Steinhorst, and Rickert (1963).
acti,~ty

evisions of the Premack Principle

You ay remember from Chapter 4 that Thorndike's definition of a satisfier ... /
criticiz because it appeared to be circular. When Skinner defined a reinfor
any even
at increases the probability of a response, he neatly avoid~d
t e problems of des ibing physical, aesthetic, or chemical characteristics of mforcers.
iological asSimilarly, he a ided the difficulties associated with desCiibing th
pects of reinforce ent. Unfortunately, his definition has also bee . criticized as circular. Walker (196 suggested that Skinner's operational d mition has elusive
and even "magical" qu ities in that it suffices when a speci procedure produces
reinforcing effects, but it annat explain cases in whicl
at same procedure has
no effect or when it results 1 a decreased frequency
responding. Similarly, Gregory Kimble, from whom web owed our initial
mition of learning (see Chapter 1), indicates that food is re1 arcing at
beginning of a meal, is neutral
midmeal, and is punishing at the en (Kimb , 1993). He even suggests facetiously
that the concept of reinforcement "s
r rom terminal ambiguity and that it is a
nominee for mercy killing" (p. 254).
The traditional defense when
e or Skinner are attacked as circular
is Meehl's (1950) "transituation
argument. ccording to this argument, a satisfier or reinforcer in one situa n can be shown to
nsituational nature of r nforcers or satisfiers protects
arion . It is argued that the
them from claims that
1r definitions are circular. 0
of the important findings
derived from Premac s research is that the transituationa rgument is inadequate
if not entirely inc ect. For example, if an animal prefers t spend 30 percent of
its time eatin
0 percent drinking, and 10 percent in an cti~ty wheel , the
Premack pri ciple tells us that we can use drinking to reinfor activity in the
wheel. li
straightforward application of Premack's principle,
cannot use
to reinforce eating in that specific animal, and we may not be ble to use
mg to reinforce wheel-running in an animal with different acti~ preferces. This demonstrates a first flaw in the transituational argument. Researc conducted by William Timberlake and his associates (Timberlake, 1980; Timberla!( &

/r:

BURRHUS FREDERIC SKINNER

111

Allison, 1974; Timberlake & Farmer-Dougan, 1991) is insm.1ctive "~tl1 respect to

failure of the t:ransituational argument, the limits of the Premack principle, and
the nature of reinforcement.

William Timberlake

Ti . berlake (Timberlake, 1980; Timberlake & Farmer-Dougan, 1991) distinguisl s

between the probability-differential hypothesis, tl1e position taken by Premack nd
t11e disCi_quilibrium hypothesis, a position tl1at logically follows from Pre ack's
(1962) di;.inking-acti~ty wheel study described earlier. In contrast with P mack's
suggestion that preferred acti,~ties can reinforce less preferred acti~ti ; the disequilibrium hypothesis states that any acti~ty can be a reinforcer if
ontingency
schedule cons~ins an animal's access to that acti~ty. Imagine
t we watch a
freely beha~ng r~ for sev~ral days . Let us say th~t, as before, the t spends 30 percent of its waking thne eatmg, 20 percent of 1ts tm1e dnnk.ing, d 10 percent of Jts
time running in an acti~ty wheel. It distributes the rema 1g 40 percent of its
time among a number of other acti~ties . According to T berlake, this proportional distribution of activ.i.,ties constitutes an equilibrium a state of acti~ty-balance
freely maintai~ed and preferred by the animal. If we es)'iblish a contingency schedule such that ume devoted to eat:1ng IS reduced bel9~ the baselme of 30 percent,
we have created disequilibrium, a condition ;!h will have motivational consequences. Under this condition of disequilibri
, eating can be used as a reinforcer for any other acti~ty. and 1\will contj ue to have reinforcmg properues
until the animal has restored baselin ~ quirllrium such that 30 percent of its time
is allo tted to eating.
.
On t11e other hand, the diseqt. ibrium hypothesis pre~icts that wheelrunning, t11e least probable of the ac mt1es apove, can also be a remforcer. For this
st somehow be imposed in which wheel runto happen, however, a schedule
ning is reduced below its 10 per nt baseline, thus producing disequilibrium. As
was the case with eating, wheel- nning can be a n;Jnforce~ for any other beha,~or
until wheel-nmning retums t 10 percent and equi!}~num IS restored.
The disequilibrium ru othesis also delineates conditions under which a specific acti~ty can become unishing. To produce punishment, a schedule must be
designed in which pe rmance of one acti,~ty serves to increase another acti~ty
above its baseline. Irnl gine that the rat in our example is given food every time that
it runs in the acti'}t{. wheel. As long as eating takes less than 30 percent of tl1e animal's time, food ill reinforce wheel-running. However, ifwheel-!;unning produces
conditionsif
n ich eating acti~ty would exceed 30 percent of the animal's time,
wheel-runni g will decrease under the wheel-food conungency. Eaun has thus become a p isher.
Ti berlake's position pro~des important new perspectives about reinforcement nd reinforcement contingencies. Like Premack's, Timberlake 's research
sho s clearly that the transituational argument concerning reinforcement is .
r t. Further, from this perspective the role of a contingency schedule is to
uce disequilibrium rather than to pro~de information relating a response

112

CHAPTER 5

rcer or to provide contiguity between a response and a reinforcer. And' finally, fi

Timberlake 's research we see that deprivation of food or water per se
are not essen !:I o make these substances reinforcers. Rather it is rest:I;ction of aco make them reinforcers.
cess to them that sen
AJthough the position
en by Premack and Timberlake are improvements
on the older idea that "reinforcer
e things that reinforce ," we are still left with a
number of unanswered questions. Fo
cample, neither position addresses the
question of baseline preferences. "Wl:!y shou
at spend more time eating than
drinking? The answer can nolo ger be "Because it r
ore reinforcing to do so!"
There is no doubt. that Skinnerian notions have
and are having, farreaching theoretica and practical implications. Recently, howev - there has been
a growin~ reoogniti~n of the limitations of operant ~rincipl es in mo il'yi~behav
ror. ~e next sectiOn we exarrune why operant pnnoples seem to have r ed
~ppli.cability.

BURRHUS FREDERIC SKINNER

to perform many different tricks, and

trained animals were put on display at
conventions, at amusement parks, and
'~sion. As of 196 1, tl1e Brelands
ing conditioned thirty-eight
over six-thousand animals), in
ens, pigs, raccoons, reindeer, co.tl<.<>toos.
poises, and whales.
Everything seemed
the Brelands until they
to exper;ence
breakdowns of
beh avior. Their
that they
in an article (Erewhose title, "The Misbewas a parody of the title
major work, The Behavior of

The Misbehavior of Organisms

found that although their
initially highly conditionable,
instinctive behavior would appear
with what had been learned. For
Marian Bai ley. (Cou rtesy of Animal
an attempt was made to train raeBehavior Enterprises.)
pick up coins and deposit them into a
metal box. Conditioning a raccoon to
pick up a single coin was no
the metal box was int:Ioduced, and
that is when the problem
seemed to have trouble letting
the coin fall into the box.
the coin inside of the container,
take it back out, and
Eventually, however, tl1e
raccoon released the
its food reinforcement.
The next phase in
two coins into the metal
box before receivin
tl1e raccoon could not let
go of the two
dip them into the container, and
tl.1em. Th e n.1bbing beha,~or
more and more
pron
though it delayed or even prevented r<>lmronrPlm
. The Brelands con
that conditioning a raccoon to place two
It seemed that the innate beha,~ors
to be overcome by operant conditioning princ:ipll e~~II
a raccoon's innate tendency to wash and manipulate its
>u<cct:>jiiJ:uJJI)' with the leamed response of placing one or more coi
.LJ<Cl d"'L"

Vve saw in the last chapter that Thorndike concluded that the same laws of I mng apply to all mammals, including humans . Skinner, like many other I ming
tH rists, agreed with Thomdike's conclusion. After observing ho different
of animals performed under a certain schedule of reinforce ent, Skinner
mmented, "Pigeon, rat, monkey, whkh is which? It d sn't matter. Of
course, the species have behavioral repertories which are
different as their
anatomies: Bu once you have allowed for differences in e ways in which they
e environment, and in the ways in rch they act upon the enmake contact wi
vironment, what re ns of their behavior sh ows as
ishingly similar properties"
(pp. 230-231 ). Skinner ent on to say tl1at one n also add the performance of
mice , cats, dogs, and hum
children and the urves would still have more or less
the same characteristics.
The alternative to believin
at t same laws of leaming apply to all mammals seems to necessitate going bacK: the concept of instinct, which the behaviorists attempted to bury forever. Th e b ~'~ng in the existence of instincts say that
differen t species have differen born ten1! ncies that interact with or even negate
the laws of learning. In oth words, because f their innate beha,~or tendencies,
certain species can be c Clitioned to do some t gs but not others. According to
this point of '~ew, so e responses should be easie o condition for some species
than for others b a use the responses of interest rna occur more naturally for
some species tl n for others.
Curre
interest in how innate beha,~or tendencies 1 eract witl1 leaming
principl . as been stimulated by two of Skinner's ex-associate ~rarian Breland
(now anan Barley) and Keller Breland. Arnled vvith a knowledg: ~perant princi s, the Brelands moved from Minnesota, where they had worked wi Skinner,
o Arkansas, where they started a business called Animal Behavior Enterp ses. By
using operant techniques the Brelands were able to train a wide variety of an4llals

Another example of the misbehavior of organisms involved the

to pick up large wooden coins and deposit them in a large "piggy bank." The
coins were placed several feet from the bank, and the pig had to transport them to
the bank before receiving reinforcement. Early conditioning was very effective, and
the pigs seemed eager to perfmm the task. As time went on, however, the an ima1s

114

CHAPTER!

performed more slowly, and on their way to the "piggy bank" they would repe tdly drop the coin, root it (push it along the ground with their snouts), pick i p,
d1 pit, root it, toss it in the air, and so on. The Brelands first believed that s'11 beha r may have been the result of low drive, so they intensified the dep vation
sche 1le that the animals were on, which only intensified the animal's isbehavior. Ev tually it took the pigs about ten minutes to transport the coin a distance
of about 'x feet, even when such delays postponed or eliminated re' forcement.
Again, it a eared that the animal's instinctive behavior associated ' th eating became more p werful th an the behavior it had learned.
From the e and other similar observations, the Brelands 961) concluded,
"It seems obviou that these animals are trapped by strong i stinctive behaviors,
and clearly we hav here a demonstration of the prepotencx of such behavior patterns oyer those wh1
have been conditioned" (p. 185). 1e Brelands called the
tendency for innate ]j 1avior pattems gradually to disB ce learned behavior instinctual drift, which the describe as follows:

llenges three assumptions made by the

behaviorists, namely, (1) that anim
to the learning situation as a tabula
msa. (bla nk tablet), (2) that differ ces am g various species are unimportant,
and (3) that any response can be onditioned t any stimulus. Rather than making
these assumptions, the Brehm
(1961) conclu , "After 14 years of continuous
conditioning and observatio of thousands of ani als, it is o ur reluctant conclusion that the beha1~ or of a species cannot be adeq ately understood, predicted,
or controlled without kn ledge of its instinctive pa ems, evolutionary history,
and ecological niche" ( 126).
Thus, we are o e again confronted with the age- ld empiricism-nativism
con troversy: Is beha1 'or leamed or is it genetically detennin d? The phenomenon
of instinctual drift eems to indicate that at least for some spe es, beh a1~or can be
nudged only a li ited amount from its instinctual basis before 1 stinctual tendencies override I , rned tendencies as the most powerful detenni rs of behavior.
What about mans? Do we have within us the remnants of our ev utionary past
toward whi we pet;odically drift? Can culture, society, or circumsta . ces push us
before we resort to more primitive forms of beha1~or? The
swer dewho is being asked. Many learning theorists such as Skinner ould say
no. 0 ers such as Freud and the sociobiologists would say yes.
:Another phenomenon that seems to show the importance of instincthe beha or in a learning situation is autoshaping. We saw earlier in this chapter that e
s aping process can be used to encourage an anim al to make a response in a situ -

BURRHUS FREDERIC SKINNER

tion that it ordina,;Iy would not make. To do so, the experimenter reinforce ineasingly closer approximations to the desired beha1~or until the desired be avior
rformed by the an im a:!. In the case of autoshaping, however, the anim seems
to s
e its own behavior. For example, Brown and J enkins (1968) foun that if a
pigeo was reinforced at certain intervals, regardless of what it was doin (noncontingent inforcement), and if a disc was illuminated just prior to the resentation
of the rei1 orcer (in this case, food), the pigeon learn ed to peck a th e disc. The
question is, Thy did the pigeon learn to peck at the disc when it ad never been
reinforced fo doing so?
One atte
t to account for autoshaping has likened it to uperstitious behavior, sa}~ng that t
pigeon may have been pecking at the dis just before food was
delivered, and , th e fore, pecking the disc would be main med as a superstitious
response. One prob m with this explanation is that al st all pigeons peck the
disc under tl1ese circt stances. It seems that if supe titious behavior were involved, some pigeons w uld peck the disc, ot11ers ' uld turn in circles, others
would peck other parts of e. test ch amber, and so o . A second explan ation of autoshaping has been based o classical conditioni g principles. Accordin g to this
explanation, t11e illuminated isc becomes a se ondary reinforcer because of its
proximity to fo od , a primary re orcer. Und the circumstances described thus
far, this explanation is reasonabl except th it does not explain why t11e pigeon
would peck at the disc. Earlier in t s cha er we saw that, indeed, discriminative
stimuli (S 0 s) become secondary rei1 rc s and thus can be used to maintain behavior, but why the animal should resp d overtly to t11e secondary reinforcer as if
it were the primary reinforcer is not cl ar.
Iilli s (1969) casts further doubt on explaAn expe,;ment by Williams an
nations of autoshaping as eith er supersti us or a classical con ditioning phenomenon. In th eir expe1im ent, lilliams arid lilliams arranged t11 e situation so
that pecking at tl1e li ghted di actually prroent . reinforcement from occurring.
Food was presented to the pi ons every fifteen se nds, un less th e pigeon pecked
at the illuminated disc, in 1 ich case food was "~thH ld on t11at trial. In this study,
pecking at the illuminate disc was nroer followed b) einforcement. In fact, the
more the pigeon pecke at t11e disc, t11e less food it rec ved. According to the explanations of autosha ng in terms of both superstiti ous eha1~or and of classical
condition ing, th e e erimental aJTangement in this study s ould have elimin ated
or, at least, drastic y reduced disc pecking. It did not, howev . The pigeons went
right on peckin t the disc at a high rate. In fact, for some pt eons disc pecking
occu rred so fre uently t11at it 1~rtually elimin ated all reinforcemer t.
A study y J enkins and Moore (1973) further complicates e situation. In
their study was found t11at if food was used as a rei nforcer, pigeons esponded to
th e disc 1 an eating posture, and if water' was used as a reinforcer, igeons responde o the disc "~th a drinking posture. In oth er words, when food
s used as
a rein rcer, the pigeons seemed to be eating the disc, and when water was 1sed as
a rei orcer, they seemed to be d1;nking the disc.
By the process of elimination , one is forced to 1~ew the autosh aping phe me on as involving instinctive beha1~or patterns. It can be assumed, for examp ,

BURRHUS FREDERIC SKINNER

116

117

for detailed study and analysis of reinforcement schedules and yielded a number of
new behavioral laws. Throughout his life, Skinner was steadfast in his insistence
that psychologists should avoid theorizing, especially about cognitive events, and
be content wiili descriptive accounts of behavior.

Criticisms

ing are only two examples of a gr ng recognition in psychology that d1e innate r
onse tendencies of
an orga. m interact vvifu the laws of learning. In other words, w t may hold true
for
e type of organism may not hold true for another type of orga m. Furtherore, what may hold true for a given organism at one developmental lev <~....may not
hold true for d1at organism at another developmental level. For a more detailed
elaboration of these points, see Seligman and Hager (1972).

Evaluation ofSkinner's Theory

Contributions
B. F. Skinner's. long and productive research program significantly influenced both
applied and purely scientific psychology. Compared to many od1er learning researchers, Skinner's system was straightforward and could be easily appl ied to
problems ranging from animal training to human behavior modification therapy.
Atanother extreme, his work led to the matching law and had an indirect impact
on current research in beha,~oral decision making.
Skinner's methodology was a deparrure from mainstream beha,~orism. Verplanck (1954) noted that Skinner's approach "not only differs from the others in
particulars of theoretical detail , but also represents a re-orientation toward d1e science" (p. 306). Whereas other researchers tended to conduct research on groups
of subjects, making nomotl1etic comparisons between different experimental conditions, Skinner utilized an ideographic approach in which single expe,;mental
subjects were observed for prolonged pe,;ods. This approach, along with almost
exclusive use of the cumulative record of responding, provided an alternative to
d1e dominant research method(s) in ilie field, and it resulted in the creation of a
specialized journal, journal of Expmi mental Analysis of Behavim. The method allowed

Some criticisms of Skinner's theory are more wananted than others. For example,
Staddon ( 1995) , who traces his lineage to Skinner as a forn1er srudent of Richard
Herrnstein, finds Skinner's influence in a number of societal problems. Most responsible are Skinner's contentions that punishment is ineffective and fuat, because humans have no free will, they cannot be held responsible for fueir behavior.
Staddon believes that d1ese Skinnerian beliefs have resulted in faulty parenting
and flawed legal practices, which in run1 have led to increased crime rates, illegitimacy, and illiteracy. Although we would not go so far as to blame Skinnerian behaviorism for complex social and economic problems, there are aspects of his
position that can be legitimately c1;ticized.
Although the ideographic method developed by Skinner allowed tl1e examination of an individual 's operant beha,~or in detail, it was very difficult to compare
results from his procedure wifu results obtained in laboratories using d1e nomothetic mefuod. A second c1;ticism follows from Skinner's refusal to develop formal
ilieory. As we noted in Chapter 1, a p1;mary function of a fueory is the explanation
of existing data and phenomena. It is important to note, in the context of Skinner's position, that there is a great difference between describing a phenomenon
and attempting to explain that phenomenon. In fue first case, careful description
is usually accurate, cannot be disputed, and tends to explain how and when behavior occurs. A fueory, on the other hand , usually endeavors to explain why a behavior appears as well as the bows and whens. Theories, unlike descriptions, are often
disputed, and such dispute may lead to scientific progress. Skinner's system did
lead to progress, but it was a progress characterized by accumulation of behavioral
phenomena rather ilian a deeper understanding of learning and motivation.

Discussion Questions
l. Outline the procedure you would use while following Skinner's ilieory to increase tl1e probability tl1at a child would become a creative adult.
2. Would you use the same reinforcers to manipulate ilie beha,~or of bod1 children and adults? If not, what would make the difference?
3. Are there some forms of adult human behavior for which you feel Skinner's
theory is not applicable? Explain.
4. vVhat would characterize ilie classroom procedures suggested by Skinner's
theory of learning? List a few differences between these procedures and
d1ose now being followed in our schools.

118

CHAPTERS

5. Assuming the conclusions Skinner reached concerning the effectiveness of

6.
7.
8.
9.

10.
11.

12.

13.
14.
15.
16.
17.
18.
19.

punishment are valid, what major change would they suggest in child rearing? Ciiminal behavior? Education?
What is the partial reinforcement effect? Briefly describe the basic reinforcement schedules that Skinner studied.
Propose an explanation for the partial reinforcement effect.
Desciibe concurrent and concurrent chain schedules of reinforcement and
give an example of each.
What is Herrnstein's matching law? For what aspects of reinforcement has
the law been found tci hold true? What are the implications of the law for
dealing with human behavior problems?
What is contingency contracting? Give an example of how it could be used.
From Skinner's point of view, what are the advantages of programmed learning and teaching machines over the traditional lecture technique of t eaching?
According to Skinner, why have we not developed a more adequate technology of behavior in this country? What would need to be done before we
would be willing to utilize such a technology in solving our problems?
Give an example of how the Premack principle can be used to modifY the b ehavior of a primary school child.
Discuss chaining from Skinner's point of view.
Explain language development from Skinner's point of view. Explain Chomsky's opposition to Skinner's explanation oflanguage development.
Distinguish between positive reinforcement, negative reinforcement, and
punishment
Explain the difference between Premack's and Timberlake's views of reinforcers.
Describe the phenomenon of instinctual drift
Describe autoshaping and attempt to account for it

Chapter Highlights
autoclitic behavior
autoshaping
chaining
concurrent chain reinforcement
schedule
concurrent reinforcement schedule
contingency contracting
contingent reinforcement
continuous reinforcement schedule
(CRF)

cumulative recording
differential reinforcement
discriminative operant
discriminative stimulus (S 0 )
disequilibrium hypothesis
echoic behavior
extinction of an operant response
f'txed interval reinforcement schedule
(F1)

fixed ratio reinforcement schedule (FR)

BURRHUS FREDERIC SKINNER

functional analysis
functional autonomy
generalized reinforcers
Herrnstein's equation
instinctual drift
magazine training
mand
matching law
mentalistic events
misbehavior of org3nisms
noncontingent reinforcement
operant behavior
operant conditioning
operant level
partial reinforcement effect
(PRE)
.
Premack principle
primary negative reinforcer

119

primary positive reinforcer

programmed learning
punishment
radical behaviorism
respondent behavior
respondent conditioning
shaping
Skinner box
spontaneous recovery of an operant
response
successive approximation
superstitious behavior
tact
teaching machine
variable interval reinforcement
schedule (VI)

';

variable ratio reinforcement schedule

(VR)