El Gamal1996
El Gamal1996
Abstract: We propose and implement a Bayesian optimal design procedure. Our procedure takes as
its primitives a class of parametric models of strategic behavior, a class of games (experimental
designs), and priors on the behavioral parameters. We select the experimental design that maximizes
the information from the experiment. We sequentially sample with the given design and models until
only one of the models has viable posterior odds. A model which has low posterior odds in a small
collection of models will have an even lower posterior odds when compared to a larger class, and
hence we can dismiss it. The procedure can be used sequentially by introducing new models and
comparing them to the models that survived earlier rounds of experiments. The emphasis is not on
running as many experiments as pgssible, but rather on choosing experimental designs to distinguish
between models in the shortest possible time period. We illustrate this procedure with a simple
experimental game with one-sided incomplete information.
1 Prologue
i We acknowledge the financial support from NSF grant #SES-9223701 to the Califoruia Institute of
Technology. We also acknowledge the research assistance of Eugene GraDer who wrote the
software for the experiments. This paper was presented at the Stony Brook workshop of Experi-
mental Game Theory, summer 1993.
experiment is worth doing and to choose among the experiments. Except in very
special situations, such as when to stop sampling, I don't think a lot of work has
been done in this area.
DeGroot: I think the reason there hasn't been very much done is because the
problems are so hard. It's really hard to do explicitly the calculations that are
required to find the optimal experiment. Do you think that perhaps the comput-
ing power that is now available would be helpful in this kind of problem?
2 Introduction
The main primitives of our procedure will be a class of experiments and a class of
models of subject behavior in those experiments. The class of experiments is
498 M. A. El-Gamaland T. R. Palfrey
I(1;0)= ~. l~(x;O)log((l ~ O ) I.
x~x \ ~=2p,l,(x;O) /
The design which maximizes our expected separation between model I and
the other n - 1 models, if model 1 were indeed the correct model, is
0* = max0~ eI(1; 0). In standard hypothesis testing procedures, we would pick one
of the models to be our null hypothesis, and perform a test of a particular size
under that null. We adopt an analogous convention in this paper by treating one
model as the primary candidate and then choosing the design that maximizes the
informativeness of the experiment if it were indeed the correct model. In all other
aspects, however, the models can start on the same footing by letting Pi = I/n; Vi.3
2 In our Bayesiandesign framework,each of the models will also have nuisance parameters. As
shown below, the likelihood functions will then be computed by assessingour priors on those
nuisance parameters,and integratingthem out.
a One could alternativelymake the problem completelysymmetricby maximizing~= t pil(i;O)
insteadofany one of the givenl(i, O)'s.We chosein this paper to followthe procedureofhighlighting
one modal as the benchmark to beat since that is closer in spirit to the more familiarhypothesis
testing framework.
EconomicalExpcrimen~ Bayesian EfficientExperimentalDesign 499
One point must be made clear. After selecting a design, obtaining the data, and
"selecting a model" from the chosen class, we are not actually "accepting"this
model as the correct one in the classical sense of the term. As is the case with
classical statistics, one cannot "accept" a model. We can reject models if the
probability that the data we observe comes from those models is too small
(tradition after Fisher (1950) dictates treating probabilities less than 1% or 5 % as
sufficiently close to zero; hence using those magic numbers as sizes for hypothesis
tests). Since we employ a Bayesian posterior that provides posterior odds on the
selected (finite) class of models, those posteriors have to add up to unity. Hence, if
there are say two models, and one of them has a very low posterior probability,
then the other must have a very high posterior probability. That is not to say that
we accept the model with the high posterior as the true model. We know that the
posterior probability of the less likely model within the given finite class is an
upper bound on its posterior probability if we considered all potential models.
Hence, ifa model is discarded by our procedure due to having too tow a posterior
probability, that result can be accepted regardless of the class of models we
started with. However, the fact that one of our models performs well is not an
indication that it will continue to do so when compared to other models. The
process for selecting "the best" model can never terminate since there is an infinite
number of eligible classes of.models, and we can only analyze a finite number of
such classes at a time.
An alternative criterion for optimizing the design can be intuited from the
above mentioned classical hypothesis testing point of view. In the classical
hypothesis testing framework as culminated in Lehmann (1959), the emphasis is
on choosing a point on the indifference curves of"size" (probability of rejecting
a model when it is true) and "l-power" (probability of accepting a model when it
is false). This would suggest choosing a design to minimize our expected loss (as
a function of accepting a false model or rejecting a true one) subject to some
budget constraint. We do not take this approach due to the well known
shortcomings of the classical hypothesi s testing approach, most notab!e of which
is that "classical statistics tends to divert attention from (the likelihood ratio) to
the two conditional probabilities of making errors" (c.f. Edwards et al. (1963)),
resulting in the well known Jeffrey's/Lindley's paradox (c.f. Lindley (1957)) that
given any significance level, and any posterior odds in favor of the null hypoth-
esis, there exists a datum which produces that posterior odds, and or which the
null is rejected at the given significant level. By defining our "utility" function in
terms of likelihood ratios, and therefore defining expected utility (our objective
function, the Kullback-Liebler information number) as the expectation of that
utility function under the null hypothesis, we avoid those problems. For any
given dollar value of the experiment, we simply maximize its informativeness
under the null hypothesis.
In this paper, we perform the first step of this infinite sequential procedure of
selecting a class of models, deciding on the design, sampling until one model gets
very high posterior odds within the class, then adding more models, and repeat-
ing the whole process. This paper takes that first step by choosing a class of
500 M. A. EI-Gamaland T. R. Palfrey
We use for our experimental study a game which, for lack of a better name, we call
"stop-go". There are two possible states of nature "a", and "b'. Nature chooses
state "a" with probability n, and state "b" with probability (1 - n). There are two
players "1" and"2". Player I is informed of the state of nature, but player 2 is not.
In both states of nature, player I has an outside option stop which guarantees
both players a payoff of 1. If player 1 does not choose stop and decides to choose
Nature
(13) 2 (t,l)
go, then player 2 gets to choose right or left, and the payoffs are shown in the game
tree in Figure 1, where the payoff A is assumed to be strictly greater than 2.
This game is as simple a game of incomplete informaton as we could construct
while still maintaining the main features that we need. The first and most
important of those features is that the Bayesian updating needed to play the
Bayes-Nash equilibrium of the game is quite simple (given 1 chooses go, what is
the probability that the state of nature is "a"). Another useful characteristic is that
except for a very small set of designs (which we avoid), the game has a unique
Bayes-Nash equilibrium, which is easy to find as a function of n and A. This in
turn makes the calculation of the likelihood of various data sets under the model
(after introducing the errors in actions and errors in beliefs) rather straightfor-
ward, which is needed for calculating the above described information numbers
for various design parameters 0 = (~, A). The game also has the advantage of
having a very simple structure which is very easy to explain to subjects, and it
mimics a variety of confrontation games with an outside option that makes it easy
for subjects to understand.
setting. At best, there will be a distribution of priors that different subjects will
actually hold when they are given the same set of experimental instructions. We
model this by making the perceived prior np,r uniformly distributed over (,x - 6,
n + 6), where n is the induced prior. In other words, we assume that when we
announce to the subjects that state of nature"a" will be drawn with probability z,
they draw a ~pe~,-, U ( ~ - 6,re + 6). We can again make the distribution of
perceived priors more sophisticated, but we choose this parsimonious specifica-
tion which still allows us to estimate a perception error nuisance parameter ~5.The
randomly drawn npcr is assumed to be common knowledge to the subjects, but
unknown to us.
We specify three models that could potentially describe the behavior of the
subjects in this game. In this section we shall state the predicted behavior under
each of the models as a function of np~r A, and ~. When finally calculating the
likelihoods below, we shall integrate out the nuisance parameters ~5(implicitly
defining the distribution of nvc,), ~o, and ct with respect to our priors.
Model I: Each individual i plays the Bayes Nash Equilibrium of the game,
defined by (ripe,, A, ~).
Model 11: Player 2 does not update rrpe, following go, and this is common
knowledge.
Model IlI: Individuals use fictitious play to construct beliefs about opponents'
play.
The most difficult model to solve is the Bayes-Nash equilibrium, where e, the
tremble rate, is common knowledge, and all players take into consideration
that they as well as their opponents will tremble with probability ~, and that
is common knowledge. The equilibrium strategies will be calculated by enu-
merating five different cases. There are two main features of this (unique)
equilibrium:
For model I, po is the probability that player 1 chooses #o if the state of nature is
"a", Pv is the probability that player 1 chooses go if the state of nature is "b", and
EconomicalExperiments:Bayesian EfficientExperimentalDesign 503
q is the probability that player 2 will choose left (if given the chance to move),
provided that they do not tremble. In other words, the actual probability of
a player 1 choosing 0o in state "a" is ( 1 - e/2)p. + e/2, the actual probability
of choosing go in state "b" is ( 1 - e/2)pb + e/2, and the actual probability of
player 2 choosing left is (1 - e/2)q + el2.
A
~per = 2+ A
p~ = A ( 1 - rip.,) _ [ ( A + 2 ) n p , , - - A]~/2
2Xpcr (1 -- ~)2Xper
p[= 1
z 1 I-A- 1 -1
q
2~per(1 -- 71~pcr) 2
Case B: 7~per ~ ~per~ e ~" _ and e < ~ :
7~pr r + 7~pe r - - 27[perT~p9 r
p.~= 1 -p~'= 1 - d =0
- 2Xper(1- - ~per)
Case C: xp~r <_~p9 e <. "
- - ~per dr" 7~pe r - - 2~perTl:pr r "
p,~= 1
2Xp~r [(A + 2)Tt'p9 r - - A]e/2
PbZ-- A(1 - Xp.,) + ' (1 - e)A(1 - 7[pr
qZ= 1
2
p Io_- 1 - p ~ = 1 _ ql _ 1
I
po=p~=q1= 1
504 M. A. EI-Gamaland T. R. Palfrey
~t I 1 ifeA/2>l
Pa = 0.5 ifsA/2=l
0 if ~A/2 < 1
I if 2(1 - s / 2 ) > 1
pit= 0.5 if 2 ( 1 - s / 2 ) = 1
0 if 2(1 - s/2) < 1
qll = 1
Player I starts with a belief that probability of player 2 choosing left is 1/2, and
player 2 starts with the belief that the probability of player 1 choosing go is 1/2 in
both states of nature. Both players play optimal responses to their beliefs, and
update them using fictitious play. In other words, initializing empl =0.5,
empa I = 0.5, empbl = 0.5, the agents are assumed to use fictitious play to obtain
player l's belief erupt about the proportion of the time player 2's choose left, and
player 2's beliefs (empat, empbt) about the proportion of time player l's choose go
when game a or b is drawn (respectively), such that in round t,
emp + #{leftlgo}
emp, = #{go} + 1
empb + #{golgameb}
empb, = #{gameb} + 1
EconomicalExperiments:Bayesian EfficientExperimentalDesign 505
where {leftlgo} is the event that an opponent (player 2) in some previous round
chose left (they have to be given the chance to move first), {go lgamex} is the event
that an opponent (player 1) in some previous round chose go when the state of
nature was xr b}, {go} is the event that player 1 chooses go, {gamex} is the
event that the state of nature in a previous round was x, and # {- } is the number of
times the event has occurred so far.
Using those definitions, we can now define player 2's updated empirical belief
at round t (via fictitious play) that game a is being played conditional on go, which
we call empn,:
empn~ = erapatT~per
empa~nper + empbtnpe ,"
With these definition, we can define the predicted strategy in round t when
a player does not tremble as follows (again the actual likelihoods of moves under
the model will take into consideration the probability of trembles, replacing O's by
e/2's and l's by 1 - e2's.
m _ I~ if (1 -- emp,)A > 1
P,.t -- .5 if (1 -- emPt)A = 1
0 l
if (1 -- empt)A < 1
i if 2 empt > 1
Pb., -
Ill ~
.5 i f 2 emp, 1
if 2 erupt < 1
{iif2emp~,>(l-emp~,) A
q~H = .5 if 2 empTtt = (1 - emprq)A
if 2 emprq < ( 1 - e m p r Q A
Now that we have defined the strategy profiles (po, Pb, q) forplayers who do not
tremble under each of our three models, we can define the likelihoods of
a particular move by defining (for M e {I, II, III}):
obsp , = (1 - +
Where like(actionilM; to; ~; np~,) is the likelihood under model M that agent
i (with his observed history) in round t, having a tremble rate toe-% and
a perceived prior river, chooses action[. If agent i is a player 1, then this
contribution to the overall likelihood is obsp~t or obsp~ if the action was 0o and
the draw of nature was a or b, respectively, and it is obsq,M if the agent is a player
2 and chose left (all computed with the appropriate e, = eoe-~, and riper)"Now, for
a given number of rounds and number of players, we can calculate the Kullback-
Liebter information numbers defined in section 2 by calculating the likelihoods of
the models under all possible data sets (i.e. all possible collections of actionsl;
i = 1..... n; t = I ..... T), and choose the optimal design.
5 The ExperimentalDesign
In this section, we shall discuss a number of practical design issues that we have so
far ignored. It must be apparent to the reader that the problem of choosing the
optimal design is not well posed. There are too many parameters over which to
optimize (the class of all possible games, the number of rounds, the number of
individuals, etc.). In the general discussion of section 2, we assumed that we
limited our attention to a class of experimental designs parametrized by a finite
dimensional vector 0aO. The optimizaton of the design was then reduced to
a problem of choosing the value of 0 that maximizes a Kullback-Liebler
information number. By limiting attention to the class of games defined above by
the game of stop-go, and parameterized by (xe(0,1), A > 2), we made the informa-
tion number of the experiment a function of (n,A), i.e. we generate a two
dimensional information surface whose peak we seek. Now, we have to take into
consideration relevant issues of perception of payoffdifferentials (e.g. an A = 2.01
cannot be assumed clearly distinguishable from 2.0, and a n = 0.401 cannot be
assumed distinguishable from 0.4). We also need to take into consideration issues
of not having too few a number of people (to avoid reputation effects and the
possibility of guessing who one is paired with) or too many (collecting too much
data all at once without getting a chance to analyze it and see if we have all the
information we need). We cannot have too few rounds (not allowing fictitious
play types to update), or too many (inducing subject boredom and/or making the
payoff per decision insignificant). Moreover, there are technical constraints to the
calculation of the information, since each information number calculation re-
quires looping over all possible data sets (whose number grows exponentially in
the number of rounds, and geometrically in the number of subjects).
We decided on running experimental sessions with 10 subjects (5 player l's and
5 player 2's), which lasts for three rounds, yielding "batches" of data, 15 games at
a time.* The experiment was conducted at the social science experimental
laboratory at Caltech, and as subjects entered the lab, they were randomly
assigned a role as player 1, or player 2, and their role as player 1 or 2 remained the
same throughout the experiment. Then, the instructions (reproduced in Appen-
dix B) were read aloud to the subjects. Subjects were told that they will each play
three rounds of the game of stop-go described above, and that they were never
going to be paired with the same opponent twice. This specification (the number
of subjects n = 10, and the number of rounds T = 3) was at the limit of our
computational abilities, and as will be seen below, they were enough to very
strongly distinguish among our three models. The generation of this Bayesian
information surface required (as discussed in section 2) the integration out of the
nuisance parameters (8o, ~, 6) parameterizing the errors in actions and errors in
beliefs in all three models. We used independent priors on the three parameters
with the prior on eo and a being U[0,1], and the prior on t5 being U[0,0.2].
For that design with n = 10, T = 3 and the priors specified above, we
computed the information surface under the assumption that model 1 is the true
model. This information surface is shown in Figure 2, and its contours are shown
in Figure 3.
A blind maximization of the information surface would drive us all the way to
A = 2, which is not allowed. Hence, a true maximum with A > 2 does not exist.
Moreover, we wish to choose a point that is dearly distinct from 2.0 to minimize
the possibility that subjects do not find A and 2.0 distinguishable. If we look for an
interior point very close to the optimum, we get (A = 3.33, n = 0.6) (inside the
"bulb" in Figure 3). Since probability 0.6 is relatively hard to explain to subjects,
and since the" surface drops rather slowly in the direction of lower n's, we selected
the nearby design
r~=0.5 A=3.33
6 Results
1 1 G L 0 6 a
2 2 G L 6 0 b
3 3 G R 0 10 b
4 4 G R 10 0 a
5 5 G L 6 0 b
1 2 G R 0 10 b
2 3 G R 0 10 b
3 4 G R 0 10 b
4 5 G L 6 0 b
5 1 G L 0 6 a
1 3 G L 0 6 a
2 4 G L 6 0 b
3 5 G R 10 0 a
4 1 G R 0 10 b
5 2 G L 6 0 b
7 Concluding R e m a r k s
It is clear from our experimental results how choosing a design to maximize the
separation between our models (in terms of a Kullback-Liebler information
number) can yield very strong results from very small data sets. Given any class of
models, the stage of distinguishing the most viable among them is only one of
many stages towards improving our understanding of the behavior of subjects in
similar experimental settings. Accordingly, it makes sense to minimize the time
spent on each such stage. In subsequent stages, we would introduce a new class of
models and proceed from there (e.g. E1-Gamal and Grether (1995) provide one
method for choosing such a class of models from a potentially large class using
existing data). The next stage would then involve reoptimizing the design and
running new experiments to choose the best within the current class of models (for
the computational issues involved in such a design-redesign procedure, and an
application, see E1-Gamal, McKelvey and Palfrey (1993b)). This procedure can
then be repeated.
Appendix A
1.5
0.5
1.2 I
0.8
0.6
0.4
0.2
20
15
10
x2(O
2.5
2 /
/
1.5
0.5
ft.
0 0.2 0.4 0.6 0'.8 i
Fig. 8. Marginal posterior on x under model II
n2(~)
14
12
10
g
M a r g i n a l s o f the P o s t e r i o r U n d e r M o d e l III
r,3(c)
i / - -
0.8
0.6
I).4
0.2
rd(~
3.5
3
2.5
2
1.5
1
0.5
z3~
15
12.5
10
7.5
Appendix B
June 18, 1993 (STOP-GO)
Decision-Making Experiment
This session is part of a laboratory study in decision making. You will be paid IN
CASH at the end of the session. The amount of money you earn will depend upon
the decisions you make and on the decisions other people make. We ask that you
do not talk or otherwise attempt to communicate with the other subjects except
according to the specific rules below. If you have a question, feel free to raise your
hand. One of us will come over to where you are sitting and answer your question
in private.
514 M. A. EI-Gamaland T. R. Palfrey
Payoff Table I:
Left Right
Left Right
Notice that which payoff table was selected matters only if the red player has
chosen GO. If the red player chooses STOP, the match ends and payoffs are $3.00
for each player in the pair. We also ask blue players to do one additional thing. If
the red player you are paired with chooses STOP, please enter a decision (either
LEFT or RIGHT) anyway. It makes absolutely no difference at all which decision
you enter after red has chosen STOP, since the match is automatically over
already. Let me emphasize this fact. This choice of yours after red has already
ended the match by choosing STOP will not affect your payoffs in any way
whatsoever. Since it has no effect on payoffs, the red player you are matched with
is not told which decision you entered in this case. At the end of a match, each
BLUE player is then told which payoff table their pair was using (recall that the
RED players already had been told this at the beginning of the match). Your
payoffs for that match are then displayed and highlighted on your screen. For
those pairs in which red selected "GO", Blue's choice of LEFT or R I G H T is then
highlighted on the screen of the red player in that pair to plainly see. After all of
this has happened you will be prompted to record on your record sheet the match
number, Red's move, Blue's move, which payoff table was used, and what your
payoff was for that match. Please do not record anything on your record sheet
until you are prompted by the computer to do so. When everyone in the room has
finished recording and is ready to begin the next match, we will then assign new
pairings for everone, and the computer will randomly select a payoff table for
each pair. Remember that table I is always selected with 50/50 chance and II is
selected with 50/50 chance, and which tables you have used in previous matches
have no bearing at all on which table the computer happens to select for you in
this match. Also remember that as soon as a payoff table has been selected for
your pair, the red player is told which one it is, but the blue player is not told until
the end of the match.
This procedure will continue for a total of 3 matches. After the last match, add
up your earnings for all 3 matches, and record this amount at the bottom of your
record sheet. As a double check, the computer will also calculate your total
earnings and display it on the screen. The experimenter will pay you this amount
in cash, one at a time, in the next room. Please bring all forms and personal effects
with you when it is your turn to be paid. You will exit by the door in the other
room. Are there any questions?
We will now explain how to enter your decisions into the computer, and how to
read the information on the screen. Please do not press any keys on your
keyboard unless prompted to enter specific information.
[Start Practice Rounds]
516 M. A. EI-Gamaland T. R. Palfrey
Please enter your name when the screen prompts you to do so. Then type in
your color assignment (r or b) when prompted. Remember that you will have this
color for the entire session of 3 matches. Please do not talk and do not press any
keys. Is everyone ready to begin a practice match?? You will not be paid for the
practice match
[Hand out "practice record sheet", but remind them that no payments will be
made for the practice match.]
[Practice match begins. The experimenter then explains what is on the screen.
Blue players walk over to see what the red players' screens look like, and vice
versa, to emphasize that red players are informed which payofftable it is, and blue
players are not so informed.]
l,Practice match 1 begins. All red players are asked to play G o and all Blue
players are asked to play Left. The experimenter then explains where the
opponent's move is listed on the screen, where the payoff is listed, and makes sure
everyone understands this. Everyone records this information.]
l,Experimenter then marches through what would have happened if the other
table had been chosen and the same move had been made.]
I'Practice match 2 starts. All red players are asked to 9lay Stop and blue players
are asked to play whatever they wish. Everyone records the information.]
[Experimenter then marches through what would have happened if the other
table had been chosen and the same move had been made. N o difference in this
case.]
l,In practice match 3, all red players are asked to play G o and blue players are
asked to play Right. Experimenter then marches through what would have
happened if the other table had been chosen and the same move had been made.]
l,The practice experiment is over and subjects are reminded to record every-
thing.]
Are there any questions? l,Answer questions'l
We will now hand out a short quiz to make sure that everyone understands the
details of what we will be doing in these 3 matches. Please raise your hand when
you have completed the quiz, so one of us can come by and check your answers.
['Restart experiment and begin real payoffs]
References