Professional Documents
Culture Documents
Zero Sum Games and The Minmax Theorem
Zero Sum Games and The Minmax Theorem
Lecture 6
Lecturer: Aaron Roth Scribe: Aaron Roth
Definition 1 A two player zero sum game is any two player game such that for every a ∈ A1 × A2 ,
u1 (a) = −u2 (a).(i.e. at every action profile, the utilities sum to zero)
So what should Max do, if he has to go first? Knowing what Min will do, he should play the the
mixed strategy that maximizes the minimum payoff that Min will be able to inflict on him. That is, he
should play:
arg max min(p · 3 − 2(1 − p), p · (−1) + (1 − p))
p
6-1
Similarly, if Min has to go first, she should play the strategy that will minimize the maximum payoff
that Max will be able to achieve when he best responds:
Going first is clearly a disadvantage – it reveals your strategy to your opponent and lets them respond
optimally. In this case, lets see how much of a disadvantage it is. What should Max do?
Since Min will always play the lower cost of her two actions, Max should play to equalize the cost of
these two actions – i.e. he should pick p such that:
3
3p − 2(1 − p) = −p + (1 − p) ⇔ 5p − 2 = 1 − 2p ⇔ p =
7
So Max should play (3/7, 4/7), and when Min best responds, he gets payoff 1 − 2p = 1/7 (and since
it is a zero sum game, Min suffers cost 1/7...)
And if Min goes first? Again, since Max will best respond, Min should play so as to equalize the
payoff of Max’s two options. She should pick q such that:
2
3q − (1 − q) = −2q + (1 − q) ⇔ 4q − 1 = −3q + 1 ⇔ q =
7
So Min should play (2/7, 5/7). In this case, when Max best responds, Min suffers cost 4q − 1 = 1/7
(and Max gets payoff 1/7)...
Hm – So in this case, it was in fact no disadvantage to have to go first! We have the max min value
of the game exactly equal to the min max value of the game. Whats more, this immediately implies that
(3/7, 4/7), (2/7, 5/7) is a Nash equilibrium of the game, since both players are getting payoff/cost 1/7,
and we have just derived that both are best responding to one another.
Lets investigate this further. We use the notation [n] = {1, 2, . . . , n}, and ∆[n] to denote the set of
probability distributions over [n]:
Definition 2 For an n × m matrix U (think about this as the payoff matrix in a two player zero sum
game if you like):
Xn
max min(U ) = max min pi · U (i, y)
p∈∆[n] y∈[m]
i=1
m
X
min max(U ) = min max qj · U (x, j)
q∈∆[m] x∈[n]
j=1
Note that if U is a zero sum game, then max min(U ) represents the payoff that Max can guarantee
if he goes first, and min max(U ) represents the payoff that he can guarantee if Min goes first. Note also
that we have defined things so that the first player is optimizing over mixed strategies, but the second
player is optimizing over pure strategies. We could have let both optimize over mixed strategies, but this
is without loss, since any player always has a pure strategy among her set of best responses.
It is apparent that playing second can only be an advantage: your strategy space is not limited by
the first player’s action, and you only have more information. In particular, this implies that for any
game U :
min max(U ) ≥ max min(U )
What is not apparent is that going first is not a disadvantage:
Theorem 3 (Von Neumann) In any zero sum game U :
6-2
Corollary 4 In any Nash equilibrium of a zero sum game, Max plays a maxmin strategy and Min plays
a minmax strategy. Note that these can be computed without needing to reason about what the other
player is doing.
Corollary 5 All Nash equilibria in Zero sum games have the same payoff – the max min value of the
game.
None of these things are true in general sum games (consider e.g. Battle of the Sexes). The minmax
theorem should also be surprising: it means that (if they are smart), there is no way of taking advantage
of knowing your opponent’s strategy and forcing them to commit to it.
The theorem is not obvious... Von Neumann proved it in 1928, and said “”As far as I can see,
there could be no theory of games without that theorem I thought there was nothing worth publishing
until the Minimax Theorem was proved”. Previously, Borell had proven it for the special case of 5 × 5
matrices, and thought it was false for larger matrices.
However. Now that we know of the polynomial weights algorithm, we can provide a very simple,
constructive proof.
Proof Suppose the theorem were false: That is, there is some game U for which min max(U ) >
max min(U ). Write v1 = min max(U ) and v2 = max min(U ) (And so v1 = v2 + for some constant
> 0). In other words, if Min has to go first, then Max can guarantee payoff at least v1 , but if Max is
forced to go first, then Min can force Max to have payoff only v2 .
Lets consider what happens when Min and Max repeatedly play against each other as follows, for T
rounds:
1. Min will play using the polynomial weights algorithm. i.e. at each round t, the weights wt of
the polynomial weights algorithm will form her mixed strategy, and she will sample an action at
random from this distribution, updating based on the losses she experiences at that round.
2. Max will play the best response to Min’s strategy. i.e. Max will play xt = arg maxx Ey∼wt [U (x, y)].
Consider what we know about each of their average payoffs when they play in this manner.
On the one hand, we know from the guarantee of the polynomial weights algorithm that:
T T
1X 1 X
E[U (xt , y t )] ≤ min U (xt , y ∗ ) + ∆(T )
T t=1 T y t=1
∗
T
X 1
= min U (xt , y ∗ ) + ∆(T )
∗
y
t=1
T
= min
∗
Ex∼x̄ [U (x, y ∗ )] + ∆(T )
y
where x̄ is the mixed strategy that puts weight 1/T on each action xt . ∆(T ) is the regret bound of the
polynomial weights algorithm – recall:
r
log n
∆(T ) = 2 .
T
But by definition, miny∗ Ex∼x̄ U (x, y ∗ ) ≤ max min(U ) = v2 and so we know:
T
1X
E[U (xt , y t )] ≤ v2 + ∆(T )
T t=1
6-3
On the other hand, on each day t we know Max is best responding to Min’s mixed strategy wt . Thus:
T T
1X 1X
E[U (xt , y t )] ≥ max Ey∼wt [U (x∗ , y)]
T t=1 T t=1 x∗
T
1X
≥ v1
T t=1
= v1
≤ /2
This proof is worth a bit of reflection. In particular, it has highlighted the particularly amazing feature
of the polynomial weights algorithm: it guarantees that no matter what happens, you do as well as if
you had gotten to observe your opponent’s strategy, and then best respond after the fact. In particular,
we have proven that every zero sum game U has a unique value v = max min(U ) = min max(U ) that is
the best value a player can hope for, if her opponent is playing optimally. Using the polynomial weights
algorithm guarantees that the player using gets payoff quickly approaching the value of the game. What’s
more, it does so without needing to know what the game is. Note that at no point is the game matrix
input to the PW algorithm! The only information it needs to know is what the realized payoffs are for
its actions, as it actually plays the game. As such, it is an attractive algorithm to use in an interaction
that you don’t know much about...
6-4