Chapter 8

Factorial ANOVA: Higher order ANOVAs

Page

1. Three-way ANOVA 8-2

2. Interpreting Effects 8-6
3. Structural model & SS partitioning 8-11
4. Contrasts 8-16
5. Planned & Post hoc tests 8-22
6. Analyzing Effects 8-23
7. Effect sizes 8-33
8. Higher-order ANOVA 8-34
9. Examples 8-35

8-1 2006 A. Karpinski

 Factorial ANOVA
Higher order ANOVAs

1. Three-way ANOVA
A three-way analysis of variance has three independent variables
o Factor A with a levels
o Factor B with b levels
o Factor C with c levels

All of the procedures we developed for a two-way ANOVA can be extended

to a three-way ANOVA. The interpretation gets more difficult and the math
is messier

For simplicity, we will examine the simplest three way ANOVA: 2*2*2
design
o Factor A with 2 levels
o Factor B with 2 levels
o Factor C with 2 levels

I will present the formulas in their general form, and will give an example of
a more complex design at the conclusion

8-2 2006 A. Karpinski

 An example of source expertise, source attractiveness, and the processing of
persuasive information

High Self-Monitors
Strong Argument Weak Argument
Expert Source Attractive Source Expert Source Attractive Source
4 4 4 2 3 4 5 3
3 6 4 3 5 3 5 5
4 3 2 4 3 5 7 6
5 4 3 3 2 3 5 7
2 5 5 2 6 2 6 7
5 4 3 4 4 3 4 6

Low Self-Monitors
Strong Argument Weak Argument
Expert Source Attractive Source Expert Source Attractive Source
3 1 5 2 5 5 6 4
5 5 4 4 6 6 4 3
5 3 3 4 4 4 4 4
4 4 2 3 7 6 2 2
3 3 4 4 6 7 4 3
2 4 6 3 7 5 5 4
njkl = 12

8-3 2006 A. Karpinski

 Graphing three-factor ANOVA designs

5.5

5
Strong Expert
Strong Attractive
4.5
Weak Expert
Weak Attractive
4

3.5

3
High Self-Monitor Low Self-Monitor

Three-way interaction

5.5
Persuasiveness

5
Strong Argument
4.5
Weak Argument
4

3.5

3
Expert Attractive Expert Attractive

High Self-Monitor Low Self-Monitor

8-4 2006 A. Karpinski

 ANOVA Table for three-way ANOVA
Sum of Squares df Mean Square F Sig.
Main Effects (Combined)
MONITOR
STRENGTH
SOURCE
2-Way Interactions (Combined)
MONITOR * STRENGTH
MONITOR * SOURCE
STRENGTH * SOURCE
3-Way Interactions MONITOR * STRENGTH * SOURCE
Model
Residual
Total

8-5 2006 A. Karpinski

2. Interpreting Effects

Interpreting main effects

o The main effect of self-monitor compares the levels of self-monitoring

(high vs. low) after averaging over the levels of argument strength and
source of argument
Self-Monitor Mean Std Dev N
High 4.10 1.40 48
Low 4.15 1.43 48
SM Effect -0.05

o The main effect of argument strength compares the levels of argument

strength (strong vs. weak) after averaging over the levels of self-
monitoring and source of argument
Strength Mean Std Dev N
Strong 3.63 1.12 48
Weak 4.63 1.50 48
Strength Effect 1.00

o The main effect of source of argument compares the levels of source of

argument (expert vs. attractive) after averaging over the levels of self-
monitoring and argument strength
Source Mean Std Dev N
Expert 4.21 1.43 48
Attractive 4.04 1.40 48
Source Effect 0.17

8-6 2006 A. Karpinski

 Interpreting two-way interactions
o The self-monitor by strength of argument interaction examines the
interaction of self-monitoring (high vs. low) and strength of argument
(strong vs. weak) after averaging over the levels of source of argument
Is the effect of self-monitoring the same at each level of strength of argument?
Is the effect of strength of argument the same at each level of self-monitoring?

Self-Monitoring
High Low
Strength of Strong 3.67 3.58
Argument Weak 4.54 4.71
Strength Effect -0.87 -1.13
n jk = 24

o The self-monitor by source of argument interaction examines the

interaction of self-monitoring (high vs. low) and source of argument
(expert vs. attractive) after averaging over the levels of strength of
argument
Is the effect of self-monitoring the same at each level of source of argument?
Is the effect of source of argument the same at each level of self-monitoring?

Self-Monitoring
High Low
Source of Expert 3.83 4.58
Argument Attractive 4.38 3.71
Source Effect -0.55 0.87
n jl = 24

o The strength of argument by source of argument interaction examines the

interaction of strength of argument (strong vs. weak) and source of
argument (expert vs. attractive) after averaging over the levels of self-
monitoring
Is the effect of strength of argument the same at each level of source of argument?
Is the effect of source of argument the same at each level of strength of argument?

Strength of Argument
Strong Weak
Source of Expert 3.79 4.63
Argument Attractive 3.46 4.63
Strength Effect 0.33 0.00
nkl = 24

8-7 2006 A. Karpinski

 Interpreting three-way interactions
o So far, the logic and interpretation of main effects and interactions is
basically the same as the two-way design
o Now, lets extend this logic to a three-way interaction

o The self-monitor by strength of argument by source of argument

interaction examines the interaction of self-monitoring (high vs. low) and
strength of argument (strong vs. weak) and source of argument (expert
vs. attractive)

The three-way interaction addresses the following questions:

Is the strength of argument by source of argument interaction the
same at each level of self-monitoring?

Is the self-monitor by strength of argument interaction the same at

each level of source of argument?

Is the self-monitor by source of argument interaction the same at each

level of strength of argument?

Lets examine each approach to the three-way interaction:

Is the strength of argument by source of argument interaction the
same at each level of self-monitoring?

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 4.08 3.58 3.50 5.67
Attractive 3.25 5.50 3.67 3.75

Source Effect 0.83 -1.92 -0.17 1.92

Difference in Source 2.75 -2.09
Effect

8-8 2006 A. Karpinski

 Is the self-monitor by strength of argument interaction the same at
each level of source of argument?

n jkl = 12 Expert Source Attractive Source

Self-monitoring Self-monitoring
High Low High Low
Strength Strong 4.08 3.50 3.25 3.67
Weak 3.58 5.67 5.50 3.75

Strength Effect 0.50 -2.17 -2.25 -0.08

Difference in Strength 2.67 -2.17
Effect

Is the self-monitor by source of argument interaction the same at each

level of strength of argument?
n jkl = 12 Strong Argument Weak Argument
Source of Argument Source of Argument
Expert Attractive Expert Attractive
Self- High 4.08 3.25 3.58 5.50
Monitor Low 3.50 3.67 5.67 3.75

Monitoring Effect 0.58 -0.42 -2.09 1.75

Difference in 1 -3.84
Monitoring Effect

o Each of the different ways of examining the three-way interaction will

lead to the exact same analysis and conclusion. The combination you
choose to present should be based on your theory/hypotheses

8-9 2006 A. Karpinski

 Table summarizing the meaning of effects in an A*B*C Design
(Maxwell & Delaney, 1990, p 318)

Meaning
Main Effects
A Comparison of marginal means of Factor A, averaging over
levels of B and C
B Comparison of marginal means of Factor B, averaging over
levels of A and C
C Comparison of marginal means of Factor C, averaging over
levels of A and B
Two-way
Interactions
A*B Examines whether the A effect is the same at every level of
B, averaging over levels of C
Equivalently, examines whether the B effect is the same at
every level of A, averaging over levels of C
A*C Examines whether the A effect is the same at every level of
C, averaging over levels of B
Equivalently, examines whether the C effect is the same at
every level of A, averaging over levels of B
B*C Examines whether the B effect is the same at every level of
C, averaging over levels of A
Equivalently, examines whether the C effect is the same at
every level of B, averaging over levels of A
Three-way
Interaction
A*B*C Examines whether the two-way A*B interaction is the same
at every level of C
Equivalently, examines whether the two-way A*C
interaction is the same at every level of B
Equivalently, examines whether the two-way B*C
interaction is the same at every level of A

8-10 2006 A. Karpinski

3. Structural model & SS partitioning

Structural Model for a three-way ANOVA

Yijk = MODEL + ERROR

Yijkl = + j + k + l + ( ) jk + ( ) jl + ( )kl + ( ) jkl + ijkl

Mean Model Components:

The overall mean of the scores

Main Effect Model Components:

j The effect of being in level j of Factor A
k The effect of being in level k of Factor B
l The effect of being in level l of Factor C

Two-way Interaction Model Components:

( ) jk The effect of being in level j of Factor A and level k of Factor B
( ) jl The effect of being in level j of Factor A and level l of Factor C
( )kl The effect of being in level k of Factor B and level l of Factor C

Three-way Interaction Model Components:

( ) jkl The effect of being in level j of Factor A, level k of Factor B,
and level l of Factor C

Error Components:
ijk The unexplained part of the score

8-11 2006 A. Karpinski

 j : The effect of being in level j of Factor A k : The effect of being in level k of Factor B

j = . j .. .... k = ..k . ....

a b

j = 0
j =1

k =1
k =0

l : The effect of being in level l of Factor C

l = ...l ....
c

l =1
l =0

( ) jk The effect of being in level j of ( ) jl The effect of being in level j of

Factor A and level k of Factor B Factor A and level l of Factor C

() jk = . jk . (....+ j + k ) ( ) jl = . j.l (....+ j + l )

a a

( ) jk
j =1
= 0 for each level of j ( )
j =1
jl = 0 for each level of j
b c

( ) jk
k =1
= 0 for each level of k ( )
l =1
jl = 0 for each level of l

( )kl The effect of being in level k of

Factor B and level l of Factor C

()kl = ..k l (....+ k + l )

b

( )
k =1
kl = 0 for each level of k
c

( )
l =1
kl = 0 for each level of l

( ) jkl The effect of being in level j of Factor A, level k of Factor B, and level l of Factor C

( ) jkl = . jkl (....+ j + k + l + jk + jl + kl )

a b

( )
j =1
jkl = 0 for each level of j ( )
k =1
jkl = 0 for each level of k
c

( )
l =1
jkl = 0 for each level of l

8-12 2006 A. Karpinski

 ijkl The unexplained part of the score
ijkl = Yijkl MODEL
(
= Yijkl + j + k + l + ( ) jk + ( ) jl + ( )kl + ( ) jkl )
o You should be able to compute and interpret each component of a three-
way ANOVA model. In addition, you should be able to decompose each
score into its structural model components

Variance partitioning for a three-way ANOVA

SS Total
SS Total Total)
(SS Corrected

SS Between SS Within
(SS Model) (SS Error)

SS Main SS 2-Way SS 3-Way

Effects Interactions Interaction

SS SS SS SS SS SS SS
A B C A*B A*C B*C A*B*C

8-13 2006 A. Karpinski

 o This SS partition only holds for balanced designs

o We showed the derivation of these SS formulas and how to compute

them for the one-way and the two-way ANOVA case. The three-way
formulas are extensions of these simpler formulas. You may find the
formulas in any advanced ANOVA book (For example, see Kirk, 1995, p
441)

o The math works out nicely (as we would expect) so that if we take the
ratio of the MS for a component of the model over the MS error, we
obtain a valid test of the model component

o ANOVA table for three-way ANOVA

ANOVA
Source of Variation SS df MS F P-value
Main effects
Factor A SSA (a-1) SSA/dfa MSA/MSW
Factor B SSB (b-1) SSB/dfb MSB/MSW
Factor C SSC (c-1) SSC/dfc MSC/MSW
Two-way Interactions
A * B interaction SSAB (a-1)(b-1) SSAB/dfab MSAB/MSW
A * C interaction SSAC (a-1)(c-1) SSAC/dfac MSAC/MSW
B * C interaction SSAB (b-1)(c-1) SSBC/dfbc MSBC/MSW
Three-way Interactions
A * B * C interaction SSABC (a-1)(b-1)(c-1) SSABC/dfabc MSABC/MSW

Model SSBet abc-1 SSB/dfbet

Within SSW N-abc SSW/dfw
Total SST N-1

8-14 2006 A. Karpinski

 o Using SPSS

UNIANOVA dv BY monitor strength source

/PRINT = DESCRIPTIVE.

Tests of Between-Subjects Effects

Dependent Variable: DV
Type III Sum
Source of Squares df Mean Square F Sig.
Corrected Model 72.833a 7 10.405 7.916 .000
Intercept 1633.500 1 1633.500 1242.778 .000
MONITOR 4.167E-02 1 4.167E-02 .032 .859
STRENGTH 24.000 1 24.000 18.259 .000
SOURCE .667 1 .667 .507 .478
MONITOR * STRENGTH .375 1 .375 .285 .595
MONITOR * SOURCE 12.042 1 12.042 9.161 .003
STRENGTH * SOURCE .667 1 .667 .507 .478
MONITOR * STRENGTH
35.042 1 35.042 26.660 .000
* SOURCE
Error 115.667 88 1.314
Total 1822.000 96
Corrected Total 188.500 95
a. R Squared = .386 (Adjusted R Squared = .338)

o Summary of the results:

Main Effects:
Self-monitoring: F(1, 88) = 0.03, p = .86
Strength of Argument: F(1, 88) = 18.26, p < .01
Source of Argument: F(1, 88) = 0.51, p = .48

Two-way interactions:
Monitoring*Strength: F(1, 88) = 0.29, p = .60
Monitoring*Source: F(1, 88) = 3.32, p = .02
Strength*Source: F(1, 88) = 0.51, p = .48

Three-way interactions:
Monitoring*Strength*Source: F(1, 88) = 26.66, p < .01

8-15 2006 A. Karpinski

4. Contrasts

We can perform contrasts using the same method we developed for two-way
ANOVA

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 2 5 6
Attractive 3 4 7 8

if (monitor=1 and strength=1 and source=1) group = 1.

if (monitor=1 and strength=2 and source=1) group = 2.
if (monitor=1 and strength=1 and source=2) group = 3.
if (monitor=1 and strength=2 and source=2) group = 4.
if (monitor=2 and strength=1 and source=1) group = 5.
if (monitor=2 and strength=2 and source=1) group = 6.
if (monitor=2 and strength=1 and source=2) group = 7.
if (monitor=2 and strength=2 and source=2) group = 8.

8-16 2006 A. Karpinski

 o To test the main effect of self-monitoring:
Self-Monitoring
High Low
1 -1
n j = 48

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 1 -1 -1
Attractive 1 1 -1 -1

ONEWAY dv by group
/CONT = 1 1 1 1 -1 -1 -1 -1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Self-Monitoring -.1667 .93609 -.178 88 .859

t(88) = -.18, p = .86

o To test the main effect of strength of argument:

Strength of Argument
Strong Weak
1 -1
nk = 48

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1 1 -1
Attractive 1 -1 1 -1

ONEWAY dv by group
/CONT = 1 -1 1 -1 1 -1 1 -1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Strength -4.0000 .93609 -4.273 88 .000

t(88) = -4.28, p < .01

8-17 2006 A. Karpinski

 o To test the main effect of source of argument:
Source of Argument
Expert Attractive
1 -1
nl = 48

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 1 1 1
Attractive -1 -1 -1 -1

ONEWAY dv by group
/CONT = 1 1 -1 -1 1 1 -1 -1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Source .6667 .93609 .712 88 .478

t(88) = 0.72, p = .48

o To test the monitoring by strength interaction:

n jk = 24 Self-Monitoring
High Low
Strength of Strong 1 -1
Argument Weak -1 1

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1 -1 1
Attractive 1 -1 -1 1

ONEWAY dv by group
/CONT = 1 -1 1 -1 -1 1 -1 1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Monitoring *
.5000 .93609 .534 88 .595
Strength

t(88) = 0.53, p = .60

8-18 2006 A. Karpinski

 o To test the monitoring by source interaction:
n jl = 24 Self-Monitoring
High Low
Source of Expert 1 -1
Argument Attractive -1 1

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 1 -1 -1
Attractive -1 -1 1 1

ONEWAY dv by group
/CONT = 1 1 -1 -1 -1 -1 1 1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Monitoring
-2.8333 .93609 -3.027 88 .003
* Source

t(88) = -3.03, p < .01

o To test the strength by source interaction:

nkl = 24 Strength of Argument
Strong Weak
Source of Expert 1 -1
Argument Attractive -1 1

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1 1 -1
Attractive -1 1 -1 1

ONEWAY dv by group
/CONT = 1 -1 -1 1 1 -1 -1 1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Strength
.6667 .93609 .712 88 .478
*Source

t(88) = 0.72, p = .48

8-19 2006 A. Karpinski

 o To test the monitoring by strength by source interaction:

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1 -1 1
Attractive -1 1 1 -1

ONEWAY dv by group
/CONT = 1 -1 -1 1 -1 1 1 -1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV 3-way 4.8333 .93609 5.163 88 .000

t(88) = 5.16, p < .01

o We can compute all the main effect and interaction tests with contrasts
because for a 2*2*2 design, all the tests are single degree of freedom tests.
For more complex a*b*c designs, omnibus tests with more than 1 degree
of freedom can be performed using simultaneous tests of orthogonal
contrasts.

8-20 2006 A. Karpinski

 o To compute these contrasts by hand, the formulas are simple
generalizations of the two-way case:
c b a
= c jkl X . jkl = c111 X .111 + ... + c abc X .abc
l =1 k =1 j =1

c b a c 2jkl
Std error ( ) = MSW
l =1 k =1 j =1 n jkl

Where c 2jkl is the squared weight for each cell

n jkl is the sample size for each cell
MSW is MSW from the omnibus ANOVA

t~

t observed =
c jkl X . jkl
standard error( ) c 2jkl
MSW
n jkl

SSC
2 dfc SSC
SS = F(1,dfw) =
SSW
=
c 2jkl MSW
n dfw
jkl

8-21 2006 A. Karpinski

5. Planned & Post hoc tests

Everything from the two-way design generalizes to the three-way design

Planned tests can be corrected (if necessary) using Bonferroni

Method for conducting post-hoc adjustments is same as for two-way design

o Obtain observed t- or F-statistic by hand (or using SPSS, but discard
printed p-value)
o Look up critical value and compare to observed value

For Tukeys HSD using marginal means: q(1-,d,)

Where = Familywise error rate
d = Number of groups in the comparison
= DFw = N-abc

For Tukeys HSD using all cell means: q(1-,abc,)

Where = Familywise error rate
abc = Number of cells in the design
= DFw = N-abc

q (qcrit )2
Compare tobserved to crit or Fobserved to
2 2

For Scheff using marginal means:

FCrit = (d 1) F =.05;d 1, N abc

For Scheff using all cell means:

FCrit = (a 1)(b 1)(c 1) F =.05;( a 1)(b 1)( c 1), N abc

Compare Fobserved to Fcrit

8-22 2006 A. Karpinski

6. Analyzing Effects

Maxwell and Delaneys (1990) guidelines for analyzing effects in a three-

factor design are considerably more complicated than for the two-factor
design (recall p 7-59)!
The principle remains the same. You must start with the highest order
significant effect. You decompose these effects into simpler effects until
you have an understanding of where the significant differences lie.

Simple (interaction) effect

o If you have a significant three-way interaction, then you need to examine
the separate two-way interactions
The A*B interaction at each level of C or
The A*C interaction at each level of B or
The B*C interaction at each level of A

o APPROACH #1: In our example, we have a significant three-way

interaction, so lets examine the source of argument by strength of
argument interaction at each level of self-monitoring

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1
Attractive -1 1

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1
Attractive -1 1

8-23 2006 A. Karpinski

 To examine the source of argument by strength of argument
interaction for high self-monitors:

ONEWAY dv by group
/CONT = 1 -1 -1 1 0 0 0 0 .
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV 1 2.7500 .66191 4.155 88 .000

To examine the source of argument by strength of argument

interaction for low self-monitors:

ONEWAY dv by group
/CONT = 0 0 0 0 1 -1 -1 1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV 1 -2.0833 .66191 -3.147 88 .002

When the variances are homogeneous, these analysis can also be

obtained with the MANOVA command.
Note: You cannot obtain simple interaction effects with GLM

MANOVA dv BY monitor (1,2) strength(1,2) source (1,2)

/DESIGN strength by source WITHIN monitor(1),
strength by source WITHIN monitor(2),
monitor * strength, monitor * source, monitor, strength, source.

Full Factorial Design Simple Effects Design

Monitor monitor
Strength strength
Source source
monitor * source monitor * source
monitor * strength monitor * strength
strength * source
monitor * strength * source source * strength WITHIN monitor (1)
source * strength WITHIN monitor (2)

8-24 2006 A. Karpinski

 * * * * * * A n a l y s i s o f V a r i a n c e -- design 1 * * * * * *

Tests of Significance for DV using UNIQUE sums of squares

Source of Variation SS DF MS F Sig of F

WITHIN+RESIDUAL 115.67 88 1.31

STRENGTH BY SOURCE W 22.69 1 22.69 17.26 .000
ITHIN MONITOR(1)
STRENGTH BY SOURCE W 13.02 1 13.02 9.91 .002
ITHIN MONITOR(2)
MONITOR * STRENGTH .38 1 .38 .29 .595
MONITOR * SOURCE 12.04 1 12.04 9.16 .003
MONITOR .04 1 .04 .03 .859
STRENGTH 24.00 1 24.00 18.26 .000
SOURCE .67 1 .67 .51 .478

(Model) 72.83 7 10.40 7.92 .000

(Total) 188.50 95 1.98

Because each of these separate two-way analyses are significant, we

need to conduct additional follow-up tests

- For high self-monitors: We can examine the effect of source of

argument within each level of strength of argument
(The main effect of source within high self-monitors and strong
argument AND within high self-monitors and weak argument)

- Alternately, for high self-monitors: We can examine the effect of

strength of argument within each level of source of argument
(The main effect of strength within high self-monitors and
expert source AND within high self-monitors and attractive
source)

- These analyses should be repeated for low self-monitors

8-25 2006 A. Karpinski

 o APPROACH #2: Alternatively, we can examine the strength of
argument by self-monitoring interaction at each level of source of
argument

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1 -1 1
Attractive

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert
Attractive 1 -1 -1 1

Using contrasts:
ONEWAY dv by group
/CONT = 1 -1 0 0 -1 1 0 0
/CONT = 0 0 1 -1 0 0 -1 1.

Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Expert Source 2.6667 .66191 4.029 88 .000
Attractive Source -2.1667 .66191 -3.273 88 .002

8-26 2006 A. Karpinski

 Using MANOVA:
MANOVA dv BY monitor (1,2) strength(1,2) source (1,2)
/DESIGN strength by monitor WITHIN source(1),
strength by monitor WITHIN source(2),
source * strength, monitor * source, monitor, strength, source.

Full Factorial Design Simple Effects Design

monitor monitor
strength strength
source source
strength * source strength * source
monitor * source monitor * source
monitor * strength
monitor * strength * source monitor * strength WITHIN source (1)
monitor * strength WITHIN source (2)

* * * * A n a l y s i s o f V a r i a n c e -- design 1 * * * *

Tests of Significance for DV using UNIQUE sums of squares

Source of Variation SS DF MS F Sig of F

WITHIN+RESIDUAL 115.67 88 1.31

STRENGTH BY MONITOR 21.33 1 21.33 16.23 .000
WITHIN SOURCE(1)
STRENGTH BY MONITOR 14.08 1 14.08 10.71 .002
WITHIN SOURCE(2)
SOURCE * STRENGTH .67 1 .67 .51 .478
MONITOR * SOURCE 12.04 1 12.04 9.16 .003
MONITOR .04 1 .04 .03 .859
STRENGTH 24.00 1 24.00 18.26 .000
SOURCE .67 1 .67 .51 .478

(Model) 72.83 7 10.40 7.92 .000

(Total) 188.50 95 1.98

8-27 2006 A. Karpinski

 o APPROACH #3: Alternatively, we can examine the source of argument
by self-monitoring interaction at each level of strength of argument

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1
Attractive -1 1

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1 -1
Attractive -1 1

Using contrasts:
ONEWAY dv by group
/CONT = 1 0 -1 0 -1 0 1 0
/CONT = 0 1 0 -1 0 -1 0 1.
Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV Strong Argument 1.0000 .66191 1.511 88 .134
Weak Argument -3.8333 .66191 -5.791 88 .000

8-28 2006 A. Karpinski

 Using MANOVA:
MANOVA dv BY monitor (1,2) strength(1,2) source (1,2)
/DESIGN monitor by source WITHIN strength(1),
monitor by source WITHIN strength(2),
monitor * strength, strength * source, monitor, strength, source.

Full Factorial Design Simple Effects Design

monitor monitor
strength strength
source source
strength * source strength * source
monitor * strength monitor * strength
monitor * source
monitor * strength * source monitor * source WITHIN strength (1)
monitor * source WITHIN strength (2)

* * * * A n a l y s i s o f V a r i a n c e -- design 1 * * * *

Tests of Significance for DV using UNIQUE sums of squares

Source of Variation SS DF MS F Sig of F

WITHIN+RESIDUAL 115.67 88 1.31

MONITOR BY SOURCE WI 3.00 1 3.00 2.28 .134
THIN STRENGTH(1)
MONITOR BY SOURCE WI 44.08 1 44.08 33.54 .000
THIN STRENGTH(2)
MONITOR * STRENGTH .38 1 .38 .29 .595
STRENGTH * SOURCE .67 1 .67 .51 .478
MONITOR .04 1 .04 .03 .859
STRENGTH 24.00 1 24.00 18.26 .000
SOURCE .67 1 .67 .51 .478

(Model) 72.83 7 10.40 7.92 .000

(Total) 188.50 95 1.98

o We should not take all three approaches; only one is necessary. The
choice you make should be the one that makes the most sense for your
theory/hypotheses

8-29 2006 A. Karpinski

 o For approach 3, we found
No significant self-monitoring by source interaction for strong
messages, F(1,88) = 2.28, p = .13.
A significant self-monitoring by source interaction for weak
messages, F(1,88) = 33.54, p < .01. We need to conduct follow-up tests
to interpret this simple interaction effect. These tests are called
simple, simple, main effects.

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert 1
Attractive -1

n jkl = 12 High Self-Monitor Low Self-Monitor

Strength of Argument Strength of Argument
Strong Weak Strong Weak
Source Expert -1
Attractive 1

Using Contrasts:
ONEWAY dv by group
/CONT = 0 1 0 -1 0 0 0 0
/CONT = 0 0 0 0 0 -1 0 1.

Contrast Tests

Value of
Contrast Contrast Std. Error t df Sig. (2-tailed)
DV High Monitor, Weak Message -1.9167 .46804 -4.095 88 .000
Low Monitor, Weak Message -1.9167 .46804 -4.095 88 .000

8-30 2006 A. Karpinski

 Using MANOVA
MANOVA dv BY monitor (1,2) strength(1,2) source (1,2)
/DESIGN source WITHIN monitor (1) WITHIN strength(1),
source WITHIN monitor (2) WITHIN strength(1),
source WITHIN monitor (1) WITHIN strength(2),
source WITHIN monitor (2) WITHIN strength(2),
monitor * strength, monitor, strength.

* * * * A n a l y s i s o f V a r i a n c e -- design 1 * * * *

Tests of Significance for DV using UNIQUE sums of squares

Source of Variation SS DF MS F Sig of F

WITHIN+RESIDUAL 115.67 88 1.31

SOURCE WITHIN MONITO 4.17 1 4.17 3.17 .078
R(1) WITHIN STRENGTH
(1)
SOURCE WITHIN MONITO .17 1 .17 .13 .723
R(2) WITHIN STRENGTH
(1)
SOURCE WITHIN MONITO 22.04 1 22.04 16.77 .000
R(1) WITHIN STRENGTH
(2)
SOURCE WITHIN MONITO 22.04 1 22.04 16.77 .000
R(2) WITHIN STRENGTH
(2)
MONITOR * STRENGTH .38 1 .38 .29 .595
MONITOR .04 1 .04 .03 .859
STRENGTH 24.00 1 24.00 18.26 .000

(Model) 72.83 7 10.40 7.92 .000

(Total) 188.50 95 1.98

Using GLM
UNIANOVA dv BY monitor strength source
/EMMEANS = TABLES(monitor*strength*source) COMPARE(source)
/PRINT = DESCRIPTIVE .
Univariate Tests

Dependent Variable: dv
Sum of
monitor strength Squares df Mean Square F Sig.
High Strong Contrast 4.167 1 4.167 3.170 .078
Error 115.667 88 1.314
Weak Contrast 22.042 1 22.042 16.769 .000
Error 115.667 88 1.314
Low Strong Contrast .167 1 .167 .127 .723
Error 115.667 88 1.314
Weak Contrast 22.042 1 22.042 16.769 .000
Error 115.667 88 1.314
Each F tests the simple effects of source within each level combination of the other effects shown.
These tests are based on the linearly independent pairwise comparisons among the estimated
marginal means.

8-31 2006 A. Karpinski

 o If these tests are planned or post-hoc, they need to be adjusted
accordingly

An alternative to the simple effect approach is the contrast-based approach.

o The traditional approach conducts 7 uncorrected omnibus tests, so we are
allowed 7 uncorrected planned contrasts. If you have more than 7
planned contrasts, you must use the Bonferroni correction.
o Post-hoc tests can be conducted using Tukey HSD or Scheff to keep
EW = .05

8-32 2006 A. Karpinski

7. Effect sizes

Formulas for partial omega-squared and r (for contrasts only) are easily
adapted to a three-factor design:

SS (effect ) [df (effect )]MSWithin

(2EFFECT ) =
SS (effect ) + [N df (effect )]MSWithin

2
Fcontrast t contrast
r= =
Fcontrast + df within 2
t contrast + df within

For example, to compute the proportion of variance accounted for by the

three-way interaction in our persuasion example

SS ( ABC ) [df ( ABC )]MSWithin

(2A*B*C ) =
SS ( ABC ) + [N df ( ABC )]MSWithin

35.042 (1)1.314
(2A*B*C ) = = .21
35.042 + [96 1]1.314

F(1,88) = 26.66, p < .001, 2 = .21

8-33 2006 A. Karpinski

8. Higher-order ANOVA

The logic we developed for two- and three-factor ANOVA can be easily
extended to four-factor, five-factor and even higher order ANOVAs
By now you have seen how the formulas generalize so that you can compute
values for any order design

Interpretation of a three-factor ANOVA is tricky enough. Things get very

hairy for higher order ANOVAs.
o For example, a significant four-way interaction (A*B*C*D) indicates
that the three way A*B*C interaction is not the same at each level of D
or that the three way A*B*D interaction is not the same at each level of
C or . . .
o We saw that to graph a three-way 2*2*2 interaction, we had to graph two
separate two-way interactions
To graph a four-way 2*2*2*2 interaction, we would have to graph
four separate two-way interactions!
To graph a five-way 2*2*2*2*2 interaction, we would have to graph
eight separate two-way interactions!
o Remember when you design a study, you will need to be able to analyze,
understand, and present the results. It is rare that a person can clearly
present a four-way interaction in a manner that the audience can
understand. Beware of conducting designs that are too complex!

As the number of factors increases, the number of omnibus tests increases

rapidly. Because the convention is to use = .05 for each omnibus test, the
probability of making a type one error is high for a multi-factor ANOVA

Number of
Number of Main Two-way Three-way Four-way Five-way Total Number
Factors Effects Interactions Interactions Interactions Interactions of tests
2 2 1 3
3 3 3 1 7
4 4 6 4 1 15
5 5 10 10 5 1 31

As a result, do not be surprised if you are asked to replicate the results of

your multi-factor ANOVA.

8-34 2006 A. Karpinski

9. Example: A 3*3*2 design

Consider an experiment comparing three types of therapy for modifying

snake phobia

o Factor A Degree of Phobia: Mild, Moderate, Severe

o Factor B Type of Therapy: Desensitization, Implosion, Insight
o Factor C Gender: Male, Female

o DV = Post-test scores on the Behavioral Avoidance Test (higher scores

indicate less phobia)

Desensitization Implosion Insight

Mild Moderate Severe Mild Moderate Severe Mild Moderate Severe
Females 10 12 10 15 12 6 13 11 10
12 9 11 12 10 7 9 7 6
13 10 9 14 11 5 11 8 8
Males 16 11 12 17 14 10 16 10 11
14 13 11 18 13 9 12 12 10
17 15 13 16 12 11 14 14 9

Treatment of Snake Phobia

25
Behavior Aviodance Test Scores

20

15 Mild
Moderate
10 Severe
5

0
Insight

Insight
Implosion

Implosion
Desensitization

Desensitization

Female Male

8-35 2006 A. Karpinski

 First, lets approach the analysis the traditional way:

UNIANOVA dv BY treat phobia gender

/PRINT = DESCRIPTIVE.
Tests of Between-Subjects Effects

Dependent Variable: DV
Type III Sum
Source of Squares df Mean Square F Sig.
Corrected Model 368.167a 17 21.657 9.356 .000
Intercept 7141.500 1 7141.500 3085.128 .000
TREAT 22.333 2 11.167 4.824 .014
PHOBIA 183.000 2 91.500 39.528 .000
GENDER 115.574 1 115.574 49.928 .000
TREAT * PHOBIA 39.333 4 9.833 4.248 .006
TREAT * GENDER .259 2 .130 .056 .946
PHOBIA * GENDER 1.815 2 .907 .392 .679
TREAT * PHOBIA *
5.852 4 1.463 .632 .643
GENDER
Error 83.333 36 2.315
Total 7593.000 54
Corrected Total 451.500 53
a. R Squared = .815 (Adjusted R Squared = .728)

We have a treatment by phobia interaction, F(4, 36) = 4.25, p = .006

We have a main effect of gender, F(1, 36) = 49.93, p < .001

o We also have main effects for treatment and for phobia, but we should
refrain from interpreting them because of the higher order interaction

o We may interpret the main effect of gender because gender is not

involved in any higher order interactions

Lets start with the main effect of gender. This analysis reveals the effect of
gender averaging across type of treatment and severity of phobia.

Gender Mean Std Dev N

Female 10.04 2.52 27
Male 12.96 2.56 27
Gender Effect -2.92

o This analysis tells us that men show less post-test phobia than women,
averaging across type of treatment and severity of phobia.

Because this test has only 1 df, no follow-up tests are necessary

8-36 2006 A. Karpinski

 Now, lets turn to the treatment by phobia interaction. This analysis tells us
that the main effect for treatment differs by the degree of phobia, averaging
across gender.

Treatment of Phobia:
Treatment by Phobia Interaction

16

14
Desensitization
12 Implosion
Insight
10

8
Mild Moderate Severe
Degree of Phobia

Degree of Phobia
Mild Moderate Severe
Treatment Desens. 13.67 11.67 11.00
Implosion 15.33 12.00 8.00
Insight 12.50 10.33 9.00

o To understand this interaction, we can examine the simple effect of

degree of phobia within each type of treatment
Degree of Phobia
Mild Moderate Severe
Treatment Desens. 13.67 11.67 11.00
Implosion 15.33 12.00 8.00
Insight 12.50 10.33 9.00

According to Maxwell & Delaneys guidelines, we need to use the

Bonferroni adjustment:
.05
FW = = 0.0167
3

8-37 2006 A. Karpinski

 These simple effect tests will be two-degrees of freedom tests. We can
not test these hypotheses with a single contrast. If we have
homogeneous variances, we can use the MANOVA command.

MANOVA dv BY gender (1,2) treat(1,3) phobia (1,3)

/DESIGN phobia WITHIN treat (1), phobia WITHIN treat (2),
phobia WITHIN treat (3), treat * phobia *gender, gender * treat,
gender * phobia, gender, treat .

* * * * A n a l y s i s o f V a r i a n c e -- design 1 * * * *

Tests of Significance for DV using UNIQUE sums of squares

Source of Variation SS DF MS F Sig of F

WITHIN+RESIDUAL 83.33 36 2.31

PHOBIA WITHIN TREAT(1) 23.11 2 11.56 4.99 .012
PHOBIA WITHIN TREAT(2) 161.78 2 80.89 34.94 .000
PHOBIA WITHIN TREAT(3) 37.44 2 18.72 8.09 .001
TREAT * PHOBIA * GENDER 5.85 4 1.46 .63 .643
GENDER * TREAT .26 2 .13 .06 .946
GENDER * PHOBIA 1.81 2 .91 .39 .679
GENDER 115.57 1 115.57 49.93 .000
TREAT 22.33 2 11.17 4.82 .014

(Model) 368.17 17 21.66 9.36 .000

(Total) 451.50 53 8.52

Simple effect of degree of phobia for participants who received

desensitization treatment:

F (2,36) = 4.99, p = 0.012

F (2,36) = 4.99, p < 0.05 (with Bonferroni correction)

Simple effect of degree of phobia for participants who received

implosion treatment:

F (2,36) = 34.94, p < .001

F (2,36) = 34.94, p < 0.05 (with Bonferroni correction)

Simple effect of degree of phobia for participants who received

insight treatment:

F (2,36) = 8.09, p = .001

F (2,36) = 8.09, p < 0.05 (with Bonferroni correction)

8-38 2006 A. Karpinski

 o We have found significant simple effects of degree of phobia for
participants who received desensitization, implosion or the insight
treatments. These are omnibus tests, so we need to do Tukey post-hoc
tests (with = .0167 ) to identify the differences.
if (treat=1 and phobia=1 and gender=1) group = 1. ...
if (treat=1 and phobia=2 and gender=1) group = 2. if (treat=3 and phobia=1 and gender=2) group = 16.
if (treat=1 and phobia=3 and gender=1) group = 3. if (treat=3 and phobia=2 and gender=2) group = 17.
... if (treat=3 and phobia=3 and gender=2) group = 18.

ONEWAY dv by group
/CONT = -1 1 0 0 0 0 0 0 0 -1 1 0 0 0 0 0 0 0
/CONT = -1 0 1 0 0 0 0 0 0 -1 0 1 0 0 0 0 0 0
/CONT = 0 -1 1 0 0 0 0 0 0 0 -1 1 0 0 0 0 0 0
/CONT = 0 0 0 -1 1 0 0 0 0 0 0 0 -1 1 0 0 0 0
/CONT = 0 0 0 -1 0 1 0 0 0 0 0 0 -1 0 1 0 0 0
/CONT = 0 0 0 0 -1 1 0 0 0 0 0 0 0 -1 1 0 0 0
/CONT = 0 0 0 0 0 0 -1 1 0 0 0 0 0 0 0 -1 1 0
/CONT = 0 0 0 0 0 0 -1 0 1 0 0 0 0 0 0 -1 0 1
/CONT = 0 0 0 0 0 0 0 -1 1 0 0 0 0 0 0 0 -1 1
(Note: Ignore Significance levels)

Contrast Tests

Value of Sig.
Contrast Contrast Std. Error t df (2-tailed)
DV 1 -4.0000 1.75682 -2.277 36 .029
2 -5.3333 1.75682 -3.036* 36 .004
3 -1.3333 1.75682 -.759 36 .453
4 -6.6667 1.75682 -3.795* 36 .001
5 -14.6667 1.75682 -8.348* 36 .000
6 -8.0000 1.75682 -4.554* 36 .000
7 -4.3333 1.75682 -2.467 36 .019
8 -7.0000 1.75682 -3.984* 36 .000
9 -2.6667 1.75682 -1.518 36 .138

For Tukeys HSD following simple effects: q(1-,r,)

Where = Familywise error rate
r = Number of groups in the comparisons
= DFw = N-abc
q(.9833,3,36) = 4.11

4.11
Compare tobserved to tcritical = = 2.91
2

8-39 2006 A. Karpinski

 o We end up with the following description of an interaction:
There is a simple effect of degree of phobia for participants who
received desensitization and for insight treatment. Tukey post-hoc
tests revealed that treatment is significantly better for mild cases than
severe cases.
There is a simple effect of degree of phobia for participants who
received implosion treatment. Tukey post-hoc tests revealed mild
phobic responded better than moderate phobic who responded better
than severe phobics (with all pairwise differences significant)

o Remember, we also could have decompose the treatment by phobia

interaction by examining the simple effect of treatment within each
degree of phobia
(But this analysis is left as an exercise for the reader)

Treatment of Phobia:
Treatment by Phobia Interaction

16

14
Mild
12 Moderate
Severe
10

8
Desensitization Implosion Insight

o However, notice how much easier these results would have been to
explain had the treatment by phobia interaction not been significant! (We
would be left with three main effects!)
o The moral of the story is that you should not just add extra factors just
to see what might happen. You want to design as concise a study as
possible while still testing your hypotheses.

8-40 2006 A. Karpinski