19

Measuring the
Effectiveness of the
Promotional Program

McGraw-Hill/Irwin Copyright © 2009 by The McGraw-Hill Companies, Inc. All rights reserved.
Pros and Cons of Measuring Effectiveness

Advantages Disadvantages

Avoid
Avoid costly
costly mistakes
mistakes Cost
Cost of
of measurement
measurement

Evaluate
Evaluate alternative
alternative Research
Research problems
problems
strategies
strategies

Increase
Increase efficiency
efficiency in
in Disagreement
Disagreement on on
general
general what
what to
to test
test

Determine
Determine ifif objectives
objectives are
are Objections
Objections of
of creatives
creatives
achieved
achieved

Time
Time
Evaluating Alternative Strategies
Measuring Effectiveness: Marketers’ Views

Issue Agree
MPM is a key priority for today’s 90%
technology companies.

I am dissatisfied with our ability to 80%

demonstrate marketing programs’
business impact and value.
It is important to define, measure, and 61.5%
take concrete steps in the area of
advertising accountability.
I can forecast the impact on sales of a 37%
10% cut in marketing spending.
 Measuring Advertising Effectiveness

What
What to
to test
test Where
Where to
to test
test
•• Source •• Laboratory
Laboratory tests
tests
Source factors
factors
•• Message •• Field
Field tests
tests
Message variables
variables
•• Media
Media strategies
strategies
•• Budget
Budget decisions
decisions

How
How to
to test
test When
When to
to test
test
•• Testing
Testing guidelines
guidelines •• Pretesting
Pretesting
•• Appropriate
Appropriate tests
tests •• Posttesting
Posttesting
 Pretesting Methods

Laboratory Field
Consumer Juries Dummy Ad Vehicles

Portfolio Tests On-air Tests

Physiological Measures

Theater Tests

Rough Tests

Concept Tests

Readability Tests
Comprehension and
Reaction Tests
 Posttesting Methods

Recall
Recall Tests
Tests

Tracking
Tracking Association
Association
Studies
Studies Measures
Measures

Methods
Methods
Single-
Single-
Recognition
Recognition Source
Source
Tests
Tests Systems
Systems

Inquiry
Inquiry Tests
Tests
Posttesting the Value of a Sponsorship
 Test Your Knowledge

Laboratory methods of testing for advertisement

effectiveness generally offer high control but low
_____.
A) realism
B) external validity
C) costs
D) generalization
E) value
Where to Test

In the Field 
 In the Lab
 Positioning Advertising Copy Test (PACT)

1. Provide measurements relevant to objectives of advertising

2. Require agreement on how results will be used

3. Provide multiple measures

4. Be based on a model of human response to communications

5. Consider multiple versus single exposure to the stimulus

6. Require alternative executions to have same degree of finish

7. Provide controls to avoid biasing effects of exposure context

8. Take into account basic considerations of sample definition

9. Demonstrate reliability and validity

 Test Points

1.Concept
1.Concept Testing
Testing

2.Rough
2.Rough Testing
Testing
Occurs
Occurs at
at
Various
Various Stages
Stages 3.Finished
3.Finished art
art or
or
commercial
commercial pretesting
pretesting

4.Market
4.Market testing
testing
(posttesting)
(posttesting)
 Concept Testing

Explores consumers’ responses to ad

Objective
Objective concepts expressed in words, pictures, or
symbols

Alternatives are exposed to consumers who

match the target audience

Method
Method Reactions & evaluations sought through focus
groups, direct questioning, surveys, etc.

Sample sizes depend on the number of

concepts and the consensus of responses

Qualitative and/or quantitative data

Output
Output evaluating and comparing alternative
concepts
Rough Art, Copy, and Commercial Testing

Comprehension and Reaction Tests

Consumer Juries

Advantages Disadvantages
Consumer may become a
Control
self-appointed expert
Number of ads that can be
Cost effectiveness evaluated is limited
Endorsements by A halo effect is possible
independent third parties
Preference for ad types
Achievement of credibility may overshadow objectivity
 Rough Testing Terms

Animatic
Animatic Rough
Rough

Terms
Terms Photomatic
Photomatic Rough
Rough

Live-action
Live-action Rough
Rough
An Animatic Example from Airwalk

*Click outside of the video screen to advance to the next slide

Spot Resulting from Animatic Rough

*Click outside of the video screen to advance to the next slide

 Pretesting Finished Print Ads

A laboratory method

Portfolio
Portfolio Includes test and control ads
Tests
Tests
Portfolio test have problems

Based on syllables per 100 words

Readability
Readability
Tests
Tests
Other factors also considered

Dummy
Dummy Distributed to random sample homes
Advertising
Advertising
Vehicles
Vehicles Product interest may still bias results
 Test Your Knowledge

_____ is a method of testing ads by placing them

randomly in certain copies of regularly distributed
magazines.
A) Vehicle source testing
B) Burke's reflection test
C) A Flesch test
D) Dummy advertising vehicle testing
E) A contextual test
 Pretesting Finished Broadcast Ads

Theater Tests On-Air Tests

• Measures changes in • Insertion in TV programs

product preferences in specific markets
• May also measure . . . • Limitations are imposed
• Interest in and reaction by “day-after recall”
to the commercial • Physiological Measures
• Reaction from an
adjective checklist
• Recall of various
aspects included
• Interest in the brand
presented
• Continuous reactions
 Physiological Measures

Galvanic
Galvanic skin
skin
Pupil
Pupil dilation
dilation response
response

Testing
Testing

Brain
Brain waves
waves Eye
Eye tracking
tracking
 Market Testing Print Ads

Recognition
Recognition
Inquiry
Inquiry Tests
Tests Tests
Tests

Testing
Testing

Tracking
Tracking Studies
Studies Recall
Recall Tests
Tests
Starch-Scored Sports Illustrated Ad
 Posttests of Broadcast Commercials

Day after recall Persuasive

tests measures

Tracking
Diagnostics
studies
Testing

Single-source Comprehensive
tracking measures

Test marketing
Comprehensive Testing by Ipsos-ASI
Problems With Current Research Methods
 Essentials of Effective Testing

Use
Use aa consumer
consumer
response
response model
model

Establish
Establish Use
Use pretests
pretests and
and
communications
communications posttests
posttests
objectives
objectives Testing
Testing

Understand
Understand and
and Use
Use multiple
multiple
implement
implement measures
measures
proper
proper research
research
 Test Your Knowledge

Good tests of advertising effectiveness must address

the nine principles established by PACT. One of the
easiest ways to do this is to follow a decision sequence
model. The first step in the model is to:
A) Understand the appropriate research
B) Create a model that uses multiple
measures
C) Establish communication objectives
D) Decide whether to use posttests or
pretests
E) Develop a consumer response model
Measuring Effectiveness of Other Programs

Sales
Sales
promotions
promotions
Shopping
Shopping cart
cart signage
signage

Ski
Ski resort-based
resort-based media
media
Non-
Non-
traditional
traditional
media In-store
In-store radio
radio and
and television
television
media
Other
Other media
media

Exposure
Exposure methods
methods
Sponsor-
Sponsor-
ships
ships Tracking
Tracking measures
measures
Measuring Effectiveness + Efficiency