Experimental Research Designs

& Quasi Research Designs

Presented By: Nazia Qayyum
SAP ID:48541
Experimental Designs

 In social science, there are two main types of

experimental methods
 True-Experiment

 Field experiment

 When researchers have great control over conditions and random

assignment is applicable then we use true-experiment but when
these all conditions do not meet, we use field experiment
 Designs
 Group experimental designs include:
 pre-experimental designs (the one-shot
case study, the one-group pretest–posttest
design, and the static-group comparison),
 true experimental designs (the pretest–
posttest control group design, the
posttest-only control group design, and
the Solomon four-group design)
 quasi-experimental designs (the
nonequivalent control group design, the
time-series design, the counterbalanced
designs), and factorial designs.
Pre-experimental Research Design
EXPERIMENTAL RESEARCH
DESIGN
Definition and Purpose

 A method to test hypothesis and establish cause-effect

relationship.

 In experimental research the researcher manipulates at least one

independent variable, controls other relevant variables, and
observes the effect on one or more dependent variables.

 Experimental research is research conducted with a scientific

approach using two sets of variables. The first set acts as a
constant, which you use to measure the differences of the second
set
Definition and Purpose

 to make causal inferences about behavior and mental processes.

 True-experimental research is carried out in laboratory settings

under the controlled conditions.

 Thus true experimental methods work according to the third goal

of scientific method known as “Explanation” (about behavior and
mental processes).
LOGIC OF EXPERIMENTAL RESEARCH
 Researchers manipulate an independent variable in an experiment to observe the
effect on behavior, as assessed by the dependent variable.

 Experimental control is the essential ingredient; experimental control is gained

through manipulation, holding conditions constant, and balancing.

 An experiment has internal validity when it fulfills the three conditions required
for causal inference: covariation, time-order relationship, and elimination of
plausible alternative causes.

 When confounding occurs, a plausible alternative explanation for the observed

covariation exists, and therefore, the experiment lacks internal validity. Plausible
alternative explanations are ruled out by holding conditions constant and
balancing
Types of Experimental Designs
 Anexperiment typically involves a comparison of two
groups (although some experimental studies have only one
group or even three or more groups).
 The experimental comparison is usually one of three types:
 Comparison of two different approaches (A versus B),
 Comparison of a new approach and the existing approach (A
versus no A)
 Comparison of different amounts of a single approach (a little of
A versus a lot of A).
RANDOM GROUPS DESIGN

 In an independent groups design, each group of subjects

participates in only one condition of the independent variable.
 Random assignment to conditions is used to form comparable
groups by balancing or averaging subject characteristics
(individual differences) across the conditions of the independent
variable manipulation.
 When random assignment is used to form independent groups
for the levels of the independent variable, the experiment is
called a random groups design.
Independent Groups design

 In an Independent Groups design, each group of subjects

participates in a different condition of the independent variable.
 The most effective independent groups design is one that uses
random assignment of subjects to conditions in order to form
comparable groups prior to implementing the independent
variable.
 When random assignment to conditions is used, the independent
groups design is called a random groups design.
Logic of Design

 The logic of the design is straightforward. The groups are formed

so as to be similar on all important characteristics at the start of
the experiment. Next, in the experiment itself, the groups are
treated the same except for the level of the independent variable.
Thus, any difference between the groups on the dependent
variable must be caused by the independent variable.
Example
 Exposure to very thin body images causes young girls to experience
negative feelings about their own body for girls as young as 5 years old.
 Anthropological studies that compare the body proportions of Barbie to
actual women reveal that the Barbie doll has very unrealistic body
proportions, yet Barbie has become a sociocultural ideal for female
beauty.
 In the experiment small groups of young girls (51⁄2–61⁄2 years old) were
read a story about “Mira” as she went shopping for clothes and prepared
to go to a birthday party. As they heard the story, the girls looked at
picture books with six scenes related to the story. In one condition of the
experiment, the picture books had images of Barbie in the scenes of the
story. In a second condition the picture books had similar scenes but the
figure pictured was the “Emme” doll.
 Finally, in the third condition of the experiment the picture books did not
depict Barbie or Emme (or any body) but, instead, showed neutral images
related to the story .
 These three versions of the picture books (Barbie, Emme, neutral) represent
three levels of the independent variable that was manipulated in the
experiment. Because different groups of girls participated in each level of the
independent variable, the experiment is described as an independent groups
design.
Manipulation

 Using the control technique of manipulation, the first two

requirements for causal inference were met in this experiment:
(1) Differences in the girls’ body dissatisfaction covaried with the
conditions of the experiment
(2) (2) body dissatisfaction came after viewing the images
(timeorder relationship).
(3) The third requirement for causal inference, elimination of
alternative explanations, was accomplished in this experiment
through holding conditions constant and balancing.
Holding Conditions Constant

 All other factors affecting Dependent variable can be kept constant.

 Example:
Several factors that could have affected the girls’ attitudes toward their body
were kept the same across the three conditions like:
All of the girls heard the same story about shopping and attending a birthday
party, and they looked at their picture books for the same amount of time. They
all received the same instructions throughout the experiment and received the
exact same questionnaire at the conclusion.
 Researchers use holding conditions constant to make sure that the
independent variable is the only factor that differs systematically
across the groups. Holding conditions constant is a control
technique that researchers use to avoid confoundings.
 By holding constant the story the girls heard in the three
conditions.
 In general, a factor that is held constant cannot possibly covary
with the manipulated independent variable. More importantly, a
factor that is held constant does not change, so it cannot possibly
covary with the dependent variable either. Thus, researchers can
rule out factors that are held constant as potential causes for the
observed results.
 It is important to recognize, however, that we choose to control
only those factors we think might influence the behavior we are
studying—what we consider plausible alternative causes. For
instance, Dittmar et al. held constant the story the girls heard in
each condition. It is unlikely, however, that they controlled
factors such as the room temperature to be constant across the
conditions because room temperature probably would not likely
affect body image (at least when varying only a few degrees).
 Nevertheless, we should constantly remain alert to the possibility
that there may be confounding factors in our experiments whose
influence we had not anticipated or considered- What then???
Balancing

 One key to the logic of the experimental method is forming

comparable (similar) groups at the start of the experiment. The
participants in each group should be comparable in terms of
various characteristics such as their personality, intelligence, and
so forth (also known as individual differences). The control
technique of balancing is required because these factors often
cannot be held constant.
 The goal of random assignment is to establish equivalent groups
of participants by balancing, or averaging, individual differences
across the conditions.
 Balancing

 When we balance a factor such as body weight, we make the three groups
equivalent in terms of their average body weight.
 Note that this differs from holding body weight constant, which would require
that all of the girls in the study have the same body weight.
 Similarly, balancing the number of Barbie dolls owned by girls in the three
groups would mean that the average number of dolls owned in the three
groups is the same, not that the number of dolls owned by each girl is held
constant at some number.
 The beauty of random assignment is that all individual differences are
balanced, not just the ones we’ve mentioned. Therefore, we can rule out
alternative explanations due to any individual differences among participants
Block Randomization
 Block randomization balances subject characteristics and potential confoundings that
occur during the time in which the experiment is conducted, and it creates groups of
equal size.
 Suppose we have an experiment with five conditions (labeled, for convenience, as A, B,
C, D, and E).
 One “block” is made up of a random order of all five conditions:
 One block of conditions → Random order of conditions
ABCDE CAE B D
 In block randomization, we assign subjects to conditions one block at a time. In our
example with five conditions, five subjects would be needed to complete the first block
with one subject in each condition. The next five subjects would be assigned to one of
each of the five conditions to complete a second block, and so on. If we want to have
10 subjects in each of five conditions, then there would be 10 blocks in the block-
randomized schedule. Each block would consist of a random arrangement of the five
conditions. This procedure is illustrated below for the first 11 participants.
 block randomization produces groups that are of equal size.
 Second, block randomization controls for time-related variables.
Because experiments often take a substantial amount of time to
complete, some participants can be affected by events that occur
during the time the experiment is conducted. In block
randomization, every condition is tested in each block so these
time-related variables are balanced across the conditions of the
experiment.
 Block randomization also works to balance other time-related
variables, such as changes in experimenters or even changes in
the populations from which subjects are drawn. For example, a
perfectly acceptable experiment could be done drawing students
from both fall and spring semester classes if a block
randomization schedule is used.
Threats to Internal Validity

 Randomly assigning intact groups to different conditions of the

independent variable creates a potential confounding due to preexisting
differences among participants in the intact groups.

 Selective subject loss, but not mechanical subject loss, threatens the
internal validity of an experiment.

 Placebo control groups are used to control for the problem of demand
characteristics, and double-blind experiments control both demand
characteristics and experimenter effects
 Potential variables that are not directly of interest to the researcher but
that could still be sources of confounding in the experiment are called
extraneous variables.

 Mechanical subject loss occurs when a subject fails to complete the

experiment because of an equipment failure (in this case, the
experimenter is considered part of the equipment). Mechanical subject
loss can occur if a computer crashes, or if the experimenter reads the
wrong set of instructions, or if someone inadvertently interrupts an
experimental session.

 Selective subject loss o ccurs (1) when subjects are lost differentially
across the conditions of the experiment; (2) when some characteristic
of the subject is responsible for the loss; and (3) when this subject
characteristic is related to the dependent v ariable used to assess the
outcome of the study.
Matched Group Design

 A matched groups design may be used to create comparable groups

when there are too few subjects available for random assignment to
work effectively.

 Matching subjects on the dependent variable task is the best approach

for creating matched groups, but performance on any matching task
must correlate with the dependent variable task.

 After subjects are matched on the matching task, they should then be
randomly assigned to the conditions of the independent variable.
Natural Group Design

 Individual differences variables (or subject variables) are selected rather

than manipulated to form natural groups designs.

 The natural groups design represents a type of correlational research in

which researchers look for covariations between natural groups
variables and dependent variables.

 Causal inferences cannot be made regarding the effects of natural

groups variables because plausible alternative explanations for group
differences exist.
Introduction

 Sometimes it is more effective to have each subject participate in all

the conditions of an experiment.
 repeated measures designs or within-subjects designs
 In an independent groups design, a separate group serves as a
control for the group given the experimental treatment. In a repeated
measures design, subjects serve as their own controls because they
participate in both the experimental and control conditions.
 Within-subjects design and dependent-group design are the
alternative names for repeatedmeasures design.
When to use Repeated Measure Design
 conduct an experiment when few participants are available,
 conduct the experiment more efficiently,
 increase the sensitivity of the experiment, and
 study changes in participants’ behavior over time
 Advantage: generally more sensitive than an independent groups
design. The sensitivity of an experiment refers to the ability to
detect the effect of the independent variable even if the effect is a
small one.
 When the research question involves studying changes in
participants’ behavior over time, such as in a learning experiment,
a repeated measures design is needed. Further, whenever
Role of practice effects in repeated measures
designs
 Repeated measures designs cannot be confounded by individual differences
variables because the same individuals participate in each condition (level) of
the independent variable.
 Participants’ performance in repeated measures designs may change across
conditions simply because of repeated testing (not because of the independent
variable); these changes are called practice effects.
 Practice effects may threaten the internal validity of a repeated measures
experiment when the different conditions of the independent variable are
presented in the same order to all participants.
 There are two types of repeated measures designs (complete and incomplete)
that differ in the specific ways in which they control for practice effects.
Counter balancing
 The specific techniques for balancing practice effects differ for the
two repeated measures designs
 General term used to refer to these balancing techniques is
counterbalancing.
 In the complete design, practice effects are balanced for each
participant by administering the conditions to each participant
several times, using different orders each time. Each participant can
thus be considered a “complete” experiment.
 In the incomplete design, each condition is administered to each
participant only once. The order of administering the conditions is
varied across participants rather than for each participant, as is the
case in the complete design. Practice effects in the incomplete
design average out when the results are combined for all
participants.
Balancing Practice Effects in the Complete
Design
 Practice effects are balanced in complete designs within each
participant using block randomization or ABBA
counterbalancing.
 In block randomization, all of the conditions of the experiment
(a block) are randomly ordered each time they are presented.
 In ABBA counterbalancing, a random sequence of all
conditions is presented, followed by the opposite of the
sequence.
 Block randomization is preferred over ABBA counterbalancing
when practice effects are not linear, or when participants’
performance can be affected by anticipation effects.
Balancing Practice Effects in the Incomplete
Design
 Practice effects are balanced across subjects in the incomplete design
rather than for each subject, as in the complete design.
 The rule for balancing practice effects in the incomplete design is that
each condition of the experiment must be presented in each ordinal
position (first, second, etc.) equally often.
 The best method for balancing practice effects in the incomplete design
with four or fewer conditions is to use all possible orders of the
conditions.
 Two methods for selecting specific orders to use in an incomplete
design are the Latin Square and random starting order with rotation.
 Whether using all possible orders or selected orders, participants should
be randomly assigned to the different sequences
 Potential challenges
 Experimental studies in education often suffer from two problems: a lack of sufficient exposure
to treatments and failure to make the treatments substantially different from each other.
 An experiment is valid if results obtained are due only to the manipulated independent variable
and if they are generalizable to individuals or contexts beyond the experimental setting.
 These two criteria are referred to, respectively, as the internal validity and external validity of
an experiment.
 Threats to internal validity include history, maturation testing, instrumentation, statistical
regression, differential selection of participants, mortality, selection– maturation interactions
and other interactive effects.
 Threats to external validity include pretest–treatment interaction, multiple treatment
interference, selection–treatment interaction, specificity of variables, treatment diffusion,
experimenter effects, and reactive arrangements.
Example

 What are the differential effects of two problem-solving

instructional approaches (schema-based instruction and general
strategy instruction) on the mathematical word problem-solving
performance of 22 middle school students who had learning
disabilities or were at risk for mathematics failure?
Quasi Experimental Designs
Introduction

 Sometimes it is just not possible to assign individual participants

to groups randomly. For example, to receive permission to
include schoolchildren in a study, a researcher often has to agree
to keep existing classrooms intact. In other words, entire
classrooms, not individual students, are assigned to treatments.
 Quasi designs are to be used only when it is not feasible to use a
true experimental design.
 It lacks at least one of the three cardinal characteristics
(randomization, control)
Quasi experimental design

 Quasi experimental design is used on large population to manipulate the natural

conditions or to introduce the new program or invention to find its effects on population.
This method s used when study can not be conducted in laboratory and random
assignment is not possible.

 Quasi-experimental research design: The word “Quasi” indicates similarity. A quasi-

experimental design is similar to experimental, but it is not the same. The difference
between the two is the assignment of a control group. In this research, an independent
variable is manipulated, but the participants of a group are not randomly assigned. Quasi-
research is used in field settings where random assignment is either irrelevant or not
required.

 The Nonequivalent Control Group Design
 This design is very much like the pretest–posttest control group design.
 In nonequivalent control group design, two (or more) treatment groups are
pretested, administered a treatment, and post tested.
 The difference is that it involves random assignment of intact groups to
treatments, not random assignment of individuals.
 For example, suppose a school volunteered six intact classrooms for a study.
Three of six classrooms may be randomly assigned to the experimental group
( X1 ) and the remaining three assigned to the control group ( X2 ). The
inability to assign individuals to treatments randomly (as opposed to assigning
whole classes) adds validity threats such as regression and interactions between
selection, maturation, history, and testing.
 The Time-Series Design
 In the time-series design, one group is repeatedly pretested until
pretest scores are stable.
 The group is then exposed to a treatment and, after treatment
implementation, repeatedly post tested.
 If a group performs essentially the same on a number of pretests
and then significantly improves following a treatment, the
researcher can be more confident about the effectiveness of the
treatment than if just one pretest and one posttest were
administered.
The Time-Series Design

 History is a problem with the time-series design because some event

or activity may occur between the last pretest and the first posttest.
 Instrumentation may also be a problem, but only if the researcher
changes measuring instruments during the study.
 Pretest–treatment interaction is certainly a possibility; if one pretest
can interact with a treatment, more than one pretest can only make
matters worse. If instrumentation or pretest–treatment interaction
threatens validity, however, you will probably be aware of the
problem because scores will change prior to treatment.
Counterbalanced Designs

 In a counterbalanced design, all groups receive all treatments but in a different

order, and groups are post tested after each treatment.
 The only restriction is that the number of groups be equal to the number of
treatments.
 The order in which the groups receive the treatments is randomly determined.
 This design is usually employed with intact groups when administration of a
pretest is not possible, although participants may be pretested.
 A unique weakness of the counterbalanced design is potential multiple-
treatment interference that results when the same group receives more than
one treatment. Thus, a counterbalanced design should be used only when the
treatments are such that exposure to one will not affect the effectiveness of
another.
Examples
 Measurement of a learner’s performance in a college on weekly
basis and then introducing a new teaching technique. Then again
measuring on weekly basis.

 In a study of the effectiveness of a new management technique one

branch office is given a new TQM method to implement another
branch office retains the traditional management policy

At the end of the a year the branch with TQM has higher quality of
production than the one that retained the traditional management
technique.

Is this a “quasi-experiment”? Why?

Advantages
 Quasi experimental designs are more frequently used because they are
more practical and feasible to conduct research.
 Where the sample size is small, and where randomization & availability
of control group is not possible, this design is preferred.
 Quasi experimental design is more suitable for real natural world setting
than true experimental designs.
 This design allows the researchers to evaluate the impact of quasi
independent variables under naturally occurring conditions.
 In some cases hypotheses are practically answered through this design.
Disadvantages
 In this design there is no control over extraneous variables
influencing the dependent variable.

 The absence of a control group and absence of control over the

research setting makes the result of this design less reliable and
weak for the establishment of causal relationship between
independent & dependent variable.
THANK YOU