Experimental Design
Experimental Design
An experiment is a test or a series of tests Experiments are used widely in the engineering world
Process characterization & optimization Evaluation of material properties Product design & development Component & system tolerance determination
All experiments are designed experiments, some are poorly designed, some are well-designed
2
Reduce time to design/develop new products & processes Improve performance of existing processes Improve reliability and performance of products Achieve product & process robustness Evaluation of materials, design alternatives, setting component & system tolerances, etc.
According to the Principle of Replication, the experiment should be repeated more than once.By doing so the statistical accuracy of the experiment is increased.
The Principle of Randomization provides protection, when we conduct an experiment, against the effects of extraneous factors by randomization. We should design or plan the experiment in such a way that the variations caused by extraneous factors can all be combined under the general heading of chance. Through the application of the Principle of randomization, we can have a better estimate of the experimental error. 6
The extraneous factor, the known source of variability is made to vary deliberately and this needs to be done in such a way that the variability it causes can be measured and hence eliminated from the experimental error.
7
We divide the field into several homogeneous parts, known as blocks, and then each such block is divided into parts equal to the number of treatments. Then the treatments are randomly assigned to these parts of a block. Through the principle of local control we can eliminate the variability due to extraneous factor(s) from the experimental error.
8
Hybrid Design
Covariance
9
The population is defined and then from the population a sample is selected randomly After being selected randomly from the population, be randomly assigned to the experimental and control groups Thus, this design yields two groups as representatives of the population
10
The two groups (experimental and control groups) of such a design are given different treatments of the independent variable. Advantage:-It is simple and randomizes the differences among the sample items. Disadvantage: -The individual differences among those conducting the treatments are not eliminated. It doesnt control the extraneous variable and as such the result of the experiment may not depict the 11 correct picture.
In the RB. design, subjects are first divided into groups, known as blocks, such that within each group the subjects are relatively homogeneous in respect to some selected variable . The variable selected for grouping the subjects is one that is believed to be related to the measures to be obtained in respect of the dependent variable
12
3.2 Cont.
The variable selected for grouping the subjects is one that is believed to be related to the measures to be obtained in respect of the dependent variable. The number of subjects in a given block would be equal to the number of treatments and one subject in each block would be randomly assigned to each treatment .
13
3.2 Cont.
In general, blocks are the levels at which we hold the extraneous factor fixed, so that its contribution to the total variability of data can be measured . The main feature of the RB design is that in this each treatment appears the same number of times in each block. The R.B
14
3.2 Cont.
The R.B. design is analyzed by the twoway analysis of variance (two-way' ANOV A)" technique. Example ; Suppose four different forms of a standardized test in statistics were given to each of five students (selected one from each of the five I.Q. blocks) and following are the scores which they obtained.
15
82 90
67 68
Form 3
Form 4
86
93
73
77
51
60
69
65
84
71
16
Example Cont.
If each student separately randomized the order in which he or she took the four tests (by using-random numbers or some similar device), we refer to the design of this experiment as a R.B. design . The purpose of this randomization is to take care of such possible extraneous factors (say as fatigue) or perhaps the experience gained from repeatedly taking the test.
17
Factorial designs are used in experiments where the effects of varying more than one factor are to be determined. They are specially important in several economic and social phenomena where usually a large number of factors affect a particular problem
Factorial designs can be of two types; (I) simple factorial designs and (2) complex factorial designs.
18
19
Control variable
Treatment A
Treatment B
Level I
Cell 1 Cell 2
Cell 3 Cell 4
20
Level II
STUDY I DATA
Training
Treatment A Control (Intelligence) Level I (Low) Level II (High) Column mean 15.5 35.8 25.6 Treatment B 23.3 30.2 26.7 19.4 33.0
Row mean
21
STUDY II DATA
Training
Treatment A Control (Intelligence) Level I (Low) Level II (High) Column mean 10.4 30.6 20.5 Treatment B 20.6 40.4 30.5 15.5 35.5
Row mean
22
study I
Mean scores of dependent variables (Say ability)
A B
study I I
Mean scores of dependent variables (Say ability)
A B
The graph relating to Study I indicates that there is an interaction between the treatment and the level, means that the treatment and the level are not independent of each other. The graph relating to Study II shows that there is no interaction effect which means that treatment and level in this study are relatively independent of each other.
25
This will usually include four treatments of the experimental variable and three levels of control variable.
26
Level I
Level II
Cell 1
Cell 2
Cell 4
Cell 5
Cell 7
Cell 8
Cell 10
Cell 11
Level III
Cell3
Cell6
Cell 9
Cell 12
27
28
Treatment B
Control Variable 2 Level I Control Variable 2 Level II
Level I
Cell 1
Cell3 Cell4
Cell5 Cell 6
Cell 7 Cell 8
29
control variable1
Level II Cell 2
3.4. Hybrid experimental designs Just what the name implies, it consists of new strains that are formed by combining features of more established designs. They are basically design to or constructed to address specific threats to internal validity. Basically there are of two types (i) Solomon Four Group Design and (ii) Switching Replications Design
31
(i) The Solomon Four Group Design It is designed to deal with a potential testing threat (Testing threat occurs when the act of taking a test affects how people scores on a pretest or protest)
Usually has four groups, two of the groups receive the treatment and two do not, further, two of the groups receive a pretest and two do not ( a hybrid of 2x2 factorial design)
32
(i) The Solomon Four Group Design It is designed to deal with a potential testing threat (Testing threat occurs when the act of taking a test affects how people scores on a pretest or protest)
Usually has four groups, two of the groups receive the treatment and two do not, further, two of the groups receive a pretest and two do not ( a hybrid of 2x2 factorial design)
33
(The Solomon Four Group Design Contd----) Within each treatment condition we have a group that is pretested and one that is not. By explicitly including test as a factor in the design, we are able to assess experimentally weather a testing threat is operating.
34
Possible Outcomes:
35
The graph shows the connection between the preset and posttest average for the same group and a line is used to connect the dots The two dots that are not connected by a line represent the two post only groups. On the posttest both treatment groups outscore both controls But when we look at posttest values there appears to be no difference between the treatment groups, even though one got a pretest and the other did not.
36
Similarly, the two control groups scored about the same on the posttest. Thus the pretest did not appear to affect the outcome. There is a main effect for the treatment.
37
each treatment group outscored its comparable control group In graph (ii) there evidence of a testing threat. This result indicates that there is a treatment effect.
38
But here both groups that had the pretest outscored their comparable non-pretest group.
That is evidence for a testing threat. (ii) Switching Replication Design Is one of the strongest of the experimental designs. And when the circumstances are right for this design, it addresses one of major problems in experimental designs i.e. the need to deny the program to some participants through random assignment
39
It can be thought of this as two pre-post treatment-control designs grafted together. That is, the implementation of the treatment is repeated or replicated And in the repetition of the treatment, the two groups switch roles i.e. the original group becomes the treatment group in phase 2 while the original treatment acts as a control. By the end of the study all participants have received the treatment.
40
The switching replication design is most feasible in organizational contexts where programs are repeated at regular intervals. For instance it works especially well in schools that are in semester system. All students are pre-tested at the beginning of the school year. During the first semester, Group 1 receives the treatment and during the second semester Group2 gets it.
41
The designs also enhance organizational efficiency in resources allocation. Schools only needed to allocate enough resources to give the program to half of the students at a time.
Possible Outcomes.
(i) Short-term persistent treatment effect When the program is given to the first group, the recipients do better than the controls. In the second phase, when the program is given to the original controls, they "catch up" to the original program group. Thus, we have a converge, diverge, reconverge 42 outcome pattern.
43
44
Both the Solomon Four-Group and the Switching Replications designs addressed specific threats to internal validity.
Remember that in randomized experiments, especially when the groups are aware of each other, there is the potential for social threats i.e. compensatory rivalry, compensatory equalization and resentful demoralization are all likely to be present in educational contexts where programs are given to some students and not to others.
45
The switching replications design helps mitigate these threats because it assures that everyone will eventually get the program. And, it allocates who gets the program first in the fairest possible manner, through the lottery of random assignment.
46
Also referred as Noise Reduction The pre-program measure or pretest is sometimes also called a "covariate" because of the way it's used in the data analysis -- we "covary" it with the outcome variable or posttest in order to remove variability or noise. Covariates are the variables you "adjust for" in your study.
47
Procedures:
The data are collected under a completely randomized design Plot Y vs X for each group separately to see if there are any points that dont appear to follow the straight line. The relationship between Y and X must be linear for each group. Check this assumption by looking at the individual plots of Y vs X for each group.
48
The variance must be equal for both groups around their respective regression lines. Check that the spread of the points is equal around the range of X and that the spread is comparable between the two groups.
Plot Y vs X for each group separately to see if there are any points that dont appear to follow the straight line.
The residuals must be normally distributed around the regression line for each group.
49
If there is evidence that the individual regression lines are not parallel, then a separate regression line must be fit for each group for prediction purposes.
50
51
If there is no evidence of non-parallelism, then the next task is to see if the lines are co-incident, i.e. have both the same intercept and the same slope.
If there is evidence that the lines are not coincident, then a series of parallel lines are fit to the data. All of the data are used to estimate the common slope.
52
53
If there is no evidence that the lines are not coincident, then all of the data can be simply pooled together and a single regression line fit for all of the data.
54
55
56