Power and Sample Size
Power and Sample Size
Lecture Topics
Re-visit concept of statistical power Factors influencing power Sample size determination when comparing two means Sample size determination when comparing two proportions
Example
Consider the following results from a study done on 29 women, all 3539 years-old
Sample Data n OC users Non-OC users 8 21 Mean SBP 132.8 127.4 SD of SBP 15.3 18.2
Section A
Power and Its Influences
Example
Example
Of particular interest is whether OC use is associated with higher blood pressure Statistically speaking we are interested in testing: - Ho: OC = NO OC Ho: OC - NO OC = 0 - HA: OC NO OC HA: OC - NO OC 0 Here OC represents (population) mean SBP for OC users, NO OC (population) mean BP for women not using OC
Sample Data n OC users Non-OC users 8 21 Mean SBP 132.8 127.4 SD of SBP 15.3 18.2
Example
Power
The sample mean difference in blood pressures is 132.8 127.4 = 5.4 This could be considered scientifically significant, however, the result is not statistically significant (or even close to it!) at the = .05 level Suppose, as a researcher, you were concerned about detecting a population difference of this magnitude if it truly existed This particular study of 29 women has low power to detect a difference of such magnitude
TRUTH
Ho Type I error Reject Ho Not HA Power 1-beta Type II error beta
alpha-level
Reject Ho
Power
Power
TRUTH
Ho Type I error Reject Ho Not Reject Ho alpha-level HA Power 1-beta Type II error beta Reject Ho Not Reject Ho Ho
TRUTH
HA Power 1-beta Type II error beta Type I error alpha-level
10
Power
Power
Power is a measure of doing the right thing when HA is true! Higher power is better (the closer the power is to 1.0 or 100%) We can calculate power for a given study if we specify a specific HA This OC/blood pressure study has power of .13 to detect a difference in blood pressure of 5.4 or more, if this difference truly exists in the population of women 35-39 years old! When power is this low, it is difficult to determine whether there is no statistical difference in population means or we just could not detect it
Recall, the sampling behavior of is normally distributed (large samples) with this sampling distributed centered at true mean difference - If Ho truth, then curve is centered at 0
11
12
Power
Power
Recall, the sampling behavior of is normally distributed (large samples) with this sampling distributed centered at true mean difference - If HA truth, then curve is centered at some value d, d 0
Ho will be rejected (for = .05) if the sample result, , is more than 2 standard errors away from 0, either above or below
13
14
Power
Power
Ho will be rejected (for = .05) if the sample result, , is more than 2 standard errors away from 0, either above or below
Ho will be rejected (for = .05) if the sample result, than 2 standard errors away from 0, either above or below
, is more
15
16
Power
Ho will be rejected (for = .05) if the sample result, , is more than 2 standard errors away from 0, either above or below
In order to increase power for a study comparing two group means, we need to do the following: 1. Change the estimates of 1 and 2 so that the difference between the two (1 - 2) is bigger
17
18
In order to increase power for a study comparing two group means, we need to do the following: 1. Change the estimates of 1 and 2 so that the difference between the two (1 - 2) is bigger
In order to increase power for a study comparing two group means, we need to do the following: 2. Increase the sample size in each group
19
20
In order to increase power for a study comparing two group means, we need to do the following: 2. Increase the sample size in each group
In order to increase power for a study comparing two group means, we need to do the following: 3. Increase the -level of the hypothesis test (functionally speaking, make it easier to reject) here, with = .05:
21
22
In order to increase power for a study comparing two group means, we need to do the following: 3. Increase the -level of the hypothesis test (functionally speaking, make it easier to reject) here, with = .10:
Power can be computed after a study is completed - Can only be computed for specific HAs (i.e., this study had XX% to detect a difference in population means of YY or greater) - Sometimes presented as an excuse for non statistically significant finding (the lack of a statistically significant association between A and B could be because of low power [< 15%] to detect a mean difference of YY or greater between) - Can also be presented to corroborate with a non-statistically significant result Many times, in study design, a required sample size is computed to actually achieve a certain preset power level to find a clinically /scientifically minimal important difference in means - In the next section we will show how to do this using Stata - Industry standard for power: 80% (or greater)
24
23