Research Methods Lecture 2

Non-experimental and Experimental Research Approaches Chapters 2 & 3

Research Designs/Approaches
Type Purpose Time frame Degree of control
High

Examples

Experimental

Test for cause/ current effect relationships Test for cause/ Current or past effect relationships without full control

Quasiexperimental

Comparing two types of treatments for anxiety. Moderate Gender to high differences in visual/spatial abilities

Research Designs/Approaches
Type Purpose Time frame
Current (crosssectional) or past Past & current

Degree of control
Low to medium

Examples

Nonexperime ntal correlational Ex post facto

Examine relationship between two variables Examine the effect of past event on current functioning.

Low to medium

Relationship between studying style and grade point average. Relationship between history of child abuse & depression.

Research Designs/Approaches
Type Purpose Time frame Degree of control
Low to moderate

Examples

Nonexperime ntal correlational Cohortsequential

Examine relat. Future betw. 2 var. predictive where 1 is measured later. Examine Future change in a var. over time in overlapping groups.

Low to moderate

Relat. betw. history of depression & development of cancer. How motherchild negativity changed over adolescence.

Research Designs/Approaches
Type Purpose Time frame
Current

Degree of control
None or low

Examples

Survey

Qualitative

Assess opinions or characteristics that exist at a given time. Discover potential relationships; descriptive.

Voting preferences before an election. Peoples experiences of quitting smoking.

Past or current

None or Low

Non-experimental Research Designs

Describes a particular situation or phenomenon. Hypothesis generating Can describe effect of implementing actions based on experimental research and help refine the implementation of these actions.

Correlational Design
Measure two variables
Study methods and grade-point average

Determine degree of relationship between them

Correlation coefficient (e.g., r = 0.50)

Allows description and prediction of the relationship

Correlational Studies

Type of descriptive research design

Advantage is that it can examine variables that

cannot be experimentally manipulated (e.g., IQ and occupational status). Disadvantage is that it cannot determine causality. Third variable may account for the association. Directionality unclear

Origins of the Correlation Coefficient

Childrens height 64 70 69 68 67 66 65 2 3 7 6 3 3 6 11 8 4 65 66 2 5 10 13 11 6 67 4 8 12 14 11 4 68 5 9 12 13 8 3 69 5 9 2 10 6 2

Correlation between parents height and childrens height

Correlation Scatterplot

Strong Positive Relationship

Correlation Scatterplot

Strong Negative Relationship

Correlational Designs

What are some correlational studies that you can do?

Ex Post Facto Study

Variable of interest is not subject to direct manipulation but must be chosen after the fact. E.g., Define two groups of people according to a certain characteristic (e.g., history of trauma) and measure how they respond in terms of anxiety to a certain stimulus (e.g., watching violent film). Limitation self-selection bias, cohort effects may explain the effect.

Personality and Hypertension, Effect of Hypertension Awareness

Hypertension Study - Screened 10,500 Employees
Matched Normotensive 1st BP Screen Hypertensive DBP

2nd BP Screen

2nd BP Screen 2-3 weeks later

Personality Study

Personality Study

3rd BP Screen 3 months

4th BP Screen 4 months

5th BP Screen 5 months Mean DBP >= 90 mmHg

Personality and Hypertension: Effect of Hypertension Awareness

Variable Group 1 Aware Hypertensive 75 Group 2 Normotensive 75 Group 3 Unaware Hypertensive 89 Group 4 Normotensive 89

% Male Age Mean* (SD) SBP/DBP Mean* (SD)

46.2 (9.2)
135.1/ 93.9 (9.2/5.1)

46.2 (8.2)
118.7/ 76.3 (11.5/5.5)

46.4 (8.3)
135.8/ 93.8 (8.2/3.4)

45.8 (8.0)
118.5/ 75.7 (10.3/4.8)

Personality and Hypertension: Effect of Hypertension Awareness

Variable Group 1 Aware Hypertensive 12.0 (5.3) Group 2 Normotensive 9.3 (5.3) Group 3 Unaware Hypertensive 9.7 (4.8) Group 4 Normotensive 9.5 (4.6)

Neuroticism Mean* (SD)

Type A Mean* (SD)

0.79 (8.5)

-3.0 (9.4)

-2.0 (9.4)

-2.6 (8.2)

* Group 1 > Group 2 & Group 3 (p < 0.01)

Personality and Hypertension: Effect of Hypertension Awareness

14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Neuroticism

Aware Hyper Normot Unaware Hyper Normot

Aware hypertensive > normotensive & unaware hypertensive, P < 0.001

Personality and Hypertension: Conclusion

Do hypertensives have a different personality than those with normal blood pressure?
No, because the unaware hypertensives did not differ from the normotensives.

Why did the aware and unaware hypertensives differ?

Possible explanations?

Personality and Hypertension: Conclusion

Awareness of hypertension status confounds assessment of the association between personality characteristics and hypertension.
Due to hypertension labeling effect; or Due to self-selection bias

Cross-Sectional Study Designs

Compares groups at one point in time

E.g., age groups, ethnic groups, disease groups.

Advantage is that it is an efficient way to identify possible group differences because you can study them at one point in time. Disadvantage is that you cannot rule out cohort effects.

Longitudinal Design

Gathers data on a factor (e.,g. confidence) over time. Advantage is that you can see the time course of the development or change in the variables
Confidence increasing with age.

Confidence increasing at a faster rate in the 30s than the

40s. Confidence decreasing in the 50s and 60s.

Disadvantage is it is costly and still subject to bias

Cohort-Sequential Design

Combines a bit of the cross-sectional design and longitudinal design

E.g., Different age groups are compared on a variable over

time.

Advantage very efficient and reduces some of the biases in the cross-sectional design since you can see the evolution of change over time. Disadvantage cannot rule out cohort bias or the problem of the unidentified third variable accounting for the change.

Naturalistic Observation

Aims to unobtrusively observe behaviour in the natural setting. Observing in the natural setting enables one to minimize or eliminate the problem of artificial behaviour in response to being studied (i.e., reactivity effects). One variation is being a participant observer (e.g., undercover agent).

Naturalistic Observation

Advantages
Observe the natural phenomena (not artificial)

Disadvantages
Observer bias Reactivity in subjects Ethics

Meta Analysis (Glass 1976)

Quantitative approach to integrate and describe results across a range of independent studies. Enables you to combine the probability (p) value for statistical tests over a number of studies. Enables you to determine the effect size of the independent variable (e.g., treatment group) across studies.

Survey Research

Collecting standarized information from people using an interview or self-report format. Typically survey knowledge or opinions. To standarized the information one uses a questionnaire with set questions. Ideally the questionnaire has been validated. Representativeness of the sample is very important.

Survey Methods

Interviews
Advantage - Comprehensive, ensure participant

understands the question, minimizes missing data, enables clarification of unclear responses
Disadvantage expensive, people more like to

refuse participation, can be risky for interviewer, interviewer may bias the responses.

Types of Survey Methods

Face-to-face interviews
Expensive and time-consuming

Telephone interviews
Need to use random-digit dialing to reach both

listed and unlisted numbers.

Mail
Return rate is usually low (20-30%).

Types of Questions

Open-ended
E.g., Can you tell me about your typical

experience with dating?

Close-ended
E.g., How do you typically meet someone to date? Introduced by someone Social event In university class or place of work At a bar Through sports or other athletic events

Sampling

Population is everyone in your population of interest. Sample is some proportion of the population. Haphazard sampling convenience sample Random sampling
There is always some degree of sampling error.

Qualitative Methods

Multimethod approach to studying people in their natural environment

It is interpretive researcher has to make sense

of the data Multimethod can use interviews, photographs, natural observation, archives, etc. It is typically conducted in persons natural environment.

Valuable to use when phenomenon not fully defined.

Qualitative Methods Limitations

Participants observations and accounts can be biased. For example, filtered by his/her style of expression, gender, social class, race, age, ethnicity, etc. People are seldom able to provide a true and full account of their experience.

Defensive Lack insight Unaware

Qualitative Methods

Transcripts

Experimental Designs

Examines differences between experimentally manipulated groups or variables (e.g., one group gets a certain drug and the other gets a placebo). At minimum, experimental (independent) variable has two levels (e.g., drug vs. placebo).
Advantage is that you can determine causality. Disadvantage is cost and many variables cannot

be experimentally manipulated (e.g., smoke exposure over time).

Method of Agreement

Experimental Designs Four Canons for Identifying Causality

Observe the element common to several

instances of the event Problem is you may inadvertently overlook a significant variable.

Method of Difference
Identify the different effects produced by two

situations that are alike in all ways but one. Fairly robust and strong method.

Experimental Designs Four Canons for Identifying Causality

Joint methods of agreement and difference
Observe the element common to several

instances of the event Form hypothesis based on observations Test hypothesis using method of difference

Method of Concomitant Variation

Identify the different effects produced by more

than two situations that are alike in all ways but one. E.g., Compare two active drugs to a placebo

Experimental Design

Because it is so difficult with human behaviour to demonstrate causation unequivocally, some argue that a theory or prediction can only achieve the status of not yet disconfirmed (Popper, 1968). Our scientific efforts are directed at finding the causal factors rather than the cause per se.

Psychological Experiment: Is Objective

Researcher strives for freedom from bias. Recognize that:

Mistakes can occur Carefully scrutinize all steps of the experiment to

identify where mistakes are likely. Take the steps necessary to minimize error.

Psychological Experiment: Focuses on a Phenomenon

This is a publicly observable behaviour.

Actions Appearances

Verbal statements
Responses to questionnaires Physiological responses.

Psychological Experiment: Is Done Under Strictly Controlled Conditions

Eliminate all factors that could influence the outcome other than the factor being manipulated. Control is needed to infer causation. All conditions are kept constant except one; the manipulated variable. The variable of interest is varied in order to test its effect.

Experimental Method

Advantages
Strength with which causal relationships can be

inferred. Ability to manipulate one or more variables. Proven to be a very useful and robust scientific method (i.e., withstood the test of time).

Experimental Method

Disadvantages
Tight controls often produce artificial

conditions that could limit the generalizability of the findings (i.e., internal vs. external validity trade-off). Time consuming. Expensive. Human behaviour is very complex and cannot be fully studied using experimental methods.

Experimental Method: Threats to Internal Validity

Learning or practice effects

Scores on a measure change on repeat testing

because participant has more familiarity with the measure and so answers more truthfully.

Natural history effects

Something happens in the social background

(e.g., society because more affluent generally) and this influences the participants responses.

Maturation
Natural developments in the participant account

for the changes (e.g., getting older).

Experimental Method: Threats to Internal Validity

Regression to the mean

High scores generally move down toward the

mean and low scores move up.

Instrumentation
If pre and post tests are not equivalent in all

ways (e.g., difficulty, readability) then differences observed may be due to instrumentation differences rather than due to your experimental manipulation.

Experimental Method: Threats to Internal Validity

Subject problems
Selection bias (e.g., participation rate). Attrition (e.g., only motivated subjects stay in

the experiment).

Experimental Method: Threats to External Validity

Subject variables
Selection bias.
Attrition bias

Artificial conditions
E.g., In order to measure a subjects blood

pressure in response to a well-fined stressor you bring him/her into the laboratory but his/her response in the laboratory may not reflect how his/her blood pressure would really respond under stress in his natural environment.

Let me know
If there are any topics from todays lecture that need fuller explanations. Anything you particularly liked about the lecture (todays or others as we go along). Anything you particularly disliked about the lecture (todays or others as we go along).